Nayan

Realtime Outage System for Mobility and Broadband.

• Designed a near real-time outage alert system for network failures using Spark Streaming jobs that consume data from Solace and use Kafka to send millions of messages to multiple systems for raising real-time alerts to customers.

CDP Upgrade

• Leading, building plans, and execution strategy for data migration from HDP to CDP.
• NIFI migration from HDP to CDP for the workflows.

MyNetEX

• Building network experience KPIs for customers using CDR records.
• This data is used in building top cells, churn analysis, data usage, call drops, and serving network SRs.

Platform Development

• Build a data disposal tool for all the solutions using Scala and Spark.
• These include scheduled jobs for HDFS deletion, handling small files.
• Build an alerting and monitoring system for each job, including batch and streaming, on Grafana using NIFI and Spark.
• Build a Data Discovery tool for monitoring the source and target HDFS filesystems, and raise an alert on any deviation.
• Build an NIFI automated pipeline for the compaction of billions of small files.
• Build a custom NAR file for consuming data from Solace.
• Significant improvement in performance above consuming through the Spark Solace Connector.

• Being a part of the platform team, developed and maintained different ETL jobs so that the data pipeline should run efficiently
• Responsible For writing ETL jobs in Scala-Spark to transfer the data from the wdsl SOAP endpoint to Hbase
• Created Data Integrity check tool in Spark for Data Pipelines to ensure no data loss and good quality of HealthCare data
• Developed ADT(admit, discharge, transfer) Generators to process the data from elastic and transform according to the business requirement

• Enterprise Data Warehouse(Infosys)
• Responsible for developing TPump, fast load, Multi-load, TPT scripts to load Target tables from flat files
• Worked on Teradata Systems and analyse and improve query performance