Shubham Sagar
Bengaluru
Summary
Senior Data Engineer with 7+ years of experience designing, building, and optimizing cloud-scale data platforms on Microsoft Azure. Proven expertise in Azure Data Factory, Databricks (PySpark), Synapse Analytics, ADLS, and Microsoft Fabric, delivering robust ETL/ELT pipelines across Supply Chain, Finance, and Digital Marketing domains.
Overview
7
7
years of professional experience
2
2
Certifications
Work History
Assistant Manager (Sr. Data Engineer)
KPMG India
Bengaluru
09.2021 - Current
- Designed and implemented scalable ETL/ELT pipelines to ingest and transform supply chain data from ERP systems, on-prem databases, APIs, flat files, and logistics platforms into Azure Data Lake and Azure Synapse Analytics.
- Collaborated with procurement, logistics, and operations teams to translate business requirements into efficient data engineering solutions.
- Architected and developed Azure data platforms using Azure Data Factory, Azure Databricks (PySpark, Spark SQL), Azure Data Lake Storage, Azure Synapse Analytics, Azure Cosmos DB, and Azure Machine Learning.
- Built and optimized ADF pipelines for data extraction, transformation, orchestration, and monitoring across multiple source systems.
- Developed Databricks notebooks using PySpark and Spark SQL for data cleansing, transformation, aggregation, and analytics.
- Created and optimized complex SQL queries and analytical metrics using Azure SQL and Azure Synapse Analytics.
- Implemented data validation, transformation logic, and quality checks using Spark DataFrames and Spark SQL.
- Integrated Azure Logic Apps for automated pipeline alerts, monitoring, and operational notifications.
- Delivered end-to-end analytics workflows with cascading pipeline dependencies using Azure Data Factory and Databricks.
- Supported production deployment, monitoring, debugging, and optimization of cloud data pipelines.
- Implemented security, monitoring, and audit mechanisms, including pipeline snapshots and alerting in Azure Synapse Analytics.
- Tools & Technologies: ADF, Azure Databricks, Spark Structured Streaming, ADLS, SQL, Python, Pandas, Microsoft Fabric, GIT
Software Engineer
Mindtree
Bengaluru
10.2018 - 09.2021
- Handled multiple ad-hoc analytics requests, tracking and analyzing customer and advertiser performance across the EMEA region.
- Automated recurring business reports (WBR, NQS) using Python and VBA, significantly reducing manual reporting effort.
- Delivered end-to-end automation of WBR and NQS reports, saving ~9 hours per week and receiving stakeholder recognition for timely delivery.
- Performed Exploratory Data Analysis (EDA) on data sourced from COSMOS Data Lake to support time-series forecasting.
- Analyzed advertiser, publisher, and partner data to identify performance gaps and recommend data-driven improvement strategies.
- Applied data analytics techniques to uncover growth opportunities and optimize account performance and conversion metrics.
- Designed and implemented ADF pipelines to ingest data from on-premises SQL Server to Azure SQL Database.
- Delivered end-to-end Azure data pipelines covering data capture, curation, and consumption.
- Ingested data into Azure Blob Storage and Azure Data Lake using Azure Data Factory from multiple source systems.
- Implemented parameterized and automated ADF pipelines to improve reusability, scalability, and operational efficiency.
- Tools & Technologies: ADF, Azure Databricks, Power BI, Excel, ADLS
Education
B.Tech/B.E. - Information Science and Engineering
Dr. Ambedkar Institute of Technology
Bengaluru07-2018
Skills
- Databricks
- ADF
- Azure Functions
- Fabric
- Azure Synapse Analytics
- Python / PySpark
- SQL / Spark SQL / T-SQL
- Power BI
Certification
- Azure Data Engineer Associate (DP-203 Microsoft)
- Azure Data Engineer Fundamentals (DP-900 Microsoft)
- Bing Ads Accredited Professional (Microsoft)
Timeline
Assistant Manager (Sr. Data Engineer)
KPMG India
09.2021 - Current
Software Engineer
Mindtree
10.2018 - 09.2021
B.Tech/B.E. - Information Science and Engineering
Dr. Ambedkar Institute of Technology