Experienced Azure Data Engineer with 4 years of hands-on expertise in building and maintaining ETL pipelines using Azure Databricks, Azure Data Factory, PySpark, and ADLS Gen 2. Proven ability to improve data reliability and performance in a product-based environment. Strong collaborator, skilled in supporting real-time analytics and scalable data solutions.
Overview
4
4
years of professional experience
Work History
Azure Data Engineer
One97 Communications Ltd (Paytm)
Pune
01.2023 - 03.2024
Designed and implemented scalable ETL workflows using Azure Databricks, Azure Data Factory to streamline data integration, migration, and processing for Paytm’s internal business platforms.
Ingested and processed multi-source data (transactional, behavioral, and logs) into Azure Data Lake Storage (ADLS) for centralized, cost-effective, and scalable storage.
Migrated legacy SQL Server workloads to Azure by configuring Azure Data Factory’s Copy Data Activity and Self-Hosted Integration Runtime, improving pipeline execution time, and platform reliability.
Utilized PySpark in Azure Databricks to perform advanced data wrangling, filtering, and enrichment, supporting downstream analytics, and ML pipelines.
Structured and curated refined datasets in Azure SQL Database, optimizing for performance, and seamless integration with Power BI dashboards and internal tools.
Collaborated with data analysts, ML engineers, and product teams to align data infrastructure with Paytm’s real-time decision-making and personalization needs.
Azure Data Engineer
Adonmo Pvt Ltd
Pune
11.2021 - 08.2022
Supported data migration activities by copying data from on-premises SQL Server and cloud sources to Azure Data Lake Storage (ADLS) using Copy Data Activity.
Performed basic data transformation and cleaning tasks using PySpark in Azure Databricks, focusing on null handling, formatting, and simple business rule applications.
Organized ingested data into structured zones within ADLS (Bronze, Silver, and Gold) to support better manageability.
Data Engineer
Vivish Technologies Pvt Ltd
Pune
12.2019 - 07.2020
Analyzed user requirements, designed and developed ETL processes to load enterprise data into the Data Warehouse.
Used PySpark in Azure Databricks for simple data processing operations, such as filtering, joining, and restructuring flat files.
Collaborated with senior data engineers and analysts to understand data requirements and participate in the testing of data pipelines.
Assistant General Manager - Ad Operations at One97 Communications Ltd. (Paytm)Assistant General Manager - Ad Operations at One97 Communications Ltd. (Paytm)
Key Account & Marketing Manager at Kuvos Tech (A DYNAMIK LABS GROUP OF COMPANY)Key Account & Marketing Manager at Kuvos Tech (A DYNAMIK LABS GROUP OF COMPANY)