Seasoned Data Engineer with 6 years of experience in big data applications, adept in Informatica ADF, IICS, Snowflake, Oracle SQL, Python and AWS S3. Proven analytical skills with a strong background in the automotive sector, particularly with Mercedes-Benz India. Skilled in data analytics, cloud solutions, BI dashboard development, and committed to driving business growth through technological innovation.
Overview
6
6
years of professional experience
1
1
Certification
Work History
Senior Consultant
Clover Infotech Pvt Ltd
05.2025 - Current
Responsible for end-to-end integration and delivery of HR data from Workday (source system) to approximately 9 external vendor systems.
The project ensures timely, secure, and accurate distribution of employee-related data for various downstream processes such as payroll, benefits, background verification, and compliance reporting.
Developed and maintained data ingestion pipelines using Workday APIs, designed Azure Data Factory (ADF) Data Flows, built scalable pipelines, automated data encryption and formatting, leveraged Azure Blob Storage and Key Vault, monitored pipeline runs, handled schema changes, configured MFT processes, automated file generation, coordinated with stakeholders, prepared documentation, and resolved data issues.
Data Engineer
Tata Technologies Ltd
01.2023 - 04.2025
Spearheaded the design and development of Azure Data Factory pipelines, orchestrating ETL workflows to efficiently process large-scale dealership data in the automobile industry.
Designed and implemented end-to-end ETL pipelines using PySpark and SQL to process large-scale datasets from multiple sources.
Implemented Azure Data Lake Storage for enhanced scalability and cost-effectiveness in data storage solutions.
Enhanced ADF pipeline performance through Data Flows, Parameterization, and Incremental Loading, significantly improving efficiency.
Configured Linked Services, Datasets, and Triggers within ADF to facilitate seamless data integration between on-premises and cloud environments.
Employed Azure Monitor, Log Analytics, and ADF Alerts to monitor, troubleshoot, and resolve pipeline failures.
Developed robust data ingestion pipelines from Azure Data Lake to Snowflake, leveraging ADF for streamlined data transfer.
Crafted Snowflake schema models, including Star and Snowflake Schemas, to optimize data retrieval and support effective reporting.
Utilized Snowflake's Time-Travel and Cloning features for in-depth historical data analysis and swift data recovery.
Applied Snowflake Performance Optimization strategies, such as clustering, materialized views, and partitioning, to enhance query execution.
Authored and refined complex SQL queries, incorporating CTEs and Window Functions, to achieve high-performance data retrieval.
Designed and fine-tuned Stored Procedures, Triggers, and Functions to automate data processing tasks.
Executed advanced SQL techniques for ETL transformations, ensuring the cleansing and structuring of raw dealership data for analytical purposes.
Associate
Amazon Development Centre India Pvt Ltd
08.2021 - 01.2023
Developed SQL scripts for efficient data migration to Snowflake database, ensuring data integrity and consistency.
Studied the Data distribution in each attribute and extracting the data by using Azure data bricks, Pyspark to perform the transformation.
Designed and implemented views within Snowflake to streamline access to Operational Data Store (ODS) information.
Automated the creation of derived tables utilizing a robust framework to enhance data processing workflows.
Engineered multiple schemas to facilitate the seamless transition of data from on-premises storage to cloud environments.
Constructed and managed Azure Data Factory (ADF) mappings and ETL jobs for extracting and loading data from the data lake to Azure SQL databases.
Analyzed data distribution across various attributes using Databricks, executing data extraction and transformation processes to support business intelligence.
Scheduled and monitored diverse workflows and tasks, ensuring operational efficiency and adherence to project timelines.
Associate Engineer
Softcode Techserv
09.2019 - 08.2021
Developed and executed SQL scripts for efficient data migration to Snowflake database, ensuring data integrity and consistency.
Designed and implemented views to streamline data access from Operational Data Store (ODS) to Snowflake, enhancing data retrieval processes.
Automated the creation of derived tables utilizing a robust framework to improve data management and reduce manual effort.
Scheduled and monitored PySpark jobs using orchestration tools of Azure Data Factory.
Constructed various database schemas to facilitate the migration of data from on-premises storage solutions to cloud environments, optimizing storage and accessibility.
Created Azure Data Factory (ADF) mappings and orchestrated ETL jobs to extract and load data efficiently from data lakes to Azure SQL databases.
Implemented Azure Stream Analytics jobs to capture event streams and transfer data to the data lake, ensuring real-time data availability.
Analyzed data distribution across different attributes using Databricks, extracting and transforming data to meet business requirements.
Scheduled and monitored diverse workflows and tasks, maintaining operational excellence and ensuring timely completion of data processes.