Om Sharma
Associate Data Engineer
Ghaziabad
Summary
Detail-oriented Data Engineer with 1+ year of experience developing ETL pipelines using SQL, Python , Azure Data Factory, and Azure Databricks. Experienced in data transformation, data warehousing, Medallion Architecture, Delta Lake, and pipeline optimization. Strong analytical and problem-solving skills with a proven ability to collaborate across cross-functional teams in Agile environments.
Overview
1
1
year of professional experience
Work History
Associate Data Engineer
SHL
04.2025 - Current
- Worked extensively with SQL, Python, Py Spark, Azure Databricks , Delta Lake to build scalable ETL solutions and analytical datasets.
- Built incremental data ingestion pipelines using Change Data Capture (CDC) techniques to reduce processing time.
- Implemented Bronze, Silver, and Gold layers following the Medallion Architecture for scalable data processing.
- Created Delta Tables with ACID transactions and schema enforcement to improve data reliability.
- Worked with Parquet, JSON, and CSV datasets stored in Azure Data Lake Storage Gen2.
- Implemented SCD Type 1 and SCD Type 2 logic for maintaining historical dimensional data.
- Wrote optimized SQL queries, stored procedures, and views for data extraction.
- Designed analytical data models using Fact and Dimension tables with Star Schema architecture.
- Automated data orchestration using Azure Data Factory, including scheduling, dependency management, and monitoring.
- Enhanced data pipeline performance by optimizing Spark transformations, partitioning strategies, and SQL queries to improve processing efficiency and resource utilization.
- Performed data validation and reconciliation using SQL to ensure data completeness and consistency.
- Designed and maintained Power BI reports by integrating data from SQL-based data sources, creating interactive visualizations and performance metrics for business stakeholders.
- Utilized Power Query to perform data extraction, cleansing, transformation, and shaping from multiple data sources, ensuring accurate and analysis-ready datasets for reporting.
- Supported CI/CD deployment processes using Git-based version control, participating in code reviews and deployment activities to ensure reliable and efficient data pipeline releases.
- Participated in Agile development processes and collaborated with cross-functional teams to deliver data engineering solutions.
Education
Bachelor of Technology - CSE
DIT University
Dehradun, India 04.2001 -
Skills
Data pipeline design
ETL development
SQL
Python
Power BI
Azure Databricks
Azure Data Lake Storage (ADLS Gen2)
Data Warehouse
Star & Snowflake Schema
Slowly Changing Dimensions (SCD)
Medallion Architecture
Agile Scrum
Timeline
Associate Data Engineer
SHL
04.2025 - Current
Bachelor of Technology - CSE
DIT University
04.2001 -