Abhishek Kumar Singh
Data Engineer
Kanpur
Summary
1.9 Years of experience in Azure data engineering with expertise in designing and optimizing scalable data pipelines using Azure services, PySpark, and SQL. Proficient in ETL workflows, data warehousing, and data orchestration, focusing on Azure Databricks, Azure Data Factory, and Delta Lake. Skilled in transforming, cleansing, and tuning data for improved performance. Leverage PySpark, Azure Databricks, Azure Data Lake Storage, and Delta Lake to ensure efficient data movement and analysis.
Overview
2
2
years of professional experience
Work History
Data Engineer
Tranzita Systems
11.2023 - Current
- Designed and implemented end-to-end ETL pipelines, increasing data processing efficiency by 40%.
- Developed optimized PySpark scripts to process over 200 million rows daily, reducing execution time by 25%.
- Built custom Azure Data Factory pipelines for seamless data integration and movement across Azure Blob Storage, ADLS, and Delta Lake.
- Utilized Delta Lake for ACID transactions, schema enforcement, and versioning, ensuring data reliability and consistency.
- Automated data validation processes reduce manual data quality checks by 80%.
Education
BCA -
CSJM University
Kanpur, Uttar Pradesh06.2023
Intermediate - undefined
Takshshila International Education Centre
04.2020
Skills
Azure Data Factory
undefinedTimeline
Data Engineer
Tranzita Systems
11.2023 - Current
Intermediate - undefined
Takshshila International Education Centre
BCA -
CSJM University