Data Engineer with 2.6 Years of expertise in designing, building and optimizing scalable ETL pipelines,cloud-based solutions. Specializes in designing and optimizing data pipelines, proficient in ETL processes, Python, SQL, and PySpark.
Overview
3
3
years of professional experience
1
1
Certification
Work History
Data Engineer
Infosys
Hyderabad
02.2022 - Current
Developed and maintained ETL pipelines to extract, transform, and load data from various sources into centralized data warehouses, ensuring data quality integrity with the help of medallion architecture in Databricks.
Implemented and optimized data pipelines with the medallion architecture framework, leveraging its principles to ensure scalability, reliability, and performance.
Loaded data from various sources into the raw layer of the Assured Data Lake using Data Factory.
Transformed data using PySpark and Spark SQL, storing it in the publish layer.
Further transformed data to the staging layer for loading into Synapse Data Warehouse tables.
Loaded transformed data into Assured Synapse tables for Power BI consumption.
worked on Microsoft Fabric where designed and configured basic pipelines to ingest, transform and load data from various sources into target systems.
Education
Bachelor of Technology - Mechanical Engineering
Godavari Institute of Engineering And Technology
Rajamahendravaram
04-2019
Skills
SQL
Python
PySpark
Azure Microsoft Fabric
Azure Databricks
Certification
DP -900 Microsoft Azure Data Fundamentals certification.