PRERNA CHAUDHARY
Noida
Summary
Results-driven Data Engineer with 6.8 years of experience specializing in scalable ETL pipelines, cloud data migration, and big data solutions. Adept at leveraging AWS, Python, SQL, and Pyspark to design and deliver efficient data engineering workflows. Experienced in automating processes with Airflow, AWS Glue, and Apache Spark to optimize performance and reduce downtime. Proven ability to collaborate with cross-functional teams to extract actionable insights from complex datasets. Skilled in data warehouse design, data modeling, and data pipeline orchestration.
Overview
7
7
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
Thomson Reuters Corporation
03.2024 - Current
- Working on integrating new use-cases for tax determination system
- Using AWS EMR, lambda and step functions to run and monitoring the ETL pipelines smoothly
Technical Lead
Indegene
06.2024 - 01.2025
- Spearheaded the migration of on-premises data to AWS cloud using DMS, DynamoDB, EMR and AWS Glue.
- Automated complex workflows with AWS Lambda and Step Functions and streamlining data processing operations.
Senior AI Engineer
ZS Associates
09.2022 - 05.2024
- Designed and deployed decision engine pipelines leveraging SQL, Pyspark, and Airflow.
- Migrated data sources and optimized storage with Iceberg table setups to enhance performance.
- Enhanced ETL pipelines using Apache Kafka for real-time data ingestion.
Software Engineer (Big Data)
Airtel Africa Digital Labs
11.2021 - 08.2022
- Built and maintained ETL pipelines to process raw big data into actionable insights.
- Developed and tested explosion logic for handling large-scale datasets efficiently.
- Utilized Hadoop ecosystem to optimize data processing workflows.
Software Engineer
Tredence Analytics
06.2019 - 11.2021
- Engineered Iguazio data pipelines for real-time processing in the manufacturing domain.
- Led successful data migration projects for top-tier e-commerce clients, improving system scalability and operational efficiency.
- Implemented data modeling best practices to ensure accurate reporting.
Education
B.Tech (CSE) - Computer Science
IGDTUW
New Delhi05.2019
Skills
- Python
- SQL
- AWS (EMR, Glue, Lambda, step functions)
- Spark, pyspark
- Hive
- Git
- Airflow
- Dataiku
- Iguazio
- Azure
Certification
- Az-900 Microsoft Certified Azure Fundamentals July 2020 — Present
- Iguazio Certified Mlops Engineer January 2021 — Present
Timeline
Technical Lead
Indegene
06.2024 - 01.2025
Senior Data Engineer
Thomson Reuters Corporation
03.2024 - Current
Senior AI Engineer
ZS Associates
09.2022 - 05.2024
Software Engineer (Big Data)
Airtel Africa Digital Labs
11.2021 - 08.2022
Software Engineer
Tredence Analytics
06.2019 - 11.2021
B.Tech (CSE) - Computer Science
IGDTUW