Pooja Patel
Pune
Summary
Data engineer with 9+ years of experience designing scalable ETL pipelines and real time data solutions using Python, Spark, Airflow and Kafka. Proven ability to build robust data infrastructure and collaborate cross functionally to deliver business critical insights.
Overview
5
5
years of professional experience
5
5
years of post-secondary education
Work History
Senior Data Engineer
Tech Mahindra Ltd
Pune
10.2021 - Current
- Designed, built, and maintained high-performance databases for reporting and analysis purposes.
- Performed advanced analytics on structured and unstructured data using SQL, Python, R.
- Implemented best practices around data governance and compliance standards such as GDPR and HIPAA.
- Implemented new database technologies such as NoSQL databases for efficient storage of large volumes of data.
- Developed data pipelines to enhance data integration processes.
- Implemented ETL processes using Apache Spark and Hadoop frameworks.
- Optimized database performance for large-scale data storage solutions.
- Designed and maintained data models for analytics and reporting purposes.
Data Engineer
KMC India Pvt Ltd
Pune
06.2021 - 10.2021
- Analyzed user requirements, designed and developed ETL processes to load enterprise data into the Data Warehouse.
- Designed and developed distributed systems leveraging Apache Kafka and other technologies.
- Integrated Kafka with Spark Streaming for real-time analytics on streaming data sets.
- Performed data ingestion from various sources like HDFS, S3 into Hadoop clusters using Flume, Kafka and other tools.
Data Engineer
Wipro Ltd
Pune
12.2020 - 05.2021
- Developed robust data pipelines for efficient data processing and storage.
- Implemented ETL processes to integrate diverse data sources into central databases.
- Documented technical specifications and workflows for future reference and training.
- Optimized existing queries to improve query performance by creating indexes on tables.
- Developed and implemented data models, database designs, data access and table maintenance codes.
- Created stored procedures for automating periodic tasks in SQL Server.
- Designed data warehousing solutions, applying dimensional modeling techniques for optimized data retrieval.
Education
PGDM - Data Science
Indian Institute of Information Technology
Bangalore 03.2020 - 02.2021
Bachelor of Engineering - Computer Science
RGPV UNIVERSITY
Bhopal 09.2012 - 06.2016
Skills
- Programming: Python, SQL, Shell scripting
- Big Data: Spark ( Pyspark, SparkSQL), Kafka, Hive
- Orchestration: Apache Airflow, GitHub Actions
- Cloud : AWS, S3, EMR, EC2, Athena, Dataproc
Timeline
Senior Data Engineer
Tech Mahindra Ltd
10.2021 - Current
Data Engineer
KMC India Pvt Ltd
06.2021 - 10.2021
Data Engineer
Wipro Ltd
12.2020 - 05.2021
PGDM - Data Science
Indian Institute of Information Technology
03.2020 - 02.2021
Bachelor of Engineering - Computer Science
RGPV UNIVERSITY
09.2012 - 06.2016