SAKSHI SHRIVASTAVA
Data Engineer
Gurgaon
Summary
Data Engineer with over 5+ years of experience in designing and implementing scalable data solutions. Expertise in Python, SQL, PySpark, Scala, Hadoop, and Hive, complemented by hands-on experience with Spark and Kafka technologies.
Overview
6
6
years of professional experience
Work History
Data Engineer
Bank of America
10.2022 - Current
- Developed and maintained scalable data pipelines utilizing Scala, Cloudera Hadoop, Apache Spark, and Kafka to process and analyze over 20 GB of data daily.
- Architected an advanced data profiling tool, incorporating 50+ metrics, resulting in a 30% boost in data monitoring efficiency.
- Optimized data storage and retrieval processes, improving query performance, and reducing storage costs by 20%.
- Worked on the projection component using ML methods to recommend approval requests, ensuring data correctness by 95%.
- Automated routine tasks with Python scripts, enhancing team productivity, and minimizing manual errors.
- Collaborated on ETL tasks, ensuring data integrity, and verifying pipeline stability.
Software Engineer
Capgemini
09.2020 - 10.2022
- Led the design and implementation of multiple end-to-end data warehousing projects in a big data environment.
- Ingested and extracted data from various sources into the AWS ecosystem, designed ingestion pipelines, and used Spark to process and load the data.
- Successfully built scalable, performant data pipelines, custom ETLs, and robust data lakes and warehouses.
- Designed and developed the ETL process, and wrote efficient Spark jobs to extract data from multiple sources and load it into the warehouse using Spark.
- Built and optimized data pipelines using Spark with Python on Hadoop and object storage.
- Worked with distributed file systems to handle large-scale data analytics and created compelling visualizations, like Power BI.
Education
Bachelor of Engineering - Information Technology
Oriental College of Technology
Bhopal, India09.2020
Skills
Big Data: Spark, Kafka, Hadoop, Hive
Timeline
Data Engineer
Bank of America
10.2022 - Current
Software Engineer
Capgemini
09.2020 - 10.2022
Bachelor of Engineering - Information Technology
Oriental College of Technology