Data Engineer with 4 years of experience building and optimizing large-scale data systems and pipelines. Skilled in Python, SQL, PySpark, and AWS. Led ETL and data warehousing initiatives, ensuring robust integration and data integrity. Collaborated cross-functionally to enhance data-driven decisions and maintain high-availability infrastructure.
Overview
4
4
years of professional experience
3
3
Certification
Work History
Senior Data Engineer
HashedIn By Deloitte, Grugram
03.2023 - Current
Spearheaded the migration and optimization of legacy ETL workflows to an event-driven architecture using AWS Lambda, reducing data processing time by 66% and enhancing efficiency in DynamoDB data loading
Successfully executed a high-volume data load of 1.6 million records into DynamoDB in under 9 minutes using AWS Glue, ensuring rapid data availability and seamless operational continuity
Automated and optimized data validation and reconciliation processes using AWS Glue and Lambda, resulting in a 40% improvement in data accuracy and a 35% reduction in data discrepancies
Integrated Amazon SNS for real-time alerts, reducing error detection and response times by 50%, leading to more proactive and efficient issue resolution
Generated optimized queries in Redshift to create insightful Tableau reports, leveraging data within the Glue catalog to support critical business decisions
Reduced ETL processing time by 30% and improved data query performance by 40% by optimizing an AWS-based data lake architecture using AWS Glue, Lambda, Amazon Redshift, and S3, leading to more efficient data workflows and faster decision-making capabilities.
Data Engineer
HashedIn By Deloitte, Grugram
02.2021 - 03.2023
Led the data migration process using Talend, successfully transferring complex trade data into a new data warehouse environment, ensuring seamless integration and minimal disruption to ongoing operations
Spearheaded the development of interactive Looker dashboards for EV charging analytics, greatly improving data accessibility and empowering stakeholders to make informed decisions
Engineered a streamlined ETL pipeline for tax data processing, integrating multiple data sources to enhance compliance and reduce processing time by 40%, supporting efficient tax reporting
Implemented a robust pipeline validation system to monitor and verify the operational performance of each ETL node, ensuring data integrity and adherence to strict functional specifications.
Education
Bachelor of Technology - Computer Science and Engineering
National Institute of Technology Uttarakhand
Skills
Technical Skills : AWS, GCP, Azure, Redshift, DynamoDB, Apache Spark, Hadoop, Data Modeling, Data Pipeline, NoSQL Databases
Data and Cloud: Programming & Tools: Python, SQL, Git, Docker
Visualization & ETL: Tableau, Looker, AWS Glue
Cloud Platform: AWS, GCP, Azure
Accomplishments
4 X Excellence Award: Awarded for exceptional data engineering and system optimization efforts
Super Squad: Received the Super Squad award for exceptional teamwork and contributions to project success
Languages
Hindi (Native proficiency)
English (Professional working proficiency)
Certification
AWS Cloud Certified Practitioner (CCP – C02)
GCP Associate Cloud Engineer (ACE)
DP-900: Microsoft Azure Data Fundamentals
Timeline
Senior Data Engineer
HashedIn By Deloitte, Grugram
03.2023 - Current
Data Engineer
HashedIn By Deloitte, Grugram
02.2021 - 03.2023
Bachelor of Technology - Computer Science and Engineering