Chirag Dhawan
Bengaluru
Summary
Currently working with Amazon and having over 6 years of experience. Worked on development of data pipelines using big data technologies for processing of high volume and high velocity data sets. Highly experienced in setting up data pipelines from scratch along with refactoring and debugging existing pipelines. Previously worked as a Software Engineer.
Overview
6
6
years of professional experience
Work History
Data Engineer II
Amazon
08.2023 - Current
- Lead the design of data pipelines being setup from scratch along with selection of services/infrastructure on AWS
- Using serverless stack like Lambda, SQS, SNS etc. for development of data pipelines
- Using AWS CDK for the setup of infrastructure
- Responsible for requirement gathering from Analytics team and enabling them with the right tools and services for development
Data Engineer
Amazon
08.2022 - 08.2023
- Responsible for development, maintenance and optimization of ingestion pipelines which serve data to analytics teams
- Responsible for setting up pipelines from ground up along with code deployment pipeline, scheduling, monitoring and execution environment for those pipelines
- Focused on improving unit test coverage for existing pipelines
- Technologies that I majorly used for development of these pipelines are - Python, Redshift, SQL, AWS(Glue, S3, etc.)
Data Engineer
G42 Healthcare
08.2021 - 07.2022
- Worked on identifying areas of improvements and optimizations in the in-house ETL application developed using Scala and Spark
- Using in-house ETL application, implemented data pipelines and worked on performance tuning of existing pipelines
- Technologies which I majorly used for development of pipelines are Python, Spark, Postgres and Docker
Sr. Business Technology Analyst
ZS
04.2018 - 08.2021
- Worked on development of pipelines for processing of high-volume patient level data sets using technologies like Spark on EMR clusters and orchestration of processes using Airflow
- Big data technologies that I worked on while developing such pipelines are – Spark SQL, Python, AWS(EC2, EMR, S3, Athena, Redshift, Airflow, etc.)
- Worked on back end of a dashboard which involved creation of data warehouse tables using Pandas (Python) library by consuming flat files from S3
- Worked on front and back end development of a web-based dashboard. Tech stack of the application was Angular JS(C3.js for visualizations) along with Flask and Pandas on back end
Software Engineer
Newgen Software
07.2017 - 03.2018
- Worked on development of banking systems which digitized processes such as account opening and processing of inward Swift messages
- Worked on development of data archival module for a bank which archived all documents after specific period of time and moved them from client's servers to a device called EMC Centera using Java
Education
Bachelor of Technology - Information Technology
Jaypee Institute of Information Technology
Noida, India05.2017
Skills
- Spark
- SQL
- Python
- Redshift
- CDK
- NoSQL
Timeline
Data Engineer II
Amazon
08.2023 - Current
Data Engineer
Amazon
08.2022 - 08.2023
Data Engineer
G42 Healthcare
08.2021 - 07.2022
Sr. Business Technology Analyst
ZS
04.2018 - 08.2021
Software Engineer
Newgen Software
07.2017 - 03.2018
Bachelor of Technology - Information Technology
Jaypee Institute of Information Technology
Similar Profiles
Lieszel WolfgrammLieszel Wolfgramm
Fulfillment Center Associate at AmazonFulfillment Center Associate at Amazon
Ethan AlcornEthan Alcorn
Transportation Manager II at AmazonTransportation Manager II at Amazon
Morena ArandaMorena Aranda
Learning at AmazonLearning at Amazon