Chetan Bhogayta
Lead Data Engineer
Summary
Results-oriented Data Engineer with 9+ years of experience in Data Engineering, ETL, and Data Warehousing across diverse domains. Expertise in AWS, Python, SQL, Spark, and Apache Airflow, with a proven track record of designing, building, and optimizing scalable data pipelines. Strong background in planning, analysis, and implementation of data solutions, leveraging cloud-based technologies to deliver robust data warehouses that drive business intelligence and analytics.
Overview
10
10
years of professional experience
Work History
Senior Engineer 2
Eli Lilly
Bengaluru
09.2021 - Current
- Designed, developed, and managed scalable ETL pipelines using AWS, Python, and PySpark to transform and load data into AWS Redshift and PostgreSQL from diverse sources such as ALSC, VEEVA, and CLUWE.
- Integrated Apache Airflow with AWS Glue to orchestrate ETL workflows, ensuring efficient job scheduling and dependency management.
- Led end-to-end migration projects from on-premises to AWS, minimizing downtime, and optimizing resource utilization.
- Architected and implemented an ETL pipeline to track clinical supply data, improving data accuracy and reporting efficiency.
- Applied data governance best practices using Immuta to enforce security, compliance, and access control policies.
- Worked on data harmonization using TAMR to standardize disparate datasets with varying data models, ensuring consistency and usability.
- Monitored employee performance to keep projects on task.
- Determined equipment failure root cause and implemented solutions.
- Led cross-functional team for successful completion of multiple large-scale projects, ensuring timely delivery within budget constraints.
- Mentored junior engineers, providing guidance in best practices and fostering professional growth opportunities.
- Conducted in-depth research to stay ahead of industry trends, applying innovative techniques to current projects.
Application Development Team Lead
Accenture Solutions Pvt Ltd
Pune
05.2019 - 09.2021
- Developed and implemented ETL processes to extract, transform, and load data from legacy systems such as Informatica PowerCenter and Ab Initio to AWS Glue.
- Extensively worked with DataFrames and Datasets using PySpark on AWS Glue to process large-scale data for a telecom client.
- Designed and built a data processing framework using AWS Lambda and Pandas to enhance automation and efficiency.
- Implemented Slowly Changing Dimensions (SCD Type-2), late-arriving data handling, and data harmonization using Informatica PowerCenter, SQL, and Unix.
- Provided product support during various testing cycles, including SIT and UAT, ensuring smooth deployment and issue resolution.
- Participated in design and planning exercises for future software rollouts.
- Collaborated with stakeholders regarding project capabilities and limitations to deliver optimal functionality.
Programmer Analyst
Bitwise
Pune
04.2018 - 04.2019
- Worked on the migration of existing Ab Initio code (graphs) to Informatica PowerCenter (mappings/workflows) to enhance performance and maintainability.
- Designed and implemented end-to-end ETL processes using Informatica PowerCenter, including job scheduling and workflow automation.
- Provided production support for ongoing ETL workflows, ensuring stability and performance.
- Automated support processes using Unix scripting improve operational efficiency and reduce manual intervention.
- Worked closely with clients to establish specifications and system designs.
- Tested and deployed scalable and highly available software products.
- Identified opportunities for process improvement within the development team, implementing changes that led to increased efficiency and collaboration among members.
- Contributed to successful project completion with effective time management and prioritization skills, meeting tight deadlines consistently.
- Communicated complex technical concepts clearly to non-technical stakeholders during project discussions and progress updates.
Programmer Analyst
Cognizant Technology Solutions
Pune
10.2015 - 04.2018
- Worked on SmartTrials 3.0, a data warehousing product designed for pharmaceutical companies to manage large-scale clinical trial data.
- Developed ETL mappings, sessions, and workflows using Informatica 9.6 to extract data from Oracle and flat files, apply business rules, and load it into an Oracle database.
- Implemented key data transformation modules, including Data Acquisition, Auto Ingestion, Harmonization, Late-Arriving Dimension Handling, and SCD Type-2.
- Automated email notifications for the landing and mart layers using UNIX shell scripting, improving monitoring and alerting efficiency.
- Enhanced user experience through the development of intuitive and user-friendly interfaces.
- Boosted productivity by automating repetitive tasks using custom-built scripts and tools.
- Documented program development and revisions to maintain records.
Education
Master of Technology - Communication Engineering
Vellore Institute of Technology
Chennai07.2015
Skills
AWS
SQL
Data Warehousing
Unix
Spark
Airflow
Git
undefinedTimeline
Senior Engineer 2
Eli Lilly
09.2021 - Current
Application Development Team Lead
Accenture Solutions Pvt Ltd
05.2019 - 09.2021
Programmer Analyst
Bitwise
04.2018 - 04.2019
Programmer Analyst
Cognizant Technology Solutions
10.2015 - 04.2018
Master of Technology - Communication Engineering
Vellore Institute of Technology
Similar Profiles
Andrew MalettaAndrew Maletta
Associate Director - Automation - Device Assembly and Packaging at Eli Lilly and CompanyAssociate Director - Automation - Device Assembly and Packaging at Eli Lilly and Company
Ayan DuttaAyan Dutta
Sr. Principal Engineer / Veeva Vault Architect at Eli LillySr. Principal Engineer / Veeva Vault Architect at Eli Lilly
HANNA HONERHANNA HONER
Supervisor – Parenteral Operations (Formulation) at Eli LillySupervisor – Parenteral Operations (Formulation) at Eli Lilly
Jennifer SextonJennifer Sexton
Senior Diabetes Sales Specialist at Syneos Health - Eli LillySenior Diabetes Sales Specialist at Syneos Health - Eli Lilly
Lakshmi Narasimha Reddy GuntaLakshmi Narasimha Reddy Gunta
Senior Engineer 2 at Ceremorphic technologiesSenior Engineer 2 at Ceremorphic technologies