CHAITANYA KIRTY
Bangalore
Summary
Lead Data Engineer at Vistaprint with a proven track record in optimizing data pipelines and developing scalable Big Data solutions. Expertise in PySpark, Snowflake, and DBT resulted in a 60% increase in pipeline efficiency while providing mentorship to teams. Strong skills in stakeholder management enhance data-driven analytics across my domain.
Overview
12
12
years of professional experience
1
1
Certification
Work History
Lead Data Engineer
Vistaprint
Bangalore
03.2022 - Current
- Spearheaded data engineering initiatives, optimizing pipelines and creating new solutions for Workforce Management analytics across all global locations.
- Designed a scalable Snowflake data mart exceeding three terabytes with DBT and Airflow.
- Led a domain-wise initiatives for 'Engineering Best practices'.
- Migrated over 30 AWS RDS sources to Snowflake using Fivetran, reducing ingestion efforts by 80%.
- Developed Looker dashboards and refined LookML models, decreasing report generation time from days to hours.
- Mentored my team in coding standards and collaborative programming practices.
- Transitioned full-load table refreshes to incremental processes in DBT, boosting pipeline efficiency by 50-60%.
- Facilitated cross-functional meetings to align data strategies with organizational objectives.
- Created a Time-series model(SARIMAX) to forecast Contacts data across all markets.
Senior Engineer
Fractal Analytics
06.2019 - 03.2022
- Migrated Python machine learning workflows to Spark on Databricks, reducing runtime from seven hours to fifty minutes.
- Developed ETL pipelines on Azure Data Factory, achieving hourly data freshness for clients.
- Led deployment of Data Science models within client environments utilizing Azure DevOps.
- Directed containerized model-serving platform with Docker and Kubernetes, enabling zero-downtime releases.
- Implemented CI/CD processes using Azure DevOps to streamline deployment workflows.
- Optimized data processing efficiency by integrating Spark workflows with existing systems.
- Facilitated collaboration between teams to enhance project delivery and performance.
Senior Software Engineer
Tavant Technologies
12.2016 - 06.2019
- Modernized Grubhub ETL from Redshift-Python to PySpark on AWS EMR, tripling throughput.
- Designed scalable Celery and RabbitMQ framework for over 200 daily ingestion jobs.
- Built new data pipelines on AWS using PySpark from MySQL, Cassandra DB, and FTP servers.
- Led initiative to optimize Spark jobs, reducing processing time by 40 percent.
- Supported BI via HiveQL and PrestoDB, facilitating self-service analytics for stakeholders.
- Ensured critical job support and debugging to provide accurate data access for stakeholders.
Senior Systems Engineer
Infosys
10.2013 - 12.2016
- Developed Hive/Sqoop workflows moving 500 GB/day between MySQL & Hive.
- Implemented PySpark processing & sampling techniques that cut QA cycle by 40%.
- Awarded Spot Award (Sep 2016) for rapid up-skilling & delivery in Big Data project.
Education
B.E. - Electrical & Electronics Engineering
Dayananda Sagar College of Engineering
Bangalore01.2013
Skills
- Snowflake and BigQuery
- Incremental processing
- ETL development
- Redshift and Databricks
- PySpark
- DBT
- Airflow and Looker
- Python programming
- SQL proficiency
- Kubernetes and Docker
- Azure DevOps and CI/CD practices
- AWS S3 and Azure Blob storage
- Agile methodologies
- Stakeholder management
- Mentoring
Certification
Databricks Certified Developer for Apache Spark (1.x-0941), 05/01/17, 05/01/20
References
Available upon request
Timeline
Lead Data Engineer
Vistaprint
03.2022 - Current
Senior Engineer
Fractal Analytics
06.2019 - 03.2022
Senior Software Engineer
Tavant Technologies
12.2016 - 06.2019
Senior Systems Engineer
Infosys
10.2013 - 12.2016
B.E. - Electrical & Electronics Engineering
Dayananda Sagar College of Engineering