Data Enthusiast with strong foundation in Programming, Big Data and experienced in building robust and reliable data pipelines, ETL processes & workflows
Overview
6
6
years of professional experience
4
4
years of post-secondary education
Work History
Senior Data Engineer
Altimetrik India Pvt Ltd
Bangalore
11.2018 - Current
Worked closely with stakeholders in understanding the information for the business needs and developed new ETL pipelines to meet there requirement's.
Developed Python scripts to process files from some specific sources to transform data and ingest into data lakes.
Experience in working with spark ecosystem using spark SQL queries on different formats like Text, CSV, Parquet, Avro and JSON files.
Built robust & reliable data pipelines in AWS Glue using PySpark by examining, unifying, Standardizing, cleaning & Identifying duplication with various customer data from internal and external data sources.
Developed python scripts to deploy Airflow DAGs for scheduling the ETL pipelines.
Senior Engineer
L & T Technology Services
Bangalore
03.2017 - 09.2018
Worked in health care domain project which involves gathering data from Hill-Rom device and providing statistics to the customer as soon as the data is collected via dashboards.
Proactively collected data from hardware devices and make the data available for customer analytical purpose .
Coordinated a team of 6 data engineers working on three different projects.