KARTIK KUMAR
Delhi
Summary
Dynamic Senior Engineer with extensive experience at Mahindra Rise, specializing in the design of scalable ETL pipelines utilizing Azure Data Factory and Databricks. Proven expertise in data modeling and Delta Lake architecture, coupled with a strong ability to collaborate across functions, ensures high data quality and the delivery of actionable insights for informed decision-making.
Overview
2
2
years of professional experience
Work History
Senior Engineer
Mahindra Rise
06.2023 - Current
- Designed and orchestrated an end-to-end ETL pipeline using Azure Data Factory and Databricks DLT, following the Medallion Architecture (Bronze, Silver, Gold).
- Ingested raw data from car infotainment systems into Azure Data Lake Gen2 and AWS S3, storing it in Parquet format for optimized processing.
- Developed scalable PySpark jobs in Databricks for data cleansing, enrichment, and aggregation, transforming raw data into Delta Lake tables.
- Modeled processed data into a Star Schema to support high-performance analytics and reporting in Power BI.
- Implemented incremental data loading strategies to maintain data freshness while reducing compute costs.
- Ensured data quality and consistency through automated validation and pipeline monitoring.
- Enabled schema evolution and metadata tracking using Delta Lake to support dynamic and evolving data source.
- Collaborated with cross-functional teams (Product, Analytics, and Operations) to resolve data issues and deliver timely business insights.
Tech Stacks : Azure Data Factory, AWS S3, Databricks DLT, PySpark, Delta Lake, Power BI, Star Schema
Education
M.TECH - Computer Applications
National Institute of Technology - Tiruchirappalli
Tamil Nadu06-2023
B.TECH - Computer Science and Engineering
Guru Gobind Singh Indraprastha University
DELHI08-2020
Skills
- Python
- C
- Apache Hadoop
- Apache Spark
- Power BI
- AWS QuickSight
- S3
- EMR
- Lambda
- Data modeling
- Delta Lake architecture
- Data warehousing
- Azure Data Factory
- Redshift
- Athena
- Glue
- SNS
- Step Function
- Event Bridge
- Data Bricks
Personal Projects
Event-Driven ELT Pipeline
Cloud : AWS
- Designed and implemented an event-driven ELT pipeline where data ingestion is automatically triggered upon arrival in S3 using AWS EventBridge.
- The pipeline orchestrates the following steps using AWS Step Functions:
- Executes an AWS Glue Job to catalog and transform the data
- Filters data based on business rules and loads it into AWS Redshift
- Sends success/failure notifications via Amazon SNS
- This architecture ensures automated, scalable, and near real-time data processing with minimal manual intervention.
Tech Stacks : AWS S3, AWS EventBridge, AWS Glue, AWS Step Functions, AWS Redshift, Amazon SNS, Python
Timeline
Senior Engineer
Mahindra Rise
06.2023 - Current
M.TECH - Computer Applications
National Institute of Technology - Tiruchirappalli
B.TECH - Computer Science and Engineering
Guru Gobind Singh Indraprastha University
Similar Profiles
Leslie TrejoLeslie Trejo
Quality Control Inspector at Mahindra RiseQuality Control Inspector at Mahindra Rise
Khushi AroraKhushi Arora
Marketing Manager at Mahindra RiseMarketing Manager at Mahindra Rise
SWETHA MUPPALLASWETHA MUPPALLA
Regional / Cluster Trainer - Hyderabad Region at Mahindra Rise (Automobiles)Regional / Cluster Trainer - Hyderabad Region at Mahindra Rise (Automobiles)
Mahesh Kumar RajputMahesh Kumar Rajput
Sr. Data Entry Clerk at Access Healthcare Pvt Ltd.Sr. Data Entry Clerk at Access Healthcare Pvt Ltd.
Neha ShuklaNeha Shukla
Senior Associate- People @APAC at ArbolusSenior Associate- People @APAC at Arbolus