Proficient in programming and skilled at developing efficient data pipelines and ETL processes. Committed to staying up-to-date with the latest data technologies and trends, and continuously seeking to improve skills and knowledge.
Overview
2
2
years of professional experience
1
1
Certification
Work History
Data Engineer
Bajaj Finserv Health LTD
06.2023 - Current
Deployed and managing Trino- A open source query engine for reducing query cost of Azure Synapse on Datalake
Automated a process of querying data on synapse and uploading file to client's SFTP storage using Airflow.
Associate Data Engineer
Bajaj Finserv Health LTD
08.2021 - 05.2023
Implemented Delta Lake House using Azure Databricks (Pyspark) and Azure Data Factory to perform ETL to ingest data in Datalake from various sources (Salesforce, MySQL & Mongo DB).
Set up Azure Synapse Workspace to query data from Datalake and managed access control.
Implemented automated process to replicate delta tables in Azure Synapse (as external tables) with capability of handling schema and datatype changes to improve query performance and decrease management efforts using Azure Kubernetes.
Implemented schema auditing framework to collect metadata of objects and their ETL.
Implemented various ETL processes to ingest ad-hoc data using AKS and Python.
Prepared analytic reports, delivering summarized results, analysis and conclusions to business teams using Powerbi.
Delivered Ad-hoc Data summaries to Business teams and upper management to help them make decisions using T-SQL (Synapse serverless)
Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability
Intern
Bimagram
06.2021 - 08.2021
Data scrapping using Python (Beautiful Soup).
Implemented API to fetch insurance information with engine number
Education
B.Tech - Information Technology
Acropolis Institute of Technology And Research
Indore
07.2022
Shri Agrasen Vidhyalaya
Indore
Skills
Python
Azure Databricks (PySpark)
Azure Kubernetes
Azure Data Factory
Azure Synapse Analytics
T-SQL
Trino / Presto
Airflow
MySQL
PowerBI
Docker
GitHub
Azure DevOps
Accomplishments
Customer Service Insights Automation and KPI Implementation: Recognized with the Heroes Award for streamlining customer service insights and establishing monthly KPIs.
Anomaly Detection Automation in Claim Approval: Received the Limelight Award for automating anomaly detection in the claim approval process, resulting in a substantial reduction in customer follow-up calls.