Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic
Abhishek Khandelwal

Abhishek Khandelwal

Pune

Summary

Data Engineer with 5+ years of experience designing and implementing scalable data solutions across cloud and on-prem environments. Skilled in building ETL pipelines, architecting data warehouses, data migrations, and enabling analytics using Databricks, Spark, Airflow, Azure, AWS, and Snowflake.

Overview

6
6
years of professional experience
1
1
Certification

Work History

Data Engineer

Particle41
10.2022 - Current
  • Architected a comprehensive data architecture for an internal reporting dashboard, pioneering the enterprise-wide data model and interactive dashboard for one of the largest audiobooks publishers in the US. Leveraged Databricks and Pyspark for distributed data processing, Apache Airflow for orchestration of complex workflows, Amazon Redshift for columnar storage and querying, and Tableau for advanced data visualization and BI analytics.
  • Engineered and deployed scalable ETL pipelines utilizing directed acyclic graphs (DAGs) in Airflow to ingest and transform data from 15+ sources including relational databases, APIs, and flat files.
  • Managed data ecosystem consisting of Azure SQL databases, AWS Redshift, self managed MySQL databases, ELK stack comprising of over 100+ TB of combined data for effective scaling and minimal downtime.
  • Achieved a reduction of over 200 man-hours annually by authoring automated reporting scripts and multidimensional data models

Data Engineer

MIBA Group
01.2022 - 09.2022
  • Implemented separate prod and dev environment thus enabling us to assign appropriate roles to users for enhanced and secured data access.
  • Created and updated existing Azure Data factory pipelines to improve the performance thus reducing the monthly cost and overall runtime by 40%
  • Led the data analytics team to develop and optimize the data model and PowerBI reports to bring report load time under 15 seconds.
  • Implemented CI/CD using Azure devops to improve visibility and code quality.
  • Partnered with stake holders to understand and provide effective solutions for their requirements.

Data Engineer

Tredence
06.2021 - 01.2022
  • Worked on building live streaming data pipeline using spark structured streaming and databricks.
  • Designed data warehouse to ingest live streaming data and use the same for creating dashboard using PowerBI with data availability within 20 seconds.

Software Engineer-Data

MAQ Software
04.2019 - 05.2021
  • Created complex business logic using SQL to process revenue and subscription data for Microsoft Teams, D365, Azure and O365. Played impactful role in Microsoft one commercial partner reporting implementation.
  • Engineered data models using SSAS for reporting purpose.
  • Created scalable data pipelines using Azure Data Factory and Databricks for ETL purpose.

Education

Bachelors of Technology - Computer Science and Engineering

Lovely Professional University
Jalandhar, PB
05.2020

Skills

  • Programming Languages: SQL, Python, PySpark, Java
  • Cloud Platforms: Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, AWS Redshift, AWS Glue, S3, EC2
  • Data Engineering & ETL Tools: Databricks, Apache Airflow
  • Databases & Data Warehousing: MySQL, T-SQL, Snowflake, Data Warehousing, Data Modeling, Data Migration
  • BI & Reporting Tools: Power BI, Tableau
  • Version Control & DevOps: Git, Azure Devops, Bitbucket

Certification

  • MSFT Transcript
  • AWS Transcript

Timeline

Data Engineer

Particle41
10.2022 - Current

Data Engineer

MIBA Group
01.2022 - 09.2022

Data Engineer

Tredence
06.2021 - 01.2022

Software Engineer-Data

MAQ Software
04.2019 - 05.2021

Bachelors of Technology - Computer Science and Engineering

Lovely Professional University

Similar Profiles

CREATE PROFILE
Abhishek Khandelwal