ADITYA MISHRA
New Delhi
Summary
Passionate Data Engineer with more than 3.5 years of expertise in developing robust and scalable data solutions. I played a pivotal role in transforming data accessibility for the organization, experienced in development of the Organization's Data Platform.
Overview
4
4
years of professional experience
Work History
Data Engineer
Gokwik
New Delhi
11.2023 - Current
- Build single source of truth from scratch for 2 organisations
- Deploying Clickhouse on EC2 to reduce at least 50% of warehouse cost and improve 2x query execution speed for consumers (currently POC done dev phase started)
- Building Real time Analytics
- Build self managed Model framework on data lake. Consumer just need sql query to build incremental model
- Designed & Build Data Lake on S3 from inception, encompassing landing zone, raw & curated tables.
- Build Ingestion framework which supports schema evolution tech stack: Spark, Scala, EMR, S3, Airflow, Python, Glue, Hudi (Parquet format)
- Leading the entire life cycle, including requirements gathering, architectural design, code development, testing, and production deployment for Frame.
- Building OLAP & OLTP data pipelines from diverse sources such as Postgres (RDS), MongoDB, GCP, S3, and Kafka
- Collaborated closely with cross-functional teams including tech, product, analytics, and data scientists to comprehend business use cases and independently deliver end-to-end solutions
- Implemented proactive monitoring with slack alerts & jira creation on pipeline failures, data flow delays, and data sanity checks, ensuring swift resolution of production issues to mitigate recurrence
- Implemented Workload Management (WLM) activities to enhance Redshift and Athena efficiency
- Automated nearly 90% of manual and ad-hoc tasks, streamlining workflows and increasing operational efficiency
- Conducted periodic reviews of the existing infrastructure, optimizing it for performance and cost-effectiveness
Data Engineer
Byju's
Bengaluru
03.2022 - 11.2022
- Built organization's centralized data platform
- Built & handled multiple product's OLAP data pipelines (click stream data)
- Built Streaming and Batch ETL/ELT data pipelines using Python, Spark and snow pipe
- Worked on various Data Migration projects
- Transformed & managed data warehouse i.e
- Snowflake & Data Lake
- Used some ETL tools like Fivetran for data ingestion
- Design and deployed data access management on snowflake.
Data Engineer
Tata Consultancy Services (TCS)
Noida
08.2020 - 03.2022
- Worked as Data Engineer for Walgreens Boot Alliance
- Worked in Data Ingestion Framework
- Worked in Data Transformation
- Led Data Ingestion Team [Team size 15], Data Ingestion Operations
- Led and developed two covid related pipelines using Databricks, Apache Spark, Scala, Azure, Blob, ADLS (data lake), and Azure ServiceBus [ team size 3 ].
Education
B.tech -
Inderprastha Engineering College
06.2020
Apache spark with scala for Big Data -
Udemy
05.2020
Core Java -
Ducat
11.2018
Programming in Java -
NPTEL (IIT Kharagpur)
10.2018
Design and Analysis of Algorithm -
NPTEL (IIT Madras)
10.2018
Skills
- Git
- Docker
- Java
- Apache Spark
- SQL
- Python
- Scala
- AWS
- Redshift
- EMR
- Snowflake
- Databricks
- Microsoft Azure
- Kafka
- Jira
- Data Modelling
- Airflow
- ETL/ ELT
- System Design
- Athena
- Clickhouse
- EC2
- Snowflake
Hobbies and Interests
Travelling, cooking, reading about world affairs
Accomplishments
- Multiple Quarterly Kwik Rewards by Gokwik
Languages
Hindi
First Language
English
Advanced (C1)
C1
References
References available upon request.
Timeline
Data Engineer
Gokwik
11.2023 - Current
Data Engineer
Byju's
03.2022 - 11.2022
Data Engineer
Tata Consultancy Services (TCS)
08.2020 - 03.2022
B.tech -
Inderprastha Engineering College
Apache spark with scala for Big Data -
Udemy
Core Java -
Ducat
Programming in Java -
NPTEL (IIT Kharagpur)
Design and Analysis of Algorithm -
NPTEL (IIT Madras)
Similar Profiles
KARAN BHATIAKARAN BHATIA
Associate Director of Sales at GoKwikAssociate Director of Sales at GoKwik
Veena ChVeena Ch
Talent Acquisition Senior Associate at GokwikTalent Acquisition Senior Associate at Gokwik
Maunil MehtaMaunil Mehta
Software Engineering Manager at GoKwikSoftware Engineering Manager at GoKwik
Sai RamSai Ram
Frontend Developer at GokwikFrontend Developer at Gokwik
Abhishek BaliAbhishek Bali
Associate Consultant at Easy Lane Management Private LimitedAssociate Consultant at Easy Lane Management Private Limited