Summary
Overview
Work History
Education
Skills
Accomplishments
Volunteer Experience
Timeline
Generic
Shubh Gupta

Shubh Gupta

Bengaluru

Summary

Experienced Data Engineer with 26 months of expertise at Deloitte, specializing in building robust data pipelines and optimizing data workflows. Proficient in Python (including Pandas and PySpark), AWS Services (AWS Glue, S3), SQL, and data warehousing (Redshift). Strong track record of automating ETL processes, managing data lakes, and ensuring data accuracy and integrity. Adept at cloud-based data solutions, big data technologies, and data governance.

Overview

8
8
years of professional experience

Work History

Data Engineer

Deloitte
07.2021 - Current
  • Designed and implemented PySpark ETL Scripts in AWS Glue for automated data extraction, transformation, and loading (ETL) processes
  • Developed Airflow DAGs to schedule, monitor, and orchestrate complex data pipelines, ensuring data consistency and reliability
  • Optimized Python code for data processing, resulting in significant improvements in pipeline performance
  • Managed data warehousing in Redshift, including schema design and query optimization for efficient data retrieval
  • Successfully migrated SQL-based data workflows to PySpark, enhancing scalability and cost-effectiveness
  • Leveraged DBVisualizer and BigQuery for advanced SQL queries and data analysis
  • Conducted thorough data profiling, quality checks, and cleansing to maintain data accuracy
  • Implemented data governance practices, ensuring data privacy, security, and compliance with industry standards
  • Collaborated with cross-functional teams, adhering to Agile methodology for project delivery.
  • Optimized existing queries to improve query performance by creating indexes on tables.

Faculty Member

AOEC
New Delhi
11.2015 - 10.2018
  • Provided coaching to students preparing for Govt Jobs like SSC JE and RRB JE in mechanical engineering.
  • Evaluating and improving learning content provided by the institute to the students.
  • Formulated well-structured syllabus of course content to detail learning goals and expected outcomes.
  • Collaborated with other faculty members on curriculum design and implementation.
  • Key areas of expertise – Thermodynamics, Heat Transfer, RAC, Fluid Mechanics, Industrial Engineering

Education

Post Graduate Program - Data Science and Engineering -

Great Lakes Institute of Management
12.2021

BE/B. Tech in Mechanical Engineering -

Manav Rachna International University
12.2015

Skills

  • Python
  • SQL
  • AWS Glue
  • AWS S3
  • Redshift
  • PySpark
  • Airflow
  • AWS Lambda
  • DBVisualizer
  • AWS (Amazon Web Services)

Accomplishments

  • Gold Medalist in [Your Major]
  • Gold Badge in SQL (HackerRank)
  • Merit Certificate (Class X)
  • 3rd prize (BHARAT KO JAANO)
  • Applause and Spot awards (Deloitte)

Volunteer Experience

  • College, Class Representative
  • TECHNO TEASERS, Participant
  • SPANDAN 2012, Volunteer

Timeline

Data Engineer

Deloitte
07.2021 - Current

Faculty Member

AOEC
11.2015 - 10.2018

Post Graduate Program - Data Science and Engineering -

Great Lakes Institute of Management

BE/B. Tech in Mechanical Engineering -

Manav Rachna International University

Similar Profiles

CREATE PROFILE
Shubh Gupta