Summary
Overview
Work History
Education
Skills
Timeline
Generic

Priyanshu Gupta

Mangalore

Summary

Results-driven Data Engineer with experience at Genpact, specializing in scalable data ingestion and automation. Expert in SQL and Spark SQL, I developed a reusable ingestion framework supporting 90+ data sources, enhancing data reliability. Strong collaborator, adept at resolving complex data quality issues and optimizing cloud-native architectures for improved performance.

Overview

3
3
years of professional experience

Work History

Data Engineer

Genpact
06.2023 - Current
  • Designed and implemented scalable data ingestion pipelines to load Parquet files from Amazon S3 into Databricks Delta tables through automated Databricks jobs.
  • Built a reusable ingestion framework supporting 90+ data sources, including both streaming and batch (ETL/ELT) workflows.
  • Developed a test automation framework to validate ingestion processes, improving data reliability and reducing manual verification efforts.
  • Performed data analysis and root cause investigation for complex data quality issues, resolving non-standard table configurations and ensuring consistency across environments.
  • Migrated and optimized on-premises SQL queries to Databricks (DBR) SQL, improving performance and supporting cloud-native architecture.
  • Validated query outputs between Greenplum (GP) and Databricks, rewriting and tuning Spark SQL queries to ensure consistency, accuracy, and performance across platforms.
  • Collaborated with cross-functional teams to streamline data processing, reduce latency, and enhance the overall data analytics ecosystem.

Software Engineering Intern

Genpact
03.2023 - 06.2023
  • Contributed to building a web platform from scratch that enabled users to compare and purchase insurance products (life, health, car, etc.) from multiple providers, acting as a digital insurance marketplace.
  • Worked as part of the front-end development team, creating responsive and user-friendly pages using HTML, CSS, and JavaScript.
  • Collaborated with team members to design and implement features that simplified complex policy information and improved the overall user experience.
  • Gained hands-on experience in web development, UI design, and agile teamwork while contributing to a project that aimed to digitize the insurance sector and make policy buying more transparent and accessible.

Education

Bachelor of Engineering - Computer Science

BMS College of Engineering
Bengaluru, India
06-2023

Skills

  • Programming & Query Languages: SQL, Spark SQL, Python
  • Big Data & Cloud Platforms: Databricks, Apache Spark, Delta Lake, AWS S3
  • Databases & Data Warehousing: Greenplum, HVR, Relational Databases
  • Data Engineering & Pipelines: ETL, ELT, Batch Processing, Streaming Data Ingestion, Data Migration
  • Frameworks & Automation: Ingestion Framework Development, Test Automation Frameworks
  • Data Quality & Analysis: Data Validation, Data Quality, Root Cause Analysis, Data Reconciliation
  • Other Tools & Practices: Git, Performance Optimization, Cloud Data Engineering

Timeline

Data Engineer

Genpact
06.2023 - Current

Software Engineering Intern

Genpact
03.2023 - 06.2023

Bachelor of Engineering - Computer Science

BMS College of Engineering

Similar Profiles

CREATE PROFILE
Priyanshu Gupta