Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Sumit Gangwar

Noida

Summary

  • Results-driven Data Engineer with expertise in Python, SQL, Snowflake, and cloud technologies (AWS 15+ services, GCP 10+ services). Proven ability to optimize data processes and reduce manual efforts by 90% through innovative ETL pipeline development. Skilled in big data frameworks with a strong focus on problem-solving, collaboration, and team management. Adept at leading teams to deliver impactful data solutions that improve operational efficiency and data quality.

Overview

3
3
years of professional experience
1
1
Certification

Work History

Data Engineer

Lumiq
Noida
06.2022 - Current
  • Developed and refined ETL pipelines, achieving a 90% reduction in operational manual efforts for large-scale data transfers.
  • Migrated over 200TB of historical data from legacy systems and cloud sources to Snowflake, Redshift, and S3, enhancing data quality and security through a config-driven approach.
  • Implemented a config-driven alerting system for pipeline failures, significantly reducing downtime and enabling prompt resolutions.
  • Designed reusable, template-based data pipelines, standardizing processes, and improving deployment speed by 60%.
  • Created versatile data connectors for SFTP, SharePoint, AWS Redshift, GCP BigQuery, JDBC, ODBC, and Snowflake, facilitating seamless data integration.
  • Developed Snowflake dashboards for real-time system monitoring and failure tracking, providing insights that reduced manual effort by 90%.
  • Utilized the Snowflake copy command to optimize costs and accelerate data ingestion speeds, resulting in a 60% reduction in loading times while maintaining high performance.
  • Created multiple AWS Glue and DMS pipelines using a standardized ingestion framework and templates, ensuring seamless integration with a 70% reduction in setup time and minimal operational issues.

Software Engineer Intern

Lumiq
Noida
02.2022 - 05.2022
  • Developed automated scripts to simplify common tasks and reduce manual effort by 30%.
  • Engaged in client interactions pertaining to Proof of Concept initiatives.

Education

B.Tech. (ECE) -

Galgotias College of Engineering & Technology
Greater Noida, U.P., India
06.2022

Skills

  • Languages: Python, SQL, PySpark, Spark
  • Tools/Platform: Airflow, Cloud Composer, RabbitMQ, Airbyte, OpenMetadata
  • AWS Services: Glue, DMS, EMR, Athena, S3, Lambda, Crawler, Secret Manager, etc
  • Data integration
  • GCP Services: Dataproc, Dataproc serverless, Cloud Composer, GCS, Secret Manager, etc
  • Data Warehouses: Snowflake, BigQuery, and Redshift
  • ETL Development
  • SQL and NoSQL databases

Certification

  • Developer - Associate certification, Amazon Web Services (AWS), 07/2023, 07/2026, AWS03705260, https://aws.amazon.com/verification

Timeline

Data Engineer

Lumiq
06.2022 - Current

Software Engineer Intern

Lumiq
02.2022 - 05.2022

B.Tech. (ECE) -

Galgotias College of Engineering & Technology

Similar Profiles

CREATE PROFILE
Sumit Gangwar