VINAY KUMAR DUDI
Lead Data Engineer
Hyderabad
Summary
Lead Data Engineer professional with around 12 years of experience designing and delivering data engineering solutions across GCP, Snowflake, AWS, and Azure. Expert in large-scale ETL, AI-driven analytics, and leading teams in semiconductor, banking, and IT domains. Strong in data architecture, modernization, and pipeline automation.
Overview
2026
2026
years of professional experience
6
6
Certifications
Work History
Lead Data Engineer
Micron Technologies
2022 - Current
- Leading a 6-member team to build analytics platforms for semiconductor data.
- Designed pipelines from RDBMS to GCP BigQuery using Cloud Dataflow, Pub/Sub, Cloud Functions, and Workflows.
- Developed HR Data Migration from Workday to Snowflake using REC (Raw, Enriched, Curated) architecture.
- Automated orchestration using Cloud Composer and Airflow, reducing overhead by 30%.
- Built GDML Framework pulling data from diverse sources using Python FastAPI, hosted on GKE, providing unified APIs to multiple applications. CI/CD via GCP Cloud Build.
- Collaborated globally with data science teams for GCP–Snowflake integration.
Senior Data Engineer
Brillio Technologies
2021 - 2022
- Built ETL pipelines migrating data from RDBMS to AWS Redshift using Glue, DMS, and Lambda.
- Created automated validation and monitoring frameworks ensuring high-quality data delivery.
- Improved Redshift performance with optimized partitioning and query tuning.
Big Data Engineer
Enquero Global LLP
2018 - 2021
- Designed and developed Azure-based ETL using ADF, Databricks, and ADLS.
- Built scalable data frameworks integrating structured and unstructured sources.
- Enhanced Spark performance by 25% and supported analytics for IT operations monitoring.
System Engineer
Tata Consultancy Services
2013 - 2018
- Supported Bank of America in Hadoop-based batch workflows for financial data.
- Delivered ETL solutions integrating Hive, Pig, and Oozie for banking analytics.
Education
Master of Computer Applications -
JNTU University
01-2013
Bachelor of Science -
SV University
01-2009
Skills
- Cloud Platforms: GCP (BigQuery, Cloud Functions, Cloud Run, Workflows, Pub/Sub), AWS (Glue, Lambda, DMS, Redshift), Azure (ADF, Databricks, ADLS, Logic Apps)
- Big Data: Hadoop, Spark, Hive, Pig, Sqoop, Dataproc
- ETL & Orchestration: Airflow, NiFi, Cloud Composer, Dataform
- Programming: Python, SQL, PySpark, Java, Scala
- Data Warehousing: Snowflake, BigQuery, Redshift
- DevOps & CI/CD: Git, Jenkins, Terraform, Docker, GCP Cloud Build
- Methodologies: Agile (Scrum, Kanban)
Certification
iLead Manager – GMI (2024)
DOMAINS WORKED
Semiconductor | Banking | Software & IT etc.
Timeline
Lead Data Engineer
Micron Technologies
2022 - Current
Senior Data Engineer
Brillio Technologies
2021 - 2022
Big Data Engineer
Enquero Global LLP
2018 - 2021
System Engineer
Tata Consultancy Services
2013 - 2018
Master of Computer Applications -
JNTU University
Bachelor of Science -
SV University