Vinay Singh
Deoria
Summary
Results-driven Data Engineer with over 3 years of experience in designing and developing scalable data pipelines for Banking, Ecommerce, and Utilities sectors. Expertise in the Hadoop ecosystem and AWS cloud services, focusing on optimizing ETL processes. Proven track record of enhancing data ingestion strategies and implementing workflow automation to facilitate large-scale data processing. Committed to delivering innovative data-driven solutions that enhance business insights and support strategic decision-making.
Overview
3
3
years of professional experience
Work History
Data Engineer
Accenture
Gurugram
03.2022 - Current
- Ingested data from AWS LDS into AWS RDS, followed by data transformation.
- Developed ETL pipelines using AWS Data Flow Studio, PySpark, and SQL.
- Triggered and monitored Apache Airflow DAGs for automated workflows.
- Validated ingested data using AWS Glue and PySpark.
- Utilized AWS EMR for managing clusters and large-scale data processing.
- Worked with file formats like Parquet, Hudi, and CSV for efficient storage.
- Designed data transformation pipelines for monthly transaction reports.
Collaborated with business stakeholders to optimize processing efficiency.
Software Engineer Trainee
Torry Harris Integration Solution
Bangalore
11.2021 - 03.2022
- Trained in Big Data and Hadoop technologies.
- I worked on a UK-based eCommerce platform as a Data Engineer Trainee.
- Performed data transformations using PySpark, SQL, and Python.
- Utilized the AWS stack for data processing and storage.
Education
B.Tech - Computer Science & Engineering
Raj Kumar Goel Institute Of Technology
Ghaziabad08-2021
Skills
- Cloud Platforms: AWS (S3, EMR, Athena, Glue, Lambda)
- Big Data Technologies: Apache Hadoop, Hive, PySpark, and Databricks
- ETL & Data Pipelines: Apache Airflow, AWS Glue, Data Flow Studio
- Data Warehousing & Storage: Redshift, Data Lakes
- Databases: MySQL, MongoDB
- Programming languages: Python, Core Java
- DevOps and Agile: Git
- Airflow
Timeline
Data Engineer
Accenture
03.2022 - Current
Software Engineer Trainee
Torry Harris Integration Solution
11.2021 - 03.2022
B.Tech - Computer Science & Engineering
Raj Kumar Goel Institute Of Technology
Similar Profiles
Uttara MerupulaUttara Merupula
Team Leader at AccentureTeam Leader at Accenture
Anchitha K RAnchitha K R
Senior Software Engineer at AccentureSenior Software Engineer at Accenture
PHANI TEJA NALLAMOTHUPHANI TEJA NALLAMOTHU
Senior Manager, Technology Architecture at AccentureSenior Manager, Technology Architecture at Accenture