Yash Pal Singh
Summary
Driven data engineer with 2.6 Years of experience wrangling big datasets.
Having experience in Data warehouse, Azure Data Factory, SQL, Azure
Databricks, Python & PySpark involves design, development and testing.
Created ETL pipeline and flow using Azure Data Factory, SQL/T-SQL and
pyspark.
Overview
3
3
years of professional experience
1
1
Certification
Work History
System Engineer (Data Engineer)
TATA Consultancy Services
12.2021 - Current
- Created PySpark frame to bring the data from input files and Database.
- Built the infrastructure required for optimal extraction, transformation, and loading (ETL) of data from a wide variety of data sources.
- Used Python for SQL/CURD operations in DB, file extractions transformation/generation.
- Created python notebook using pyspark to read the file and transform the data based on the business requirement.
- Created calculated column using available data in
source and done transformation as per business
requirement. - Developed the pipeline, datasets, data-flow and
linked services to perform ETL activities. - Analyzed the SAS based program and transform them into latest technologies like SQL and Azure based technologies.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
- Enhanced data quality by performing thorough cleaning, validation, and transformation tasks.
Education
B.Tech - Information Technology
Dr. Akhilesh Das Gupta Institute of Technology
Delhi07.2021
Diploma - Information Technology
Ambedkar Institute of Technology
Delhi04.2018
12th -
CBSE
03.2015
10th -
CBSE
04.2013
Skills
- Python
- PySpark
- SQL/T-SQL
- Azure Data Factory
- Databricks
- Azure Data warehouse
- Azure SQL Server
- GITHUB
- SSIS
- Delta
- SparkSQL
- Data Analysis
- Data Transformation
- ETL
- Adaptability
- Creativity
- Teamwork
- Problem-solving
Languages
- Hindi
- English
Certification
- Microsoft Certified: Azure Fundamentals (AZ-900)
- Microsoft Machine Learning from DreamUny
- Python from WebTeck
- Azure Data Factory from Udemy
HOBBIES
- Travelling
- Watching Movies
Timeline
System Engineer (Data Engineer)
TATA Consultancy Services
12.2021 - Current
B.Tech - Information Technology
Dr. Akhilesh Das Gupta Institute of Technology
Diploma - Information Technology
Ambedkar Institute of Technology
12th -
CBSE
10th -
CBSE
Similar Profiles
Vrunda ChindhadeVrunda Chindhade
HR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services LimitedHR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services Limited
Sreya BasuSreya Basu
Procare Analyst at Tata Steel(Tata Consultancy Services)Procare Analyst at Tata Steel(Tata Consultancy Services)
Arun VArun V
Program Manager at Tata consultancy services | Financial Ombudsman ServicesProgram Manager at Tata consultancy services | Financial Ombudsman Services
Naimish AgnihotriNaimish Agnihotri
Assistant Delivery Manager at Tata Consultancy Services, Global Shared ServicesAssistant Delivery Manager at Tata Consultancy Services, Global Shared Services
Nishikanta SahuNishikanta Sahu
System Engineer / Data Engineer at Tata Consultancy ServicesSystem Engineer / Data Engineer at Tata Consultancy Services