Yatindra Kumar Janghel
Bengaluru
Summary
Senior Data Engineer with over 11 years of experience in data engineering, specializing in Google Cloud Platform (GCP), SQL, StreamSets, Terraform, and Python.Demonstrated proficiency in designing, implementing, and optimizing data pipelines and infrastructure. Adept at leveraging cloud technologies to drive data-driven decision-making and enhance operational efficiency
Overview
11
11
years of professional experience
Work History
Software Engineering Specialist
BT Group
Bengaluru
10.2022 - Current
- Working as a Specialist Cloud Data Engineer which involves Designing and developing comprehensive end-to-end ETL frameworks utilizing Directed Acyclic Graphs (DAGs) StreamSets ,dataform & docker.
- Developed infrastructure using Terraform and implemented single-click deployment with Git.
- Crafted the complete data product journey.
- Requirement elicitation and analysis, followed by design, development and code reviews.
- Automated the repetitive tasks and workflows with Python scripts, significantly reducing manual effort and increased productivity.
- Developed automated data pipelines and optimized data processing workflows using Dataform, minimizing manual tasks, enhancing efficiency, and boosting query performance.
- Implemented data quality checks and validation rules within Dataform to ensure high data integrity.
- Developed and tested RESTful APIs in Python using Flask, with comprehensive endpoint validation and automation in Postman.
Design Engineer Professional
BT Group
Bengalury
06.2019 - 08.2022
- Designed and developed end-to-end ETL framework in GCP using Cloud Functions, Dataflow, BigQuery, StreamSets, and Python.
- Implemented a framework to load flat files from various source systems and HDFS filesystem into Oracle External Tables.
- Collaborated with business stakeholders for design implementation design and approach.
- Migrated all XFB feeds to HAAS, involving design and development activities
- Developed packages, stored procedures, and functions using PL/SQL.
- Utilized Python libraries like BeautifulSoup and Scrapy for web scraping, gathering valuable data from online sources.
Associate
BTGroup
Bengaluru
02.2017 - 05.2019
- Created controls for detecting and correcting revenue leakages using Money Map and RAID, based on customer requirements.
- Conducted analytics on CDR/operation data to identify revenue leakages and developed PL/SQL packages, stored procedures, and functions
- Migration of code artifacts and data, and set up a housekeeping framework to efficiently manage and load data.
System Engineer
Tata Consultancy Services
Bengaluru
03.2013 - 02.2017
- Development and deployment of Packages, Stored Procedures and Functions using PL/SQL.
- First hand analysis of the defects found during V&P and UAT.
- Writing the design document of various modules and preparing data model
Education
B.E (ET&T) -
RCET
Bhilai06-2012
Skills
- Google cloud platform( Cloud function,dataflow,bigquery,airflow,dataform,pubsub)
- Sql/PLSQL
- Hadoop
- Python
- CI/CD
- GIT
- Streamsets
- Datawarehousing
- Design and development
- Performance Optimization
- Data modeling
- ETL
- RAID
Accomplishments
- Successfully led the migration of data to GCP, optimizing transfer processes, ensuring data integrity and security, and reducing operational costs by 25%
- Designed and optimized SQL queries and ETL operations, reducing operation times by 35%.
- Implemented real-time data processing frameworks using Dataflow and StreamSets, enhancing data accuracy by 45%.
Timeline
Software Engineering Specialist
BT Group
10.2022 - Current
Design Engineer Professional
BT Group
06.2019 - 08.2022
Associate
BTGroup
02.2017 - 05.2019
System Engineer
Tata Consultancy Services
03.2013 - 02.2017
B.E (ET&T) -
RCET
Similar Profiles
Ewan LilleyEwan Lilley
Customer service advisor at BT GroupCustomer service advisor at BT Group
Charlotte JonesCharlotte Jones
Regulatory and Business Change Manger at BT GroupRegulatory and Business Change Manger at BT Group
Vivek SikarwarVivek Sikarwar
Service Reliability Engineer at BT GroupService Reliability Engineer at BT Group