Motivated and experienced Data Engineer with over 5+ years of experience developing new and innovative design processes, helping companies to meet and exceed projected expectations in the field of data including data analysis, SQL, data transformations, scripting, automation and customer engagement. This makes me data-driven, and customer focused. I believe in taking ownership and working backwards that provides insight and enables sound business decision-making.
Overview
5
5
years of professional experience
Work History
Senior Data Engineer
Larsen and Toubro Infotech (LTIMindtree)
Pune
05.2019 - Current
As a part of RTB team, I managed 100+ ETL applications (across Africa continent) using different technological platforms - Hadoop, Teradata utilities, Informatica Power Center, Linux / Unix, Mainframe, TWS, CA Wade, Airflow
Responsible for migration of different banking applications from Informatica 10.1 to Virtual Server Component in Informatica 10.4 which increased efficiency and capability of Data Processing in Informatica
Used ETL pipelines to transform and clean data, improving data quality and accuracy by 80-85%
Developed a data lake architecture that allowed for efficient storage and retrieval of large datasets.
Created ETL scripts to move and transform data from various sources into a centralized repository of Hadoop.
Modified complex business table structure to use partitioning and CT tables and use of Bucketing and Tez Engine instead of Traditional Spark and Un-Partitioned Datasets.
Collaborated with engineers, support teams to help partners learn to use Big Data products efficiently.
Senior Data Engineer
LTIMindtree(On-site)
Johannesburg
06.2023 - 12.2023
Initiated Data Rail Program which aimed to transform legacy Teradata warehouse to Hadoop platform which assists in reading data via SQL's and run them via Spark-SQL and ingest into Hadoop via a common code structure.
Managing a team of 12 Big Data Engineers both on-site and off-site from support and development point of view.
Built complex business and financial analytical reports utilizing multiple sources of information from different systems to increase data revenue for ABSA client by 2x.
Conversion of Teradata Marts and extracts into Hadoop Marts and extracts for 12 African countries, using different technological platforms - Hadoop, Hive, Spark, Teradata utilities, Linux / Unix, CA Wade.
Conducted research and development activities related to new technologies, frameworks and tools used in Big Data Analytics projects.
Promoted customer success in building and migrating applications, software and services on Big Data platform.
Education
Bachelor of Science - Information Technology
SRM Institute of Science And Technology
Chennai
05.2019
Skills
Scala
SQL
Hadoop(Big Data/Hive)
Informatica
Workload Schedulers
Data Modeling
Data Warehousing
Data Migration
Data Analysis
Teradata and Oracle Utilities
ETL Pipeline Processing
Linux Shell Scripting
AWS(Amazon Web Services)
Associations And Achievements
Accolades, Pune, Awarded as - 'Rising Star', 'Shooting Star award', ' Go-MX Award' for delivery excellence, hard dollar savings for client.
Aakash Research Labs, Chennai, 08/2016, 04/2019, Project Manager and Technical Lead
Languages
English
Hindi
Japanese
Timeline
Senior Data Engineer
LTIMindtree(On-site)
06.2023 - 12.2023
Senior Data Engineer
Larsen and Toubro Infotech (LTIMindtree)
05.2019 - Current
Bachelor of Science - Information Technology
SRM Institute of Science And Technology
Similar Profiles
Arpit BakliwalArpit Bakliwal
Network Security Lead – Senior Specialist at LTIMindtree Ltd [Formerly :Larsen & Toubro InfoTech]Network Security Lead – Senior Specialist at LTIMindtree Ltd [Formerly :Larsen & Toubro InfoTech]