Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Rishabh Rastogi

Noida

Summary

  • Total 16 years of experience in IT industry.
  • 4+ years of experience in Microsoft Azure Cloud technologies.
  • 10 years of experience in big data technologies.
  • 2 years of experience in Mobile application Development.
  • A Self-starter with a positive attitude, willingness to learn new concepts/technology and
    acceptance of challenges.
  • Excellent Technical, Interpersonal and Management skills.

Overview

17
17
years of professional experience
4
4
years of post-secondary education
1
1
Certification

Work History

Senior Azure Data Engineer

TCS - United Heath Group
11.2022 - Current
  • Designed and developed end-to-end scalable and robust data solutions and architecture using Azure Data Factory, Azure Databricks and Oracle ExaData for a Healthcare Client
  • Developed a re-usable framework for developing 100 + complex reports using python. The reports being generated in pdf and text formats.
  • Provided technical guidance and mentorship to junior team members, fostering a collaborative learning environment within the organization.
  • Designed internal process improvements to automate repetitive tasks, shortening data delivery times.
  • Developed comprehensive migration strategies, ensuring seamless transitions to the Azure platform.
  • Led end-to-end implementation of multiple high-impact projects from requirements gathering through deployment.
  • Tools & Technologies used: Azure Databricks,Azure Data Factory, ADLS Gen2, EXA Data, PySpark, SparkSQL, SQL, Python

Azure Data Engineer

TCS-Ericsson
05.2021 - 10.2022
  • Working in agile to collect user requirements and implement project with daily status reporting in scrum call.
  • Created Pipeline’s to extract data from on premises source systems to azure cloud data lake storage; Extensively worked on copy activities and implemented the copy behaviour’s such as flatten hierarchy, preserve hierarchy and Merge hierarchy
  • Creating pipeline for Data Onboarding (Full/Delta Load) from SAP Hana using Databricks notebook through ADF pipeline
  • Extensively worked on Azure Data Bricks with the help of Spark-SQL to implement client requirements.
  • Creating ADF pipeline for API onboarding of data
  • Deployed the codes to production environments with the help of Azure Devops
  • Using Apache Airflow as the Data Orchestration tool for scheduling DAGs and triggering ADF pipeline and databricks notebook
  • Tool & Technology used : ADF, ADLS, Azure Databricks, Azure Devops, Apache Airflow, Posgresql, Pyspark, SparkSQL, Hive, JIRA


Data Engineer

TCS-American Express
03.2020 - 04.2021
  • Solutioning of automating the FXIP scorecard
  • Data Understanding and Data Exploration
  • Data Cleaning and Data Ingestion using HiveQL
  • Design and development of FXIP scorecard for AD in different categories using pySpark
  • Tools & Technologies used: Pyspark, Hive, HDFS, Unix

Data Engineer

TCS-GE Transportation
01.2019 - 03.2020
  • Design and developed ABM(Analytics Based Maintenance) application using PySpark in Azure Databricks for Locomotives engines. It is a predictive based maintenance done on the basis of engine health, its usage and several other parameters
  • Implemented a new requirement to the existing application of Digital Pool using pySpark which tracks the incoming engines for maintenance and aligning them with any engine requirements with said specifications.
  • Migrated Talend code to python for the requirement of sending alerts in case of job success/failure along with the basic validation results in case of success.
  • Tools & Technologies used: Azure Databricks, PySpark, SparkSQL, Hive, Posgresql, Talend, Python



Big Data Engineer

TCS-Walgreens
01.2017 - 02.2019
  • Responsible for requirement gathering, design, documentation and development of features
  • Development of Spark jobs for data transformation in Scala
  • Development of multiple Sqoop job for data ingestion from various sources and scheduling the same using ESP
  • Automated Data Quality checks runs on the source data & send daily notification for any DQ violations
  • Performance tuning of Spark job created by Onshore/Offshore associates. This include understand memory requirement and configuring parameters so that cluster resources should be utilized in efficient manner
  • Automated File archival/purging process as per retention policy.
  • Analyzing the system for new enhancements/functionalities and perform Impact analysis of the application for implementing ETL changes.
  • Build, Deploy and Test the project code in Test (Lower) and in Production (Higher) Environments.
  • Tool & Technology used : Pyspark, Hive, Scala, HDFS, Unix, Spark, JIRA

Big Data Engineer

TCS- Apple GBI
05.2015 - 01.2017
  • Designing and Developing data pipeline using SparkSQL and Hive to ingest, transform and analyze operational data
  • Designing Hive solutions, query development, performance tuning and optimization. Troubleshooting various issues related to joins, memory exceptions in Hive
  • Reengineering of various existing applications using SparkSQL and fine tuning recourses for long running Spark Applications to utilize better parallelism and executor memory for more caching
  • Tools & Technologies used: Hadoop, Hive, Spark, SparkSQL, oozie, Sqoop, GIt, Pyhton, shell script


Team Lead

Jaguar Land Rover
09.2012 - 05.2015
  • Siemens Tecnomatix administration, support, re-architecture
  • Creating roles and users, assigning groups, Customer Centric Business Transformation.
  • Teamcenter, NX CAD systems installation, administration, support.
  • Creating roles and users, assigning groups in TcUA.
  • Release ,Un-release request
  • Request for deletion of Workflow
  • Request deletion of Item, Item revision,Dataset
  • Modify part name and numbers with admin privileges
  • Grant access to users for modifications of parts.


L2, L3 Support Engineer

TCS- Motorola Inc
07.2010 - 07.2012
  • Supporting and Maintaining 9 PLM applications in Motorola Solutions
  • OMF Client Adminstration & Troubleshooting: Account administration, Rule editing, Event queue monitoring, PDM1 Citrix server administration
  • Workflow Administration: Roles, Role Assignment Setup, Workflow routing setup
  • MOI (Metaphase Orcle Interface) & MCMS (Motorola Solutions Contract Manufacturing System) BOM Issues: Trouble shooting issues related to the interfaces between Teamcenter and other Motorola Solutions Application e.g ERP, MCMS, ICCS
    ECN Troubleshooting: Troubleshooting engineering change notices at various steps in the Life Cycle
  • Server Administration: Performance tuning, OMF configuration tool for editing config files, space alerts, memory alerts, Weblogic server admin, etc
  • Teamcenter Utilities/Tools: Running various utilities as when requested by user e.g BOM Loader, BOM dumper, Mass Loading of Role Assignment, Conversion from assembly to component and vice versa etc
  • Reports: Writing SQL queries for generating reports

Mobile Application Developer

TCS Innovation LABs
08.2008 - 04.2010
  • Design, Development of various mobile applications for BREW, Android phones using FlashLite 2.x, Adobe Flash CS4, ActionScript 2.0, Java, C, C++
  • Filed patent SMART CALLER ID SYSTEM patent application no. PCT/IN2010/000553
  • Filed patent PORTABLE WATER PURIFICATION SYSTEM patent application no. PCT/IN2010/000552


Education

Bachelors of Engineering -

Bangalore College of Engineering & Technology
Bangalore
07.2003 - 05.2007

Skills

  • Hands-on experience in Azure Data factory and its Core Concepts like Datasets, Pipelines, Activities, Scheduling and Triggers

  • Good working knowledge on Azure Databricks

  • Hands on Databricks Delta feature

  • Developed different types of notebooks to read and write the data into cloud

  • Having good knowledge on Spark RDD, Dataframe and Spark SQL

  • Created Report Framework in Databricks to generate 100 complex different types of reports in pdf format

  • Excellent knowledge of ADF building components – Integration Runtime, Linked
    Services, Data Sets, Pipelines, Inner Pipelines and Activities

  • Designed and developed data ingestion pipelines from on-premises to different layers
    into ADLS using Azure Data Factory

  • Good knowledge on SQL server Polybase concepts

  • Experience with integration of data from multiple data sources

  • Knowledge on Data Extraction from On-Premise Sources and Delta Extraction methods
    from Source Systems to ADLS

  • Worked on Get Metadata Activity, look up, Store Procedure, For each,IF and execute Pipeline activities in ADF V2

  • Worked on Parallel Databricks job processing using ADF Pipeline

  • Implemented dynamic pipeline to extract the multiple files into multiple targets with the
    help of single pipeline

  • Implemented alerts mechanism in Azure Data Factory Pipelines

Maintaining versioning of the azure data factory pipeline and as well as the Databricks notebooks in workspace

Implemented several big data analytics jobs in Spark written in Scala on 1000 nodes cluster and 280 nodes cluster

Certification

MapR Certified Data Analyst

Timeline

Senior Azure Data Engineer

TCS - United Heath Group
11.2022 - Current

Azure Data Engineer

TCS-Ericsson
05.2021 - 10.2022

Data Engineer

TCS-American Express
03.2020 - 04.2021

Data Engineer

TCS-GE Transportation
01.2019 - 03.2020

MapR Certified Data Analyst

03-2017

Big Data Engineer

TCS-Walgreens
01.2017 - 02.2019

Big Data Engineer

TCS- Apple GBI
05.2015 - 01.2017

Team Lead

Jaguar Land Rover
09.2012 - 05.2015

L2, L3 Support Engineer

TCS- Motorola Inc
07.2010 - 07.2012

Mobile Application Developer

TCS Innovation LABs
08.2008 - 04.2010

Bachelors of Engineering -

Bangalore College of Engineering & Technology
07.2003 - 05.2007

Similar Profiles

CREATE PROFILE
Rishabh Rastogi