Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Rishabh Singh

Noida

Summary

Dynamic data engineering professional with more than 7 Years of experience, specializing in Azure and GCP solutions. Proven expertise in optimizing data pipelines and enhancing operational efficiency. Skilled in PySpark and collaboration, delivering scalable architectures that significantly reduce costs while ensuring high data quality and reliability.

Overview

7
7
years of professional experience
1
1
Certification

Work History

Assistant Manager

KPMG
Gurgaon
01.2023 - Current
  • Led and executed data engineering solutions across Azure and Google Cloud Platform (GCP), implementing cloud-native architectures to enhance scalability, reliability, and cost efficiency.
  • Configured and optimized resources on Azure (Synapse, Data Factory, Databricks), and GCP (BigQuery, Cloud Storage) to support end-to-end data processing workflows, improving operational efficiency, and reducing costs.
  • Developed and maintained automated data pipelines using Apache Airflow for orchestrating ETL workflows across hybrid cloud environments (Azure and GCP), ensuring high availability, fault tolerance, and scalability.
  • Leveraged Google BigQuery for large-scale data analysis and processing, performing complex queries on massive datasets, with a focus on cost control through partitioning and clustering techniques.
  • Designed and implemented Azure Data Factory pipelines for seamless data integration, automating the ingestion of data into Azure Data Lake, Azure SQL Data Warehouse, and Synapse.
  • Built and managed data pipelines using Azure Databricks for scalable data transformation and processing using PySpark, enabling real-time data processing and analytics at scale.
  • Applied PySpark optimizations for efficient Spark code execution on clusters to remove any bottlenecks.
  • Continuously monitored and optimized the performance of Airflow DAGs, Azure Data Factory pipelines, and BigQuery queries to ensure minimal latency and cost-effective execution.
  • Collaborated with data scientists, analysts, and stakeholders to design optimized data models, ensuring data quality, and scalability for analytics and reporting.

Senior Technical Associate L1

Publicis Sapient
Noida
09.2021 - 01.2023
  • Assisted in the development of data pipelines using Azure Data Factory, gaining hands-on experience in data integration and orchestration
  • Supported the implementation of Azure Databricks for data processing, including Spark programming and optimization of PySpark scripts
  • Contributed to the enhancement of data storage solutions on Azure, employing best practices for data partitioning and indexing
  • Collaborated with senior data engineers to troubleshoot and resolve issues in data pipelines, ensuring smooth operations

Software Engineer

GlobalLogic (A Hitachi Company)
Noida
09.2019 - 09.2021
  • Worked with Azure Synapse for data warehousing and Synapse pipelines for data warehousing
  • Design and developed new API endpoints in .NET Core for internal teams to call API and get data
  • Worked with CI/CD pipelines for app development and deployment to Azure App Services
  • Written SQL server stored procedures, functions and more for handling API requests and other tasks
  • Written python code in Azure Function to run the business logic to perform calculations on submitted data

Associate Software Engineer

Conduent
Noida
01.2018 - 09.2019
  • Written custom tailored C# applications using internal stitched libraries to formulate healthcare claims
  • Written SQL stored procedure to handle data in the database and making SSRS reports for reporting
  • Designed a portal using ASP.NET MVC architecture, Bootstrap, etc

Education

B.Tech - IT

Moradabad Institute of Technology
Moradabad, Uttar Pradesh
07.2016

Skills

  • Google BigQuery
  • Airflow
  • Azure
  • Google Cloud Platform (GCP)
  • Azure Data Factory
  • Azure Synapse
  • Azure DataBricks
  • SQL
  • PySpark
  • Python
  • Data Modelling
  • Hadoop
  • Spark
  • GIT
  • Microsoft Azure DevOps
  • SAFe
  • Scrum

Certification

  • Databricks Data Engineer Associate
  • Microsoft Azure Data Engineer (DP-203)
  • Microsoft Azure Data Fundamentals (DP-900)
  • Microsoft Azure Fundamentals (AZ-900)
  • Microsoft Power Platform Fundamentals (PL-900)
  • SAFe 5 Practitioner
  • DataBricks Lakehouse Fundamentals

Timeline

Assistant Manager

KPMG
01.2023 - Current

Senior Technical Associate L1

Publicis Sapient
09.2021 - 01.2023

Software Engineer

GlobalLogic (A Hitachi Company)
09.2019 - 09.2021

Associate Software Engineer

Conduent
01.2018 - 09.2019

B.Tech - IT

Moradabad Institute of Technology

Similar Profiles

CREATE PROFILE
Rishabh Singh