Summary
Overview
Work History
Education
Skills
Certification
Websites
Timeline
Generic

Abeg Patel

Jharsuguda

Summary

Experienced data engineer with 3.8 years of expertise in designing robust data architectures and end-to-end solutions in AWS and Azure. Proficient in Microsoft Fabric and Power BI for scalable data systems and business insights. Proven ability to deliver insights, ensure data accuracy, and support strategic decisions through collaboration with cross-functional teams.

Overview

4
4
years of professional experience
1
1
Certification

Work History

Azure Data Engineer

KPMG India
08.2024 - Current
  • Developed and maintained ETL pipelines using Python and PySpark for data extraction, transformation, and loading into the Fabric Lakehouse and Data Warehouse.
  • Implemented ETL, data quality checks, and medallion architecture to process data in the gold layer, and stored it in Azure SQL to serve data to the user interface via API.
  • Integrated data from GET APIs and Salesforce web services, applying Medallion Architecture for optimized data processing.
  • Implemented incremental loading to reduce full load times, and enhance data efficiency.
  • Applied Slowly Changing Dimension Type 1 (SCD 1) to manage dimension changes.
  • Scheduled and automated ETL pipelines using Azure Data Factory.
  • Enabled efficient reporting and analysis for sales and tenant management KPIs dashboard using Power BI visualizations.
  • Technologies Used: Python, PySpark, Azure Databricks, Azure SQL, Data Factory, Lakehouse, Data Warehouse, and Power BI.

GCP Data Engineer

WhiteBlue Cloud Services
05.2024 - 08.2024
  • Developed and maintained APIs, focusing on data integration using Python and SQL for the extraction, transformation, and loading (ETL) of data from various source systems (e.g., BigQuery, PostgreSQL), and external data sources.
  • Developed GET and POST APIs using Google Cloud Functions, and orchestrated workflows using Cloud Composer to manage and automate API-related tasks.
  • Built batch data pipelines using Cloud Composer to transfer data from Cloud SQL to Amazon RDS.
  • Technology Used: Python, Oracle, PostgreSQL, MS SQL Server, GCP (GCS, BigQuery, Cloud Function, Cloud Composer), AWS RDS.

AWS Data Engineer

Cognizant Technology
09.2021 - 02.2024
  • Designed and implemented ETL processes using PySpark, Python, and SQL to extract, transform, and load data from diverse sources, including Oracle databases, various file formats, and external locations. Integrated AWS Redshift for advanced analytics.
  • Automated the manual reporting process for duplicate, fatal, and error records within the ETL pipeline, leveraging AWS SES for automated email notifications, resulting in an 80% improvement in processing time.
  • Implemented automated, secure monitoring of Databricks jobs using Databricks cluster and jobs APIs.
  • Automated JIRA process using Python REST APIs to streamline project management tasks.
  • Created an ETL workflow that generates PDFs from Oracle data into AWS S3 by leveraging Databricks' Open Analytics Platform.
  • Technology Used: Oracle, AWS Databricks, PySpark, Python, AWS S3, AWS SES, and AWS Redshift.

Education

Bachelor of Technology -

GIET University
Rayagada, Odisha
01.2021

Intermediate -

AICS
Cuttack, Odisha
01.2017

Matriculation -

DAV Public School
Sundargarh, Odisha
01.2015

Skills

  • Python
  • SQL
  • Pyspark
  • Oracle
  • Amazon RDS
  • Azure SQL
  • Azure Databricks
  • Microsoft Fabric
  • Amazon Redshift
  • Google BigQuery
  • Cloud Function
  • Cloud Composer

Certification

  • AWS Cloud Practitioner external certification
  • Google Cloud Digital Leader external certification
  • Fabric analytics engineer associate
  • Cloud computing by NPTEL
  • Machine Learning by Ekovation

Timeline

Azure Data Engineer

KPMG India
08.2024 - Current

GCP Data Engineer

WhiteBlue Cloud Services
05.2024 - 08.2024

AWS Data Engineer

Cognizant Technology
09.2021 - 02.2024

Bachelor of Technology -

GIET University

Intermediate -

AICS

Matriculation -

DAV Public School

Similar Profiles

CREATE PROFILE
Abeg Patel