Summary
Overview
Work History
Education
Skills
Profile Summary
Timeline
Generic

Snehal Godge

PUNE

Summary

Having 4 years of experience in Azure Data Engineer with a passion for problem-solving and a track record of delivering successful data projects. Expertise in designing and implementing complex data solutions in the Azure Cloud. Looking to join a dynamic team to continue developing technical and personal skills.

Overview

9
9
years of professional experience

Work History

Data Engineer

Atlas Copco
05.2024 - Current


Roles And Responsibilities

  • Created metadata driven framework to Ingest data from various sources in data lake
  • Build Generic Notebook to load data in bronze layer.
  • Added dynamic scd 1 logic while loading data to bronze layer
  • Created Generic ADF pipeline which work using metadata to copy from any source database to ADLS Gen2 storage account
  • Added proper logging and exception handling to trace any issue during pipeline execution.

Azure Data Engineer

Expleo Solution Limited
3 2022 - 05.2024
  • Build Ingestion framework to Ingest data from various sources to Platform
  • Validate schema and store data in azure blob storage for further processing
  • Process incoming data using Azure Databricks and Datafactory
  • Encrypt PII data
  • Apply transformation and aggregation on Silver data to give granular data in gold layer
  • Ingest data into reporting databases(Azure SQL).
  • Build Great expectation DQ framework
  • Designed and developed Retention framework
  • Streamlined complex pipelines by breaking them down into manageable components for easier implementation and maintenance.
  • Optimized data processing by implementing efficient ETL pipelines and streamlining database design.

Azure Data Engineer

Konecranes and Demag Pvt LTD
01.2020 - 02.2022
  • Get Worldwide sells file from upstream
  • Transform raw data to build different KPI
  • Build Sells KPI
  • Build Sells Anomaly KPI
  • Create ADF pipelines and triggers to process this data using databricks
  • Store output data in Azure blob storage which will be used by reporting systems.

Test Engineer

Eclerx Services Ltd
09.2015 - 12.2019
  • Understood and analyzed the customer requirements
  • Involved in the requirement study, design and development of the test cases
  • Test Execution of system test suite
  • Managed Defect tracking and reporting

Education

B.E - Electronics and Telecommunication Engineering

SVERI'S College of Engineering
Pandharpur, Maharashtra

Skills

Azure Databricks

PySpark

Python

Spark SQL

CI/CD

Azure Data factory

Big Data Processing

SQL Expertise

Profile Summary

  • Hands-on experience in – Azure Data Lake Store (ADLS), Azure SQL, Azure Data Factory (ADF),Azure Data Bricks (ADB), Azure DevOps, Azure Synapse Analytics, Azure BLOB storage, Key Vault, SQL Server, PySpark, Spark SQL, etc
  • Experience in building Azure Data Bricks (ADB) Spark-Python Notebooks to perform datatransformations.
  • Hands-on experience in Data Frames, and Spark SQL.
  • Posses Good knowledge in Delta lake, Auto Loader and Delta Live tables
  • Designed and developed an metadata driven framework to process Databricks notebook jobs
  • Have Good knowledge in spark optimization using various techniques such as catche,Persist and Z ordering
  • Orchestrated data integration pipelines in ADF using various Activities like Get Metadata,Lookup, For Each, Wait, Execute Pipeline, Set Variable, Filter, until, etc.
  • Implemented a dynamic pipeline to extract multiple files into multiple targets with the help of asingle pipeline.
  • Build Data Quality framework using Great expectation.
  • Designed developed data retention framework
  • Posses Sound knowledge in data warehousing concepts
  • Automated execution of ADF pipelines using Triggers.
  • Extensively worked on data source types – SQL server,Kafka, JSON, CSV, Parquet etc.
  • Experience in SQL – Joins/co-related queries/sub-queries etc.
  • Good knowledge of tables, stored procedures, views, etc.

Timeline

Data Engineer

Atlas Copco
05.2024 - Current

Azure Data Engineer

Konecranes and Demag Pvt LTD
01.2020 - 02.2022

Test Engineer

Eclerx Services Ltd
09.2015 - 12.2019

Azure Data Engineer

Expleo Solution Limited
3 2022 - 05.2024

B.E - Electronics and Telecommunication Engineering

SVERI'S College of Engineering

Similar Profiles

CREATE PROFILE
Snehal Godge