Summary
Overview
Work History
Education
Skills
Certification
Languages
Timeline
Generic

Subhash Sunkari

Hyderabad

Summary

Results-driven Data Engineer skilled in SQL, Python, and Azure Data Factory. Proven ability to develop and implement complex ETL processes, facilitating efficient data transformation and cloud solutions.

Overview

4
4
years of professional experience
1
1
Certification

Work History

Specialist Data Engineer

LTIMindtree
Hyderabad
03.2024 - Current
  • Performing various data validations using databricks.
  • Created the notebooks for calculations and workpapers.
  • Created secret scopes, Azure key vaults for storing environment specific variables.
  • Created jobs and shared same job id's with backend team for the triggering purpose.
  • Loaded the data to calculation layer by partition on specific funds.
  • Created wrapper notebook and used the same for running parallel jobs specific to client.
  • Created config files for the client specific business areas.
  • Working with Microsoft Azure Storage, SQL Server, Data Factory, Databricks.
  • Creating ADF Triggers to run the Pipeline.
  • Working on Azure Data Factory.
  • Designing & developing code, testing, debugging & trouble shooting.
  • Requirement gathering.
  • Hands on with Code push using Azure Repo.
  • Have used different types of transformations in PySpark.
  • Attended sprint meetings to gather business requirements.
  • Attended daily Scrum meetings, scrum retros and scrum review meetings with client.
  • Client: EY

Senior Associate

Synechron
Hyderabad
04.2023 - 01.2024
  • Migrating the business data from different layers like Raw, Harmonized and Curated using ADF and ADB.
  • Loading and transforming the data from the CSV, Parquet, JSON format into the table format.
  • Also responsible for encrypting the PII data while the data transformation stage.
  • Worked with Microsoft Azure Storage, SQL Server, Data Factory, Databricks.
  • Creating ADF Triggers to run the Pipeline.
  • Working on Azure Data Factory.
  • Designing & developing code, testing, debugging & trouble shooting.
  • Requirement gathering.
  • Hands on with Code push using Azure Repo.
  • Designed and developed Batch processing and real-time processing solutions using Azure Data Factory, Databricks clusters, and Stream Analytics.
  • Developed Spark Scala scripts and User-Defined Functions (UDFs) to efficiently read data from Azure Blob storage in Azure Databricks.
  • Implemented Azure Data Factory (ADF) extensively for ingesting data from different source systems like relational and unstructured data to meet business functional requirements.
  • Implemented end-to-end data extraction, transformation, and loading (ETL) processes using Azure Data Factory (ADF) and Azure HDInsight.
  • Created Linked services to land the data from different sources to Azure Data Factory.
  • Migrated on-premises data (Oracle/Teradata) to Azure Data Lake Store using Azure Data Factory.
  • Client: Everest Re insurance Company

Consultant

Capgemini
Hyderabad
07.2021 - 02.2023
  • Developed multiple reusable components for moving data from ADLS to SQL and SQL to ADLS.
  • Developed pipelines for connecting to different SFTP servers and pulling data to ADLS through ADF.
  • Configured Logic Apps for sending email notifications through ADF and handling error logs.
  • Created Triggers like Scheduled and Event based triggers.
  • Created Data Factory Pipelines using Copy Data Activity, Get metadata Activity Lookup Activity, Stored Procedure Activity. To generate underlying data for the reports and to export cleaned data from CSV, Text file to Datawarehouse.
  • Experience in Implementing Pipelines, datasets in Azure data Factory.
  • Experience on Migrate on-premises data source to load the data into staging area in blob storage.
  • Worked with various Transformations like Lookup, Merge, Sort, Multicast, Conditional Split and Data Flows.
  • Implemented Copy activity in Azure Data Factory Pipeline Activities for On-cloud ETL processing Designing and deployment of reports for the end user request using web interface.
  • Involved in deploying code from Dev to QA and from QA to UAT using Azure DevOps.
  • Developed core application of project called Dr Pepper weekly application.
  • Developed ETL solutions using Spark SQL in Azure Databricks for data extraction, transformation and aggregation from multiple file formats and data sources for analyzing & transforming the data.
  • Extensive experience in Analyzing, Developing, Managing various stand-alone client-server enterprise applications using Python.
  • Experience in Analyzing in data using Python, SQL, Microsoft Excel, PySpark, SparkSQL for Data mining and Data Cleansing.
  • Proficient in SQL Databases with Python.
  • Client: Coca-Cola

Sr Software Engineer

Ubique Systems
03.2021 - 05.2021
  • Involved in requirements gathering and creating Scope of Work documents other such activities.
  • Interaction with project manager, development team for requirement analysis.
  • Map source system data elements to target system and develop, test, and support extraction, transformation, and load processes.
  • Extracting the data from different data sources load that data into staging area in Azure Data Lake.
  • Developed Pipelines in Azure Data factory using Multiple Activities.
  • Implemented Copy activity in Azure Data Factory Pipeline Activities for On-cloud ETL processing Designing and deployment of reports for the end user request using web interface.
  • Configured Logic Apps for sending email notifications through ADF and handling error logs.

Education

Bachelor of Technology - Computer Science Engineering

JNTU
Hyderabad
11-2015

Skills

  • SQL and MS SQL Server
  • Python programming
  • Spark SQL and Azure Data Factory
  • Azure Databricks and Synapse
  • Azure Data Lake Storage Gen2
  • Microsoft Fabric

Certification

  • Microsoft Certified Fabric Data Engineer Associate
  • Microsoft Certified Data Engineer Associate

Languages

English
First Language

Timeline

Specialist Data Engineer

LTIMindtree
03.2024 - Current

Senior Associate

Synechron
04.2023 - 01.2024

Consultant

Capgemini
07.2021 - 02.2023

Sr Software Engineer

Ubique Systems
03.2021 - 05.2021

Bachelor of Technology - Computer Science Engineering

JNTU

Similar Profiles

CREATE PROFILE
Subhash Sunkari