Summary
Overview
Work History
Education
Skills
Awards
Certification
Timeline
Generic

ADITYA KAKARAPARTHI

Hyderabad,Telangana

Summary

Azure Data Engineer with 3 years of experience in creating / building / deploying / managing data pipelines using Azure Data services such as Azure Databricks, Azure Datafactory, ADLS Gen2, Azure Synapse Analytics complemented by proficiency in Kafka. Actively seeking a data engineer role to leverage my expertise and enhance my skills.

  • Experienced in designing, developing, and maintaining robust data pipelines with a focus on scalability and efficiency.
  • Highly proficient in utilizing Azure Databricks, Azure Data Factory, ADLS Gen 2, PySpark, SQL, Kafka, and other technologies for developing end-to-end ETL pipelines.
  • Demonstrated expertise in implementing data ingestion solutions for diverse sources, including both batch and streaming data, ensuring reliability and optimal performance.
  • Hands-on experience with metadata-driven data platform to handle pipelines with real-time Spark Structured streaming from Kafka, ADLS, Blob and autoloader.
  • Handled ingestion of data of more than 100TB from Apache Kafka into ADLS Gen2 in delta format.
  • Executed data movement from various sources having different file formats, like CSV, JSON, AVRO and PARQUET using Auto loader to migrate historical and seeding data.
  • Experienced in performance optimization and troubleshooting, with a proven track record of optimizing job configurations to improve efficiency and reduce operational costs.
  • Collaborative team player, dedicated to delivering high-quality data solutions aligned with business objectives.

Overview

3
3
years of professional experience
1
1
Certification

Work History

Data Engineer

Accenture
07.2021 - Current
  • Worked in data platform to handle pipelines with real-time Spark Structured Streaming from connectors like Kafka, Delta, ADLS, Blob, end-to-end implemented on Databricks medallion design for global clothing and accessories retail client to handle data in tune of 100s of TBs
  • Played key role in developing and managing large-scale data engineering platform for global fashion retailer, involving both batch and stream data processing
  • Developed end-to-end delta-to-delta streaming pipelines to handle both batch (scheduled) & real-time data processing
  • Designed and developed data pipelines and orchestration workflows utilizing Azure Databricks, Azure Data Factory, and Azure Data Lake Gen2
  • Implemented optimization strategies in data pipeline orchestration, resulting in significant 20% cost savings
  • Developed data ingestion pipelines through Kafka streaming as well as file based history data ingestion from CSV, AVRO and Parquet files
  • Conducted stress test on pipelines to decide on cluster configurations required on critical pipelines to maintain SLA during Black Friday week when there is spike of over 10 times usual data volume
  • Implemented Azure Data Factory pipelines for orchestrating workflows, facilitating seamless movement and loading of data from various sources to designated targets.

Education

Bachelor's - Computer Science and Engineering

Saveetha School of Engineering
Chennai, India
06.2021

Skills

    Programming Languages: Python, SQL

    Data Warehousing: ADLS

    Big Data Frameworks: Azure Databricks, Azure Data Factory, Apache Kafka, PySpark, SQL Server

    Orchestration and Source Control: Github, Jenkins

Awards

Sparkling Star Award, 05/2024, Received Sparkling Star Award for consistently ensuring smooth operations and timely delivery of work items, while playing a key role in mentoring junior team members.

Certification

  • Microsoft Certified: Azure Fundamentals
  • Microsoft Certified: Azure Data Fundamental

Timeline

Data Engineer

Accenture
07.2021 - Current

Bachelor's - Computer Science and Engineering

Saveetha School of Engineering

Similar Profiles

CREATE PROFILE
ADITYA KAKARAPARTHI