Summary
Overview
Work History
Education
Skills
Certification
Timeline
Hi, I’m

Aditya Sharma

Data Engineer
Pune
Aditya Sharma

Summary

Highly skilled and experienced Data Engineer with a proven track record of designing, developing, and implementing innovative data solutions. Strong expertise in data modeling, ETL processes, and database management. Seeking a challenging role where I can utilize my technical skills to drive business growth and success through data-driven insights.

Overview

4
years of professional experience
4
years of post-secondary education
4
Certifications

Work History

LTIMINDTREE

Data Engineer
02.2020 - Current

Job overview

  • Microsoft Azure - DP-203, DP-900 certified Azure data engineer associate having 4 years of experience
  • Experience in real time Hadoop/Big Data technologies for data storage, querying, processing and analysis
  • Ability to handle large amounts of Structured and Semi-structured data processing
  • Developed PowerShell scripts to extract files from a remote server within the same domain
  • Implemented dynamic code to pass path references and retrieve files from specified locations
  • Ensured secure and efficient transfer of files between servers using PowerShell commands.
  • Using various python libraries such as Pandas, Datetime, OS, developed Python automation scripts to extract failure rows from a logs txt file
  • Implemented Python code to find dynamic path to the file location and extract the CSV file from source
  • Converted and extracted data into CSV format and stored it in a specified location
  • Implemented automated email functionality to send the CSV file to stakeholders
  • Implemented data preprocessing pipelines to clean and prepare extracted data for analysis
  • Developed production-ready ELT/ETL Pipelines, Datasets & Triggers and versioned them using CI/CD pipelines
  • Designed & Built Metrics Pipelines to capture high-level data statDevelop and manage data transformation processes using ADF activities such as data wrangling, data flows, and mapping data from source to destination
  • Integrate ADF pipelines with other Azure services like Azure Synapse Analytics, Azure Databricks, and Azure Machine Learning for end-to-end data integration and analytics solutions
  • Developed and maintained stored procedures, views, and functions in SQL server to optimize data extract, transform and load (ETL) processes
  • Experienced in importing various type of file format such as csv, parquet, orc and avro using Azure Data Factory
  • Having good knowledge on MySql
  • Familiar with using Spark DataFrame persistency and caching mechanisms to optimize query performance and minimize data processing overhead
  • Having good understanding of complex data to flattened file such as xml and Json
  • Having good understanding of RDD and DataFrames
  • Able to read and write different type of file format using spark.read and spark.write
  • Knowledge of Spark RDD best practices in data engineering and data science domains, such as data preprocessing, feature engineering
  • Familiarity with Spark DataFrame schema and data type operations, such as adding, renaming, and dropping columns, casting data types, and handling null value
  • Generated, maintained and analyzed Azure monitoring dashboards, reports, and trends, minimizing customer pain points
  • Created data transfer pipelines between Azure services and on-premises systems, resulting in a 95% network throughput increase
  • Design and implement data storage solutions using Azure services such as Azure SQL Database, Azure Cosmos DB, and Azure Data Lake Storage
  • Optimize data processing and storage for performance and cost efficiency
  • Automated routine tasks using Python scripts, increasing team productivity and reducing manual errors.
  • Established robust monitoring processes to proactively detect system anomalies or performance bottlenecks before they impact users or critical operations.

Education

Rajiv Gandhi Proudyogiki Vishwavidyalaya
Bhopal

Bachelor of Engineering from Computer Science
04.2015 - 04.2019

Skills

Data Warehousing

undefined

Certification

Microsoft Certified (DP 203): Data Engineer Associate

Timeline

Microsoft Certified (DP 203): Data Engineer Associate

02-2024

Databricks Lake House Fundamentals

12-2023

Microsoft Certified (DP 900): Data Fundamentals

11-2023

Microsoft Certified: Azure Fundamentals

10-2023

Data Engineer

LTIMINDTREE
02.2020 - Current

Rajiv Gandhi Proudyogiki Vishwavidyalaya

Bachelor of Engineering from Computer Science
04.2015 - 04.2019

Similar Profiles

CREATE PROFILE
Aditya SharmaData Engineer