Summary
Overview
Work History
Education
Skills
Timeline
Generic
Shristi Pradhan

Shristi Pradhan

Bengaluru

Summary

Experienced Senior Data Engineer with 9 years of expertise in designing, building, and optimizing scalable, high-performance data pipelines for enterprise applications. Proficient in Azure Databricks, PySpark, SQL, Python, Snowflake, and Azure Data Factory (ADF), with a strong focus on delivering robust, end-to-end data engineering solutions. Adept at leading teams and managing full project lifecycles—from requirement gathering and sprint planning to client interaction and solution delivery. Experienced in implementing CI/CD pipelines using Azure DevOps, enhancing deployment efficiency and operational reliability. Demonstrated success in leading data migration initiatives, automating complex workflows, and driving cross-functional collaboration to support data-driven business decisions and long-term scalability.

Overview

9
9
years of professional experience

Work History

Software Development Engineer III

Sigmoid Analytics
Bengaluru
03.2025 - Current
  • Leading a team of 4 Data Engineers, providing technical guidance and ensuring timely delivery of high-quality data solutions.
  • Developing end-to-end data engineering frameworks using Azure Data Factory (ADF) and Azure Databricks (ADB) to streamline data ingestion, transformation, and storage processes.
  • Collaborating directly with clients and stakeholders for requirement gathering, sprint planning, and aligning technical solutions with business objectives.
  • Designing scalable, reusable data pipelines using PySpark and SQL, enabling seamless data flow across systems and supporting complex analytics use cases.
  • Spearheading efforts in CI/CD implementation for data workflows using Azure DevOps, reducing release cycle time and increasing deployment reliability.
  • Establishing best practices in data quality, version control, and documentation, fostering a robust data engineering culture within the team.
  • Driving code reviews, technical grooming, and regular knowledge-sharing sessions to uplift team capabilities.
  • Working across cross-functional teams including BI, and Product teams to support integrated data solutions and business insights.

Senior Data Engineer

EY GDS
Bengaluru
07.2021 - 03.2025
  • Designed and developed high-performance ETL pipelines leveraging Azure Databricks, PySpark, and SQL for big data processing, ensuring efficient handling of terabyte-scale datasets.
  • Led a data migration from Alteryx to Snowflake, improving data processing efficiency by 25% and enhancing scalability.
  • Developed PySpark-based transformation frameworks, leveraging distributed computing to optimize job execution time.
  • Built highly optimized SQL queries for real-time analytics and fast query performance in Snowflake and Azure SQL.
  • Automated Azure Data Factory (ADF) pipelines, ensuring seamless data orchestration, transformation, and movement across cloud platforms.
  • Implemented CI/CD pipelines for data workflows using Azure DevOps, streamlining code deployment, reducing errors, and ensuring faster integration of changes into production environments.
  • Optimized SnowflakeSQL queries and schemas, reducing query execution time and enhancing performance.
  • Mentored and led a team of junior data engineers, improving skillsets and ensuring successful project deliveries.
  • Collaborated closely with business and analytics teams to define data requirements and implement solutions that drive decision-making.

Senior Project Engineer

Wipro Limited
Pune
06.2016 - 06.2021
  • Designed and developed big data solutions using Azure Databricks, PySpark, and SQL to handle large-scale data processing efficiently.
  • Optimized PySpark performance, leveraging RDD, DataFrame API, and caching techniques to enhance data pipeline efficiency.
  • Built and maintained ETL workflows in Talend and Sqoop, ensuring seamless data ingestion and integration from multiple sources.
  • Developed and optimized Hadoop-based data pipelines using HDFS, Hive, and Snowflake, improving query performance and storage efficiency.
  • Automated data ingestion jobs using Autosys, ensuring timely and reliable data availability.
  • Designed and implemented partitioned Hive tables and optimized Snowflake queries to reduce storage costs and boost performance.
  • Collaborated with cross-functional teams to define data strategies and implement best practices for scalable data solutions.

Education

Bachelor of Technology - Computer Science - Computer Science

Galgotias University
Greater Noida
05-2016

Senior Secondary - PCM -

Tagore Public School
Prayagraj
05-2011

High School -

Maharishi Vidya Mandir
Prayagraj
05-2009

Skills

  • Azure databricks
  • Azure Data Factory
  • PySpark
  • SQL
  • CI/CD implementation
  • Data pipeline design

Timeline

Software Development Engineer III

Sigmoid Analytics
03.2025 - Current

Senior Data Engineer

EY GDS
07.2021 - 03.2025

Senior Project Engineer

Wipro Limited
06.2016 - 06.2021

Bachelor of Technology - Computer Science - Computer Science

Galgotias University

Senior Secondary - PCM -

Tagore Public School

High School -

Maharishi Vidya Mandir
Shristi Pradhan