Shristi Pradhan
Bengaluru
Summary
Experienced Senior Data Engineer with 9 years of expertise in designing, building, and optimizing scalable, high-performance data pipelines for enterprise applications. Proficient in Azure Databricks, PySpark, SQL, Python, Snowflake, and Azure Data Factory (ADF), with a strong focus on delivering robust, end-to-end data engineering solutions. Adept at leading teams and managing full project lifecycles—from requirement gathering and sprint planning to client interaction and solution delivery. Experienced in implementing CI/CD pipelines using Azure DevOps, enhancing deployment efficiency and operational reliability. Demonstrated success in leading data migration initiatives, automating complex workflows, and driving cross-functional collaboration to support data-driven business decisions and long-term scalability.
Overview
9
9
years of professional experience
Work History
Software Development Engineer III
Sigmoid Analytics
Bengaluru
03.2025 - Current
- Leading a team of 4 Data Engineers, providing technical guidance and ensuring timely delivery of high-quality data solutions.
- Developing end-to-end data engineering frameworks using Azure Data Factory (ADF) and Azure Databricks (ADB) to streamline data ingestion, transformation, and storage processes.
- Collaborating directly with clients and stakeholders for requirement gathering, sprint planning, and aligning technical solutions with business objectives.
- Designing scalable, reusable data pipelines using PySpark and SQL, enabling seamless data flow across systems and supporting complex analytics use cases.
- Spearheading efforts in CI/CD implementation for data workflows using Azure DevOps, reducing release cycle time and increasing deployment reliability.
- Establishing best practices in data quality, version control, and documentation, fostering a robust data engineering culture within the team.
- Driving code reviews, technical grooming, and regular knowledge-sharing sessions to uplift team capabilities.
- Working across cross-functional teams including BI, and Product teams to support integrated data solutions and business insights.
Senior Data Engineer
EY GDS
Bengaluru
07.2021 - 03.2025
- Designed and developed high-performance ETL pipelines leveraging Azure Databricks, PySpark, and SQL for big data processing, ensuring efficient handling of terabyte-scale datasets.
- Led a data migration from Alteryx to Snowflake, improving data processing efficiency by 25% and enhancing scalability.
- Developed PySpark-based transformation frameworks, leveraging distributed computing to optimize job execution time.
- Built highly optimized SQL queries for real-time analytics and fast query performance in Snowflake and Azure SQL.
- Automated Azure Data Factory (ADF) pipelines, ensuring seamless data orchestration, transformation, and movement across cloud platforms.
- Implemented CI/CD pipelines for data workflows using Azure DevOps, streamlining code deployment, reducing errors, and ensuring faster integration of changes into production environments.
- Optimized SnowflakeSQL queries and schemas, reducing query execution time and enhancing performance.
- Mentored and led a team of junior data engineers, improving skillsets and ensuring successful project deliveries.
- Collaborated closely with business and analytics teams to define data requirements and implement solutions that drive decision-making.
Senior Project Engineer
Wipro Limited
Pune
06.2016 - 06.2021
- Designed and developed big data solutions using Azure Databricks, PySpark, and SQL to handle large-scale data processing efficiently.
- Optimized PySpark performance, leveraging RDD, DataFrame API, and caching techniques to enhance data pipeline efficiency.
- Built and maintained ETL workflows in Talend and Sqoop, ensuring seamless data ingestion and integration from multiple sources.
- Developed and optimized Hadoop-based data pipelines using HDFS, Hive, and Snowflake, improving query performance and storage efficiency.
- Automated data ingestion jobs using Autosys, ensuring timely and reliable data availability.
- Designed and implemented partitioned Hive tables and optimized Snowflake queries to reduce storage costs and boost performance.
- Collaborated with cross-functional teams to define data strategies and implement best practices for scalable data solutions.
Education
Bachelor of Technology - Computer Science - Computer Science
Galgotias University
Greater Noida05-2016
Senior Secondary - PCM -
Tagore Public School
Prayagraj05-2011
High School -
Maharishi Vidya Mandir
Prayagraj05-2009
Skills
- Azure databricks
- Azure Data Factory
- PySpark
- SQL
- CI/CD implementation
- Data pipeline design
Timeline
Software Development Engineer III
Sigmoid Analytics
03.2025 - Current
Senior Data Engineer
EY GDS
07.2021 - 03.2025
Senior Project Engineer
Wipro Limited
06.2016 - 06.2021
Bachelor of Technology - Computer Science - Computer Science
Galgotias University
Senior Secondary - PCM -
Tagore Public School
High School -
Maharishi Vidya Mandir
Similar Profiles
MOHITH REDDYMOHITH REDDY
SDE 1 - Data Engineering at SIGMOID ANALYTICSSDE 1 - Data Engineering at SIGMOID ANALYTICS
Debapriya DasDebapriya Das
Director Data Science (Data Science Leadership) at Sigmoid AnalyticsDirector Data Science (Data Science Leadership) at Sigmoid Analytics
Neeraj GuptaNeeraj Gupta
Technical Lead at Sigmoid AnalyticsTechnical Lead at Sigmoid Analytics
Pranit SunkadPranit Sunkad
Assistant Manager - HR at SMFG India Credit Company LimitedAssistant Manager - HR at SMFG India Credit Company Limited
Prabhanjan PadakandlaPrabhanjan Padakandla
Client Services Lead Associate at Yahoo! Inc.Client Services Lead Associate at Yahoo! Inc.