Deepesh Saxena
Senior Lead Data Engineer
Noida,UP
Summary
Senior Lead Data Engineer (Technical Lead) leading Azure Databricks, Azure Data Factory, and Lakehouse delivery for enterprise-scale data platforms. Designs batch and real-time pipelines with PySpark, Apache Spark, Kafka, and Delta Lake, processing 100M+ records per day and 5–10M+ JMS messages daily. Leads distributed teams, drives architecture decisions, and improves ingestion reliability, source onboarding, and downstream data availability.
Overview
4
4
Certifications
19
19
years of professional experience
Work History
Senior Lead Data Engineer (Technical Lead)
Accenture
02.2022 - Current
- Serve as hands-on technical lead for a BI & Data Engineering team of 12+ engineers, owning architecture, design decisions, and complex implementations.
- Designed and implemented end-to-end real-time streaming pipelines ingesting 5–10M+ JMS messages/day into Azure-based Lakehouse using Kafka and Azure Event Hub.
- Built scalable real-time stream processing solutions using Apache Flink on Azure Kubernetes Service (AKS), enabling high-throughput data transformation and routing.
- Enabled fault-tolerant streaming with Dead Letter Event Hubs, ensuring 99.9%+ ingestion reliability and zero data loss.
- Improved ingestion reliability to 99.9%+ success rate by enforcing message-size validation, schema standards, and retry mechanisms.
- Developed scalable batch ingestion frameworks processing TB-scale datasets from JDBC, SFTP, REST, Snowflake, and PostgreSQL with support for incremental, CDC, and full-load patterns.
- Reduced downstream data availability by 40–50% by implementing Delta Lake-based Medallion architecture (Bronze/Silver/Gold) for optimized data processing.
- Accelerated onboarding of new data sources by 30% through development of reusable ingestion framework and standardization of infrastructure provisioning.
- Key Achievements: -
- 1. Awarded ACE Award (Mar ’26 & Mar ’24) – Accenture’s highest recognition for technical excellence and delivery impact. – Recognized individual by leadership for successfully stabilizing and scaling mission-critical data pipelines.
- 2. Led critical platform initiatives for Atlas Data Hub, contributing to an “Exceptional” Internal Audit rating recognized by senior leadership and client CEO, including 360+ Flink pipeline upgrades, ADO workload identity enablement, and enterprise certificate renewals.
Senior BI Consultant
Tata Consultancy Services
10.2016 - 02.2022
- Led BI teams of up to 8 members, delivering enterprise BI and analytics solutions for global clients.
- Served 1 year onsite in Leiden, Netherlands, as primary BI consultant and client liaison, facilitating effective communication between stakeholders.
- Gathered and translated business requirements into robust data models and BI solutions.
- Designed and implemented a scalable data pipeline using Azure Data Factory and Databricks, handling data ingestion, transformation and loading.
- Maintained and enhanced large-scale enterprise DWH, ensuring operational stability and scalability to meet evolving business needs.
Senior BI Consultant / Tech Manager
Tata Consultancy Services
09.2014 - 09.2016
- Led 7-member team to manage and deliver BI solutions across geographies, ensuring alignment with business objectives.
- Conducted effort estimation, feasibility analysis, and delivery planning to drive project success.
- Enforced coding standards and testing practices to maintain high-quality software releases.
Data & BI Engineer
Tata Consultancy Services
10.2011 - 09.2014
- Delivered BI and ETL solutions onsite in Coventry, UK for 1 year.
- Automated operational tasks using Shell scripting and optimized SQL-based data processing.
- Automated operational tasks with Shell scripting, streamlining SQL-based data processing.
- Conducted impact analysis, supported UAT, and resolved production issues.
ETL Developer
HCL Technologies
05.2010 - 08.2011
- Developed and enhanced ETL components to improve data integration processes.
- Performed unit testing and created documentation to ensure long-term maintainability.
- Identified data gaps and recommended solutions to enhance data quality.
Programmer Analyst
Cognizant Technology Solutions
11.2007 - 04.2010
- Developed ETL code for data warehouse initiatives, enhancing data integration processes.
- Configured and maintained testing environments, including virtual machines and containerized deployments, supporting reliable software testing for 10+ engineers.
- Led cross-functional teams in Agile environments to enhance project delivery and efficiency.
- Implemented best coding practices, resulting in improved code quality and maintainability.
Education
Bachelor of Technology (B. Tech.) -
IET
Lucknow, Uttar Pradesh, India01-2007
Skills
Azure Databricks
Azure Data Factory
Azure DevOps
Terraform
SQL
Streaming pipeline design
Lakehouse architecture
Medallion architecture
Data quality management
Change data capture
Team management
Stakeholder management
Mentoring
Delivery ownership
Architecture decisions
Certification
Databricks Certified Data Engineer Professional – Mar 2026
AWARDS & RECOGNITION
ACE Award – Accenture (Mar ’26 & Mar ’24), Special Achievement Award – TCS (Leiden, Netherlands), Multiple TCS Awards: Star of the Quarter, Beyond Excellence, On-the-Spot Awards
Timeline
Senior Lead Data Engineer (Technical Lead)
Accenture
02.2022 - Current
Senior BI Consultant
Tata Consultancy Services
10.2016 - 02.2022
Senior BI Consultant / Tech Manager
Tata Consultancy Services
09.2014 - 09.2016
Data & BI Engineer
Tata Consultancy Services
10.2011 - 09.2014
ETL Developer
HCL Technologies
05.2010 - 08.2011
Programmer Analyst
Cognizant Technology Solutions
11.2007 - 04.2010
Bachelor of Technology (B. Tech.) -
IET