Sudeshna Ghosh Dastidar
Azure Data Engineer At TCS
Bangalore
Summary
Senior Data Engineer with 15+ years of total IT experience, including 4+ years in Microsoft Azure Big Data Engineering. Strong expertise in Azure Databricks, Azure Data Factory, ADLS Gen2, Azure SQL, and Azure Lakehouse architectures. Deep understanding of Apache Spark internals (RDD, DataFrame, Dataset) with hands-on experience in PySpark, Spark SQL, batch and streaming pipelines. Proven ability to architect and implement scalable, high-performance data solutions, mentor engineering teams, and collaborate with cross-functional stakeholders to deliver governed, production-ready data platforms.
Overview
16
16
years of professional experience
4
4
Certifications
Work History
Azure Data Engineer / Data Architecture
Microsoft Limited
11.2022 - Current
- Architect and implement enterprise Azure data solutions using Azure Databricks, ADLS Gen2, Azure SQL, Azure Data Factory, and Cosmos DB.
- Designed and delivered Lakehouse-based ingestion and transformation architectures aligned to Medallion Architecture (Bronze, Silver, Gold).
- Led 2TB+ on-prem to Azure SQL Managed Instance migration, completed within 4-hour downtime window with zero production issues.
- Engineered real-time ingestion from AWS DynamoDB to Azure Cosmos DB (1M+ records)
- Developed secure cross-cloud data transfer pipelines handling 500GB+ daily
- Enhanced search reliability across cloud environments by migrating 10 Elasticsearch indices from AWS to Azure VM using snapshot/restore and remote reindexing, ensuring zero data loss.
- Participate in talent acquisition activities, including technical interviews and onboarding of new engineers.
- Collaborated as a solution architect collaborating with application, infra, and security teams
Tableau & Data Engineer
State Street Bank and Trust Company
07.2020 - 10.2022
- Standardized and modeled 23+ enterprise data sources using Python-based transformation frameworks.
- Applied dimensional and normalized data modeling techniques to support analytics and reporting.
- Optimized ETL pipelines, reducing processing time by 35%.
- Built Python automation to track reporting volumes across 100+ business units, saving 60+ hours per month.
- Partnered with business stakeholders to translate analytical requirements into scalable data models.
Database Developer
J.P. Morgan Chase Bank
06.2017 - 06.2020
- Delivered advanced SQL query optimization and troubleshooting across large enterprise databases.
- Resolved 35+ production defects, clearing a 193-ticket backlog in under 3 months.
- Designed automated ingestion pipelines processing 500K+ records per batch.
- Improved batch performance by 40% through SQL tuning and indexing strategies.
Data Warehouse Developer
Comcast Cable
07.2014 - 05.2017
- Delivered data warehouse and ETL enhancements on Oracle platforms
- Improved ETL runtimes by 40% through performance tuning
Database Developer
The Hartford
06.2013 - 06.2014
- Refactored complex PL/SQL procedures for reporting and analytics systems
- Supported large-scale enterprise data warehouse environments
Database Developer
Nedbank Limited
01.2010 - 05.2013
- Led migration of 1M+ banking records with zero data loss
- Delivered data integration solutions on Oracle and Teradata platforms
Education
Bachelor of Technology (B.Tech) - Information Technology
Haldia Institute of Technology
01.2009
Skills
Cloud & Azure: Azure Databricks, Azure Data Factory, ADLS Gen2, Azure SQL Database, Azure Cosmos DB, Azure Data Explorer
Big Data: Data Lakes, Lakehouse, Apache Spark
Databricks & Spark: Spark Architecture, RDD, DataFrame, Dataset, PySpark, Spark SQL, Databricks Notebooks, Workflows, Unity Catalog, SQL Warehouse, Serverless Compute
Programming: Python, PySpark, SQL, PL/SQL
SQL & Databases: Advanced SQL (complex queries, performance tuning, troubleshooting) Azure SQL, Oracle, SQL Server, PostgreSQL, Teradata
Data Engineering: Batch & Streaming Data Processing, Medallion Architecture (Bronze–Silver–Gold), Delta Lake, Parquet, Avro, JSON, CSV
Data Modelling: Dimensional Modeling (Star/Snowflake), 3NF, Data Warehousing Concepts
DevOps & CI/CD: Azure DevOps, GIT
Analytics: Tableau, Power BI
Accomplishments
- Delivered end-to-end Azure Lakehouse solutions using Databricks and Medallion Architecture.
- Reduced database licensing costs by 25% through Oracle-to-PostgreSQL modernization.
- Improved Spark and SQL performance by 30–40% through tuning and optimization.
- Recognized for mentoring engineers and contributing to capability-building initiatives.
Certification
Microsoft Azure Fundamentals (AZ-900)
Awards
TCS Gems Award – STAR OF THE MONTH (Mar & Oct 2025), TCS Gems Awards (2013, 2011) – Best Team, TCS Gems Awards (2011) – STAR TEAM, Special Initiatives
Timeline
Azure Data Engineer / Data Architecture
Microsoft Limited
11.2022 - Current
Tableau & Data Engineer
State Street Bank and Trust Company
07.2020 - 10.2022
Database Developer
J.P. Morgan Chase Bank
06.2017 - 06.2020
Data Warehouse Developer
Comcast Cable
07.2014 - 05.2017
Database Developer
The Hartford
06.2013 - 06.2014
Database Developer
Nedbank Limited
01.2010 - 05.2013
Bachelor of Technology (B.Tech) - Information Technology
Haldia Institute of Technology