Mahesh Gaddam
Kunrnool
Summary
Lead Consultant with 10+ years of overall IT experience, including 3 years of hands-on Databricks and Azure Data Factory work on Azure Cloud. Currently leading data engineering delivery on a banking Enterprise Data Platform engagement, combining hands-on SQL and dimensional-modelling work with authorship of High-Level Design and technical decision documentation for a Centralised Feature Store spanning Unity Catalog, Delta Lake, and Lakebase. Holds production support responsibility across a TB-scale platform ingesting from 100+ source systems, and leads a team of approximately 10, managing delivery cadence, task delegation, and engineer mentoring. Engages directly with client architecture and data leadership through formal design sign-off cycles, operating at both the engineering and architecture layers of a modern lakehouse platform.
Overview
1
1
Certification
11
11
years of professional experience
Work History
Lead Consultant
Hoonar Tekwurks Private Limited
Pune, India
02.2026 - 09.2026
- Lead data engineering delivery on a banking Enterprise Data Platform engagement on Databricks and Azure Cloud, ingesting from 10+ source systems at TB-scale.
- Manage a data engineering team of approximately 10, delegating workstreams and coordinating delivery cadence across architecture, ingestion, and data-quality tracks.
- Mentor and guide engineers on Databricks, Unity Catalog, and Azure Data Factory practices within the team.
- Hold production support responsibility for the EDP Databricks environment, including triage and root-cause diagnosis of data-lineage and pipeline-logic defects.
- Authored the High-Level Design for a Centralized Feature Store on Databricks, defining architecture across Unity Catalog, offline/online feature stores, Lakebase, and Model Serving, refined through iterative stakeholder review.
- Owned and maintained a Key Decision Document register capturing platform-defining technical decisions — source-of-record strategy, mandatory processing framework, and CI/CD standard — improving architectural traceability.
- Drove resolution of open architecture questions, including online/offline sync cadence and feature-computation scope boundaries, through direct engagement with client data architecture stakeholders.
- Evaluated Delta UniForm against Apache Iceberg for offline feature storage, identifying a platform-level sync limitation ahead of a formal technology recommendation.
- Designed an integration pattern for exposing governed feature data to a downstream CRM platform via Lakebase, Feature Serving, and Azure API Management.
- Reverse-engineered Azure Data Factory pipeline orchestration and Databricks notebook logic to reconstruct end-to-end lineage for a banking source system's ingestion layers.
- Authored a Design and Implementation Knowledge Transfer document structuring a shared ingestion/audit framework and layer-by-layer pipeline architecture, strengthening platform continuity for the team.
- Diagnosed a production data-lineage gap between a liabilities-scoped account view and its underlying fact table, isolating a missing key mapping from a query-level defect.
- Reviewed transformation logic inside a metadata-driven Azure Data Factory ingestion framework, identifying timezone-handling defects, join-driven fan-out risk, and a row-count validation check silently excluding in-flight records.
- Developed advances MIS reporting SQL using named CTEs, QUALIFY-based deduplication, and fact-first filtering, paired with a dedicated data-quality test suite.
- Built and extended governed dimensional views on Unity Catalog using SCD Type 2 and multi-source UNION ALL patterns, adding new attributes with type-safe handling across inconsistent source schemas.
- Reviewed a Unity Catalog column-masking pipeline, identifying security and code-quality gaps and strengthening governance posture.
- Engaged directly with a client Chief Data Officer and lead architect through formal design and sign-off review cycles spanning LDM and HLD deliverables.
- Identified an unresolved conflict between two finance-reporting requirements with no assigned ownership, surfacing it for stakeholder resolution.
Technical Manager
Visionet Systems Pvt Ltd
Bangalore
06.2022 - 09.2025
- Led end-to-end Teradata-to-Azure migration assessment and workload classification strategy.
- Owned design and governance of source-to-target mapping documents for migration.
- Designed and developed a silver-to-gold transformation framework for analytics datasets.
- Acted as hands-on technical lead for production Databricks Spark pipelines.
- Implemented governance, security, and cost optimisation through tooling and tuning.
- Led an 8-member engineering team for delivery, mentoring, and cutover.
- Databricks Spark pipelines using medallion architecture, translating STTM-defined logic into Scalable, optimised cloud-native solutions.
- Re-engineered 100+ Teradata scripts into scalable Azure Data Factory pipelines and Databricks Spark notebooks, reducing operational complexity by ~40%.
- Built automation frameworks using Bash and Python to analyze legacy scripts, extract Metadata, identify dependencies, and accelerate migration readiness by 70-90%.
- Established data reconciliation, performance benchmarking, and validation frameworks to Ensure accuracy, data parity, and SLA compliance post-migration.
- Implemented governance, security, and cost optimization strategies using Unity Catalog
- Cluster tuning, workload isolation, and optimized job orchestration.
- Led an 8-member engineering team, driving delivery, mentoring engineers, and coordinating
Data Engineer
AstraZeneca India Pvt Ltd
Chennai
06.2019 - 06.2022
- Developed and supported Spark ELT pipelines for enterprise analytics platforms.
- Implemented layered data lake architecture and ingestion patterns on AWS (S3, EMR
- Athena, Redshift).
- Monitored and troubleshot Airflow workflows and production data issues to ensure SLA Compliance.
Lead Data Engineer
Optimum Infosystem Pvt Ltd
Chennai
12.2016 - 05.2019
- Architected and built an Azure-native credit and collections data platform using Azure
- Databricks (Spark), Lakeflow Declarative Pipelines, Azure Data Factory, ADLS Gen2, Delta Lake
- And Unity Catalog, processing millions of retail and MSME lending records in a regulated
- Banking environment.
- Designed secure, incremental ingestion pipelines using Azure Data Factory and Databricks
- Auto Loader, sourcing data from enterprise MDM systems, loan and customer master
- Platforms, and external AWS S3, with secrets managed via Azure Key Vault and service
- Principals.
- Implemented Bronze-Silver-Gold medallion architecture with complex credit, delinquency
- (DPD), settlement, write-off, and legal vs agency allocation logic using Spark SQL and PySpark
- Reducing end-to-end processing latency by 30-40%.
- Enforced enterprise-grade data quality and schema governance using Lakeflow Expectations
- Delta constraints, and Unity Catalog, enabling consistent, auditable datasets for risk, finance
- And regulatory reporting.
- Delivered Gold-layer Customer 360 and credit exposure datasets in Delta Lake, supporting
- Underwriting analytics, credit scoring, portfolio monitoring, and downstream Azure-based
- Integrations.
- Operationalized pipelines using Databricks Workflows and Lakeflow Jobs, achieving 99%+
- Reliability through automated retries, audit logging, SLA monitoring, and idempotent design.
- Optimized Azure platform performance and cost through Databricks cluster tuning, autoscaling
- Job clusters, optimized storage layouts, and workload isolation, aligning with cloud
- FinOps best practices.
- Client: Standard Chartered
- Project: Cloud Data Platform Engineering
Software Engineer
Innova Solutions Pvt Ltd
Chennai
11.2015 - 11.2016
- Developed 15+ Informatica PowerCenter mappings with complex transformations including
- Lookups, aggregators, joiners, and SCD Type 1 & 2.
- Optimized ETL workflows through performance tuning, improving execution time by up to 40%.
- Optimum Infosystem Pvt Ltd - Chennai
- Lead Data Engineer
- Client: Standard Chartered
- Project: Cloud Data Platform Engineering
Education
Bachelor of Technology -
Sri Venkateswara University
01-2013
Skills
Data Engineering & Processing
- Azure Databricks
- Apache Spark (PySpark)
- Spark SQL
- Delta Lake
- Solace & Kafka
Databricks Platform & Orchestration
- Lakehouse Architecture, Medallion (Bronze / Silver / Gold)
- Spark Lakeflow Declarative Pipelines, Lakeflow Jobs Declarative
- Automation Bundles (DABs)
- Unity Catalog
Cloud Orchestration & Data Services
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Azure Key Vault
Monitoring & Version Control
- Azure DevOps (ADO) – Repos, CI/CD Pipelines
- GitHub
- Grafana
Data Architecture & Modelling
- Dimensional modelling
- Incremental & CDC Frameworks
Certification
- Databricks Certified Data Engineer - Professional, Databricks
- Databricks Certified Data Engineer - Associate, Databricks
Accomplishments
- Best Employee Award, Visionet Systems, 2025-01-01.
- Organizer - Technical Event (Techfest), Visionet Systems Pvt Ltd.
- Team Synergy Award(2026-06-25), Hoonartek.
Timeline
Lead Consultant
Hoonar Tekwurks Private Limited
02.2026 - 09.2026
Technical Manager
Visionet Systems Pvt Ltd
06.2022 - 09.2025
Data Engineer
AstraZeneca India Pvt Ltd
06.2019 - 06.2022
Lead Data Engineer
Optimum Infosystem Pvt Ltd
12.2016 - 05.2019
Software Engineer
Innova Solutions Pvt Ltd
11.2015 - 11.2016
Bachelor of Technology -
Sri Venkateswara University