Akash Tyagi

Gurugram

Summary

Senior Data Engineer with 9+ years of experience in Big Data, Cloud, and Data Engineering, delivering enterprise-grade solutions for leading clients including Dunnhumby, Tata Digital, RBL Bank, Godfrey Phillips India and Maruti Suzuki. Proficient in Hadoop, Spark, Hive, PySpark, Python, SQL, and workflow management tools including Azure Data Factory (ADF), Apache Airflow, and Apache NiFi. Skilled in designing ETL/ELT pipelines, data lakes, and data warehouses across Azure and Google Cloud Platform (GCP). Hands-on expertise in Azure Data Lake Storage (ADLS), Google Cloud Storage (GCS), and large-scale data migration, integration, and real-time streaming solutions. Certified in cloud data engineering with a proven record of enabling scalable, high-performance, and business-driven data platforms.

Overview

Languages

Certification

years of professional experience

Work History

Senior Data Engineer

Dunnhumby

12.2023 - 08.2026

Led a cross-functional initiative to streamline deployment workflows by consolidating PIE (custom client logic) and CDS Deng (data engineering workflows), reducing deployment time by 40%, and accelerating client delivery.
Implemented automated data quality checks to reduce manual intervention and improve fault tolerance, decreasing severity tickets by 80%, and significantly enhancing system reliability.
Built scalable ETL pipelines to process and transform large-scale customer transaction data for top retail clients, reducing data preparation effort, and enabling faster analytics for product, and data science teams.
Built a PnP Data Mart that centralised client data, enabling the ADS team to analyse datasets with greater accuracy and efficiently back-trace records, improving overall data reliability.
Developed scalable data processing pipelines orchestrated with Airflow, leveraging Dataproc clusters to process large-volume datasets efficiently on GCP.
Designed and deployed a GenAI-driven multi-agent investigation platform for infra and data issues across hybrid cloud, using Vertex AI, MCP, and Dialogflow CX, reducing manual intervention and accelerating incident resolution.

Senior Consultant

Celebal Technology

03.2020 - 12.2023

Developed an analytical tool that improved data visibility and analytics, enabling teams to track key metrics (GMV, NMV, expenses), and make faster, data-driven strategic decisions across multiple verticals.
Leveraged Apache Spark operations to implement data transformation logic, successfully processing and delivering cleaned data to target locations.
Engineered a near real-time (streaming job) pipeline for raw data ingestion using REST API, storing customer journey data in Delta Lake to enhance analytics and business insights.
Designed and implemented a robust framework to automate data ingestion from the datalake to the lakehouse, enhancing data pipeline efficiency and accuracy.
Successfully implemented a data integration/migration solution using Apache NiFi, effectively streamlining and managing the flow of data across systems.
Designed and implemented a data quality management system that enabled senior leaders at Tata Digital to assess and improve data quality across all brands, enhancing the accuracy of enterprise-wide analytics.
Architected and engineered a data quality management system that provided senior leaders at Tata Digital with a comprehensive analysis of data quality across all brands.
Designed and developed a Marketing DataMart to enable efficient campaign execution and improved analytics for diverse initiatives.
Designed a cloud-native data lake and ingestion framework on GCP using GCS, DataProc, and BigQuery to standardise onboarding, validation, and delivery of analytics-ready datasets across multiple sources.
Built a scalable API-driven data ingestion platform on GCP to process nested JSON data and deliver analytics-ready marketing datasets in BigQuery for campaign performance analysis.

Data Engineer

Mindtrail Technologies

03.2017 - 02.2020

Developed and orchestrated a data ingestion pipeline to efficiently load data from multiple sources into the data lake, improving the reliability and scalability of data integration.
Developed a data integration and migration solution utilising Azure Data Factory, enabling seamless processing across OLTP and data warehousing environments.
Collaborated with SQL Server databases, efficiently managing SQL queries and developing stored procedures to ensure data normalisation, and integrity.
Developed shell scripts for efficient data ingestion into Hadoop clusters, and collaborated with Cloudera Manager to monitor system health and optimise cluster performance.
Utilised Sqoop to efficiently import and export data between HDFS and Hive, enabling seamless data transfer across systems.
Engaged in data transformation and modelling of raw datasets, refining them for utilisation across multiple Power BI dashboards that provide enhanced business insights for senior management analysis.
Transformed and modelled raw datasets for Power BI dashboards to deliver actionable business insights for senior management.

Education

Btech. - ECE

The NorthCap University

Gurgaon, HR

2017

Skills

AI Tools: Vertex AI, MCP, LangGraph, RAG, DialogFlowX
Big Data & Distributed Systems: Hadoop, Spark, and Cloudera
Data Processing Frameworks: MapReduce, Databricks, Kubernetes
Data Integration and Workflow Orchestration: NiFi, Sqoop, Apache Airflow, Azure Data Factory (ADF), and Oozie
Cloud Platforms: ADF, Databricks, Event Hub, Logic Apps

Data governance and cataloguing: Databricks Unity Catalogue, data quality management
Data Warehousing & Storage: Azure Data Lake Storage (ADLS), Delta Lakehouse, SQL Data Warehouse
Programming and Scripting: SQL, Python, Java, Shell Scripting
File formats: ORC, Parquet, Avro, JSON, and CSV