Summary
Overview
Work History
Education
Skills
Timeline
background-images
Priyesh

Priyesh

Data Engineering & Architecture Leader
Gurgaon,HR

Summary

Data Engineering leader with 14+ years of experience, specializing in big data platforms, real-time data pipelines, and cloud-native architectures across fintech, telecom, and digital media. Expertise in designing and scaling enterprise-grade Data Lakes, Warehouses, and Mesh solutions on both on-prem and AWS (EMR, Glue, Redshift, Lake Formation). Architected and delivered multiple high-throughput, production-grade pipelines powering mission-critical analytics and regulatory reporting. Deep knowledge of Spark, Kafka, Airflow, and data governance frameworks including CDC, lineage tracking, and modular orchestration. Early background in mobile and web development adds full-stack systems insight. Proven ability to lead cross-functional teams (20+), drive technical strategy, and align data platforms with business outcomes.

Overview

16
16
years of professional experience

Work History

AVP – Data Engineering & Architecture

Airtel Payments Bank
06.2022 - Current
  • Led the migration of the entire on-prem data infrastructure to AWS cloud, transforming legacy systems into scalable, cloud-native services.
  • Architected and deployed an enterprise-wide Data Mesh platform, enabling domain-oriented ownership and self-serve capabilities across verticals like UPI, Retailer, Merchant, and Finance.
  • Designed and operationalized a centralized Data Warehouse and a governed Lakehouse architecture using AWS S3, Glue, Redshift, and Lake Formation.
  • Enabled compliance and audit-readiness through metadata governance, lineage tracking, and data quality frameworks using Apache Atlas, Lake Formation, and Great Expectations.
  • Established cross-functional collaboration practices, empowering data consumers with secure, scalable, and analytics-ready datasets.
  • Led a team of 20+ data engineers, driving project delivery, mentoring, and promoting cloud-first engineering excellence.

SDE IV – Data Engineering

Airtel Payments Bank
09.2019 - 05.2022
  • Worked primarily in an on-prem environment, designing and maintaining large-scale data pipelines for operational and analytical use cases.
  • Created a robust Hybrid Data Lake Architecture using Apache Spark, Hive, Airflow, Cassandra, and Elasticsearch.
  • Built 1TB+/day ingestion pipelines and orchestrated them with Apache Airflow.
  • Designed CDC workflows from Oracle Flexcube using GoldenGate → Kafka → Spark → Redshift.
  • Developed a scalable, multi-source Data Hub with searchable indexes using Elasticsearch and integrated metadata management.
  • Laying the groundwork for AWS migration by standardizing data structures and pipeline modularization.
  • Reduced SLA breaches by 30% through modular orchestration using Apache Airflow.

Senior Software Consultant – Full Stack & Data Engineering

AI Matt College Marketplace Pvt. Ltd.
04.2018 - 09.2019
  • Developed scalable full-stack ed-tech platforms using Django, PostgreSQL, and MongoDB.
  • Integrated analytics layer using Hadoop, Spark, and Kafka for student data insights.
  • Delivered secure and scalable data workflows across systems.

Tech Lead – Big Data Platforms

HT Media Ltd.
08.2016 - 03.2018
  • Led big data architecture efforts using Spark, Kafka, Hadoop, and Hive.
  • Designed scalable pipelines for editorial and advertising analytics.
  • Managed development of digital portals (e.g., HTCampus.com) using Python, Redis, MongoDB, MySQL, and XAMPP stack.
  • Optimized real-time content delivery, caching, and backend APIs.

Senior Software Engineer

ReTech Labs, Inc.
02.2016 - 05.2016
  • Developed microservices for a retail management platform using Django, Celery, Redis, MongoDB, and RESTful APIs.
  • Improved API performance with background processing and caching.

Senior Software Engineer & Consultant

Teramatrix Technologies Pvt. Ltd.
08.2009 - 02.2016
  • Joined Teramatrix Technologies full-time as a Senior Software Engineer in Dec 2013.
  • From Aug 2009 to Nov 2013, worked as a freelance consultant for Teramatrix Technologies and other clients, delivering 35+ full-stack solutions using Django, Android/iOS, REST APIs, MongoDB, MySQL, and AWS EC2/S3.
  • Focused on backend systems, dashboard development, API integrations, and cloud deployment across education, healthcare, and retail domains.
  • Led the development of telecom network monitoring and IoT systems using Python, Django, MongoDB, MySQL, and Redis.
  • Delivered real-time dashboards and alerting platforms, and participated in performance optimization and system architecture design.
  • Implemented Redis caching for real-time performance and alerting.

Education

Bachelor of Technology (B.Tech) - Computer Science

Kurukshetra University
01.2009

Skills

  • Cloud Platforms: AWS (S3, Glue, Athena, Redshift, EMR, MSK, Lambda, OpenSearch, Lake Formation), GCP
  • Big Data: Apache Spark (batch & streaming), Hadoop, Kafka, Hive, Delta Lake, HDFS
  • ETL & Orchestration: Apache Airflow, MWAA, Step Functions, AWS Glue Jobs
  • Governance & Quality: Apache Atlas, Lake Formation, Great Expectations, DataHub
  • Databases: Oracle, PostgreSQL, Redshift, Cassandra, MongoDB, MySQL
  • Programming: Python (Expert), SQL, Java, Scala
  • DevOps: Terraform, Kubernetes, Docker
  • Security: IAM, KMS, CloudTrail, Macie, S3 Policies

Timeline

AVP – Data Engineering & Architecture

Airtel Payments Bank
06.2022 - Current

SDE IV – Data Engineering

Airtel Payments Bank
09.2019 - 05.2022

Senior Software Consultant – Full Stack & Data Engineering

AI Matt College Marketplace Pvt. Ltd.
04.2018 - 09.2019

Tech Lead – Big Data Platforms

HT Media Ltd.
08.2016 - 03.2018

Senior Software Engineer

ReTech Labs, Inc.
02.2016 - 05.2016

Senior Software Engineer & Consultant

Teramatrix Technologies Pvt. Ltd.
08.2009 - 02.2016

Bachelor of Technology (B.Tech) - Computer Science

Kurukshetra University

Similar Profiles

CREATE PROFILE
Priyesh Data Engineering & Architecture Leader