Summary
Overview
Work History
Education
Skills
Interests
Timeline
Workshop/Trainings Attended
Personal Information
Strengths
Generic
Kamlesh Kumar

Kamlesh Kumar

Senior Data Engineer
GREATER NOIDA

Summary

Senior Analytics Manager / Data Platform SME with 15+ years of experience in designing and delivering scalable PySpark-based data pipelines for high-volume data processing and analytics in distributed environments. Demonstrated expertise in optimizing PySpark workloads for performance, stability, and efficient resource utilization across Hadoop ecosystems. Leverage deep technical and domain expertise to drive data-driven decision-making, operational efficiency, and platform reliability, consistently transforming complex data into actionable insights that support organizational objectives and innovation.

Overview

16
16
years of professional experience

Work History

Senior Data Engineer

Airtel International LLP
GREATER NOIDA
05.2022 - Current

Project: Airtel Africa

Role: Analytics Manager / Data Platform SME
Environment: Apache Hadoop, PySpark, Trino, Hive, Kafka, Oracle, PostgreSQL, Striim CDC, UNIX, Tableau

Key Responsibilities & Achievements

  • Architected and delivered enterprise-scale Hadoop-based data platforms for large-volume, multi-node data processing, ensuring scalability, reliability, and performance.
  • Owned HDFS data management, including partitioning strategies and storage optimization, significantly improving query performance and processing efficiency.
  • Designed and optimized ETL frameworks using Hive and PySpark, enabling seamless ingestion and transformation of data from multiple heterogeneous sources.
  • Developed scalable PySpark pipelines to process transactional and analytical datasets, supporting downstream reporting and analytics use cases.
  • Built and maintained production-grade PySpark ETL scripts to ensure accurate, timely, and consistent data availability for business reporting.
  • Automated end-to-end reporting workflows by integrating PySpark jobs with scheduled executions (Cron), reducing manual effort and improving delivery SLAs.
  • Partnered with data analysts and business stakeholders to translate requirements into PySpark-driven analytical datasets and KPI reports, enabling data-driven decision-making.
  • Designed and optimized complex SQL queries across Oracle and PostgreSQL systems, ensuring high performance, accuracy, and scalability.
  • Developed UNIX shell automation scripts to orchestrate data extraction, transformation, and reporting processes, improving operational consistency and reducing errors.
  • Implemented robust data validation and reconciliation controls within SQL and ETL pipelines to ensure financial-grade data accuracy and reliability.
  • Administered and governed Striim CDC pipelines for near real-time data replication between source systems and data warehouses.
  • Monitored, tuned, and optimized Striim CDC processes to maintain low-latency data flows and high availability across platforms.
  • Proactively troubleshot and resolved CDC, ETL, and production issues, ensuring continuous data availability for critical business operations.
  • Contributed to multiple successful project deliveries by providing architectural insights, system capability assessments, and solution recommendations.
  • Designed customized data solutions aligned to business needs, improving overall system performance and analytical capabilities.
  • Provided L3-level support to end users and downstream teams, resolving complex technical issues with minimal business impact.
  • Identified process gaps and led continuous improvement initiatives, increasing productivity, automation coverage, and data quality.
  • Supported long-term data and analytics strategy by advising on emerging technologies, best practices, and scalable platform design.

Data Engineer

Wipro Limited
GREATER NOIDA
03.2019 - 04.2022

Project Overview Key Responsibilities & Contributions

Project: Telenor Global

Client: Telenor Myanmar (AEP)
Role: Technical Consultant / Data Architect
Environment: Apache Hadoop, Hive (SQL), Kafka, Sqoop, Vertica, Talend (ETL), AWS (EC2, S3), UNIX, Qlik Sense, NPrinting

Led the implementation of a next-generation BI and analytics platform for Telenor Myanmar on a private cloud (on-premise standalone platform), supporting enterprise analytics requirements while ensuring seamless integration with existing frontline systems.

  • Played a key role in architecting and implementing Hadoop-based analytics solutions, leveraging core components including HDFS, YARN, MapReduce, NameNode, DataNode, JobTracker, and TaskTracker.
  • Analyzed source system data and collaborated with upstream teams to gather schema, metadata, and connectivity details for ingestion into Hadoop.
  • Designed and implemented data ingestion pipelines using Hive, Sqoop, Kafka, and Talend to onboard structured and semi-structured data.
  • Utilized AWS EC2 and S3 for processing and storage of selective datasets, supporting hybrid analytics workloads.
  • Designed logical and physical data models for multiple data feeds to be accommodated in Vertica, ensuring scalability and query performance.
  • Owned estimation and planning for new developments and enhancements, supporting delivery commitments, and roadmap planning.
  • Led end-to-end solution documentation, including architecture designs, data flows, and operational handover artifacts.
  • Translated complex business and functional requirements into detailed technical designs and scalable data solutions.
  • Defined and enforced best practices, standards, and governance guidelines for Hadoop and analytics development.
  • Coordinated knowledge transfer and handover to operations teams, ensuring production stability and long-term maintainability.
  • Supported analytics consumption through Qlik Sense and NPrinting, enabling self-service BI and automated reporting.

Developer Analyst

Wipro Limited
Noida
12.2017 - 03.2019
  • Worked as L3 Developer & Lead in BI & MIS for both delivery and production support. Played a role as Team Member in various innovation and developed in Oracle PL/SQL & Hive
  • Handled errors using Exception Handling extensively for the ease of debugging and displaying the error messages in the application.
  • Responsible for Data Movement as per purging policy from Exadata to Hadoop.
  • Did Reports analysis and Monthly CDRs analysis using Hive.
  • Responsible for SQL tuning and Optimization.
  • Work closely with various teams across the company to identify and solve business challenges utilizing large structured, semi-structured, and unstructured data in a distributed processing environment.
  • Project Name: Telenor Global
  • Client: Telenor Myanmar (BI/MIS)
  • Environment: SQL, PL/SQL, Oracle 11g, UNIX, Hive, Apache Hadoop, Sqoop
  • Summary: As a Senior Developer, I am responsible for development, support, maintenance and implementation of a complex project module. I have good experience in application of standard software development principles. I work as an independent team member, capable of applying judgment to plan and execute your tasks. I have in-depth knowledge of SQL, Unix,Hive. I respond to technical queries / requests from team members and customers. I always believe to coach, guide and mentor junior members in the team.

Developer

Wipro Limited
Noida
03.2013 - 12.2017
  • Working as Prepaid-IN Lead for both delivery and production support. Played a role as Team Member in various innovation like Dealer Commission, DCC etc, all developed in Oracle PL/SQL.
  • Analyze large datasets to provide strategic direction to the company.
  • Involved in the continuous enhancements and fixing of production problems.
  • Generated server side PL/SQL scripts for data manipulation and validation and materialized views for remote instances.
  • Developed PL/SQL triggers and master tables for automatic creation of primary keys.
  • Created PL/SQL stored procedures, functions and packages for moving the data from staging area to data mart.
  • Created scripts to create new tables, views, queries for new enhancement in the application using TOAD.
  • Created indexes on the tables for faster retrieval of the data to enhance database performance.
  • Involved in data loading using PL/SQL and SQL
  • Loader calling UNIX scripts to download and manipulate files.
  • Performed SQL and PL/SQL tuning and Application tuning using various tools like EXPLAIN PLAN, SQL
  • TRACE, TKPROF and AUTOTRACE.
  • Extensively involved in using hints to direct the optimizer to choose an optimum query execution plan.
  • Used Bulk Collections for better performance and easy retrieval of data, by reducing context switching between SQL and PL/SQL engines.
  • Created PL/SQL scripts to extract the data from the operational database into simple flat text files using UTL_FILE package.
  • Creation of database objects like tables, views, materialized views, procedures and packages using oracle tools like Toad, PL/SQL Developer and SQL
  • Plus.
  • Partitioned the fact tables and materialized views to enhance the performance.
  • Extensively used bulk collection in PL/SQL objects for improving the performing.
  • Created records, tables, collections (nested tables and arrays) for improving Query performance by reducing context switching.
  • Used Pragma Autonomous Transaction to avoid mutating problem in database trigger.
  • Extensively used the advanced features of PL/SQL like Records, Tables, Object types and Dynamic SQL.
  • Handled errors using Exception Handling extensively for the ease of debugging and displaying the error messages in the application.
  • Doing Reports analysis and Monthly CDRs analysis using Hive.
  • Responsible for SQL tuning and Optimization.
  • Work closely with various teams across the company to identify and solve business challenges utilizing large structured, semi-structured, and unstructured data in a distributed processing environment.
  • Project Name: SSTL
  • Client: MTS-Mobile
  • Environment: SQL, PL/SQL, Oracle 11g, UNIX, Hive, Apache Hadoop, Sqoop
  • Summary: As a Software Developer, I am responsible to ensure Network Operations for IN (Intelligent Network) service for CDMA based Cellular Network of MTS and Vodafone Falcon for Plan configuration, Charge Configuration, Complaint resolution & Root cause analysis, Voucher arrangement, Reconciliation, Automations, Offline Benefits Configurations etc. on Huawei IN (OCS).

Associate System Analyst

NSE.IT LTD
Mumbai
04.2011 - 11.2012
  • Role: Database Designer/Developer
  • Project Name: CART (Credit Appraisal Rating Tool)
  • Client: SIDBI (Small Scale Industrial Bank of INDIA)
  • Environment: SQL, PL/SQL, Oracle (9i, 10g)
  • Improved system efficiency by analyzing and troubleshooting complex software issues.

TSE

Reliance Tech Services
Mumbai
05.2010 - 04.2011
  • Role: Technical Support
  • Project Name: Order Management System
  • Environment: SQL, UNIX
  • Organized and detail-oriented with a strong work ethic.

Education

B.E. - Electronics & Communication

Shri Sant Gadge Baba College, Bhusawal
Maharashtra
05.2009

HSC - Science

U.P. Board
Ballia, Uttar Pradesh
05.2004

SSC -

U.P. Board
Ballia, Uttar Pradesh
05.2002

Skills

Interests

Reading, Playing Cricket, Traveling

Timeline

Senior Data Engineer

Airtel International LLP
05.2022 - Current

Data Engineer

Wipro Limited
03.2019 - 04.2022

Developer Analyst

Wipro Limited
12.2017 - 03.2019

Developer

Wipro Limited
03.2013 - 12.2017

Associate System Analyst

NSE.IT LTD
04.2011 - 11.2012

TSE

Reliance Tech Services
05.2010 - 04.2011

B.E. - Electronics & Communication

Shri Sant Gadge Baba College, Bhusawal

HSC - Science

U.P. Board

SSC -

U.P. Board

Workshop/Trainings Attended

  • Attended Internal Training for SQL.
  • Attended Internal Training for PL-SQL.
  • Attended Internal Training for NoSQL data stores (MongoDB)

Personal Information

Nationality: Indian 

Marital Status: Married 

Living Languages: English, Hindi 

Permanent Add: B1-1045, TOWER 11, PURVANCHAL ROYAL CITY, GREATER NOIDA, UP-201308 


I hereby declare all the information presented above is true to the best of my knowledge. I assure you to meet up to your expectations if given an opportunity.

Strengths

  • Initiative, Planning, Organizing and Executing Skills
  • Innovative, Detailed-Oriented, Conscientious, Adaptable, Quick learner and Responsible
  • Capability to handle Documentation and Project Work under deadlines.
Kamlesh KumarSenior Data Engineer