Nithesh Kumar M
Bangalore
Summary
Senior Big Data Engineer with 13+ years of experience designing and building scalable data pipelines across banking, payments, and enterprise analytics domains. Strong expertise in Apache Spark (Scala), SQL, Kafka, DBT, and Airflow, with hands-on experience in batch and near real-time data processing. Proven track record in modernizing legacy data platforms, building analytics-ready datasets, orchestrating complex workflows, and enabling business reporting, reconciliation, and operational analytics. Experienced in Return & Refunds data frameworks, billing modernization, and data quality pipelines, with strong collaboration across business, analytics, and platform teams.
Overview
14
14
years of professional experience
Work History
Big Data Engineer / Senior Data Engineer
Big Data Engineer / Senior Data Engineer
Bangalore
07.2022 - Current
- Designed and implemented modern data pipelines using DBT and Apache Airflow to transform raw transactional data into analytics-ready datasets for enterprise billing and reporting use cases.
- Led modernization of legacy billing systems by replacing monolithic ETL processes with modular, scalable data transformation workflows.
- Built real-time and batch data processing pipelines to segregate billable and non-billable events, enabling near real-time financial visibility.
- Developed and maintained complex Airflow DAGs with dependency management, retries, SLA monitoring, and failure handling.
- Implemented data quality checks, reconciliation logic, and validation frameworks to ensure accuracy, completeness, and audit readiness.
- Collaborated with cross-functional teams including product owners, QA, and DevOps to ensure seamless integration and deployment.
- Supported operational and financial analytics, including return and refund lifecycle tracking, SLA adherence, and cost analysis.
- Used Jenkins and Git-based CI/CD pipelines to automate builds, testing, and deployments.
Data Engineer
Infosys Ltd
Bangalore
07.2014 - 06.2022
- Built and optimized large-scale data processing applications using Apache Spark and Scala for banking and financial services clients.
- Developed real-time and batch data pipelines to process high-volume transactional and reference data.
- Implemented Spark SQL and HiveQL-based transformations to support analytics, reconciliation, and reporting use cases.
- Designed data quality and reconciliation reports to validate data consistency and ensure golden-source accuracy.
- Worked on real-time streaming and monitoring platforms leveraging Kafka and Spark for infrastructure and application health metrics.
- Optimized Spark jobs for performance by tuning memory, execution plans, and partitioning strategies.
- Participated in requirement gathering, solution design, estimation, and client discussions with onshore teams.
- Used Jenkins, Bitbucket, and CI/CD tools to support automated builds and deployments.
- Provided production support, defect analysis, and performance improvements across multiple releases.
DBA | Project Engineer
Bharath Electronics Limited
Bangalore
12.2011 - 12.2013
- Managed and maintained large-scale MySQL databases supporting national-level data digitization initiatives.
- Performed database administration activities including backups, restores, monitoring, and performance tuning.
- Generated operational and ad-hoc reports using complex SQL queries for business and regulatory needs.
- Ensured high availability and 24x7 operational support for mission-critical databases.
- Supported data ingestion, validation, and consistency checks during large-volume enrollment processes.
- Worked closely with application teams to resolve data issues and optimize database interactions.
Education
Bachelor of Science -
Visvesvaraya Technological University (VTU)
Bangalore06-2011
Skills
- Apache Spark (Scala), SQL, Spark SQL, Kafka (streaming fundamentals), Hadoop ecosystem
- DBT (Data Build Tool), Apache Airflow, DAG design and scheduling, end-to-end data pipeline orchestration
- Hive, MySQL, analytical data modeling, data warehouse concepts
- Jenkins, Git, Bitbucket, TeamCity, CI/CD pipelines, deployment automation
- Data Quality checks, reconciliation frameworks, SLA monitoring, operational analytics, financial analytics, return & refund lifecycle analytics
- IntelliJ, Eclipse, JIRA, Linux/Unix, WinSCP
Timeline
Big Data Engineer / Senior Data Engineer
Big Data Engineer / Senior Data Engineer
07.2022 - Current
Data Engineer
Infosys Ltd
07.2014 - 06.2022
DBA | Project Engineer
Bharath Electronics Limited
12.2011 - 12.2013
Bachelor of Science -
Visvesvaraya Technological University (VTU)