Sureshvelan Kathavarayan
Data Engineer
Chennai
Summary
Data Engineer with over 9 years of experience designing, building, and optimizing large-scale data pipelines and analytics platforms. Seeking a challenging role to leverage expertise in Apache Spark, Hadoop, AWS, and cloud-native technologies to deliver scalable, high-performance data solutions. Committed to ensuring data quality, reliability, and security while enabling actionable insights for business stakeholders and supporting strategic data-driven initiatives.
Overview
9
9
years of professional experience
1
1
Certification
1
1
Language
Work History
Data Engineer
Tata Consultancy Services
01.2025 - Current
- Designed and delivered end-to-end Apache Spark–based data pipelines for large-scale banking datasets, ensuring high data accuracy, regulatory compliance, and timely business reporting.
- Contributed significantly to legacy-to-AWS cloud migration initiatives by transitioning critical Oracle and SQL data to S3, EMR, and Athena, improving platform scalability, performance, and cost efficiency.
- Own and manage the complete lifecycle of data pipelines, from ingestion and transformation to validation and delivery across Hive and AWS ecosystems such as S3, EMR, and Athena.
- Execute large-scale legacy-to-cloud data migrations, ensuring high data integrity, optimized performance, and seamless continuity for downstream analytics and reporting systems.
- Build and optimize scalable data processing solutions using Apache Spark and custom enterprise frameworks to handle high-volume, complex datasets efficiently.
- Coordinate and control end-to-end workflow orchestration through AWS Step Functions and Control-M, proactively monitoring executions to meet SLAs and operational targets.
- Implement rigorous data governance by applying reconciliation controls, business-rule validations, encryption, and masking to safeguard sensitive financial and customer data.
- Strengthen delivery reliability by enabling CI/CD processes, maintaining clear technical documentation, and resolving data issues through detailed analysis and corrective enhancements.
Data Engineer
NTT Data
06.2024 - 12.2024
- Automated manual data processing and operational workflows using Unix shell scripting and orchestration tools, significantly reducing human effort, errors, and execution delays.
- Improved the performance of large-scale data pipelines by tuning Spark jobs and SQL queries, resulting in faster execution times and more efficient processing of high-volume datasets.
- Established and governed robust data quality controls, including reconciliation checks, row-level validations, audit allocation, and format verification to ensure trusted datasets.
- Implemented comprehensive data security and compliance measures by enforcing encryption, masking, and role-based access for sensitive banking and customer information.
- Proactively supported production data pipelines by monitoring workloads, resolving critical incidents, performing root cause analysis, and deploying long-term corrective solutions.
- Enhanced performance and cost efficiency by tuning Spark applications and SQL queries through optimized resource allocation, partitioning, and execution strategies.
- Conducted impact assessments for upstream and downstream changes, managing schema evolution while preserving backward compatibility and data consistency.
- Delivered stable and automated data operations by enabling CI/CD pipelines, Unix shell–based automation, and end-to-end validation to ensure timely, error-free releases.
Data Engineer
LTIMindtree
09.2022 - 06.2024
- Successfully managed UAT and production releases by validating data accuracy, coordinating deployments, and ensuring smooth go-lives with no critical post-release issues.
- Served as a key production support engineer, rapidly resolving high-priority data and system incidents within tight timelines and earning recognition for reliability and ownership.
- Deployed automated monitoring and alerting solutions that minimized manual intervention and improved the overall stability of data processing operations.
- Enabled successful regulatory audits and compliance checks by delivering clear data lineage, validation evidence, and reconciliation documentation.
- Managed UAT and production releases by validating data outputs, coordinating deployments, and obtaining formal business approvals.
- Established standardized coding frameworks and reusable components, driving consistency, maintainability, and faster development cycles across the team.
- Supported Agile delivery through accurate effort estimation, sprint planning participation, and continuous tracking of milestones and deliverables.
- Built and optimized Spark-based ETL pipelines and Hadoop/Hive data platforms, automating ingestion, enrichment, scheduling, and reporting for large-scale retail and wholesale datasets.
Data Engineer
Virtusa Consulting Services and Pvt. Ltd.
03.2017 - 09.2022
- Collaborated closely with business, QA, and analytics teams to translate complex requirements into reliable, scalable, and high-performance data solutions.
- Received recognition from stakeholders for ownership, accountability, and consistently delivering business-critical data initiatives on time.
- Led the full lifecycle of enterprise data pipelines, from design and development to deployment and production support, using Spark, PySpark, and Hive for financial data processing.
- Ensured high operational reliability by actively monitoring jobs, diagnosing failures, conducting root cause analysis, and implementing long-term remediation measures.
- Enhanced processing performance and scalability by optimizing Spark and SQL workloads through refined execution strategies, partitioning, and resource tuning.
- Executed complex data migrations across clusters and big data platforms, successfully transitioning multiple modules while quality and maintainability by managing CI/CD and version control, performing peer code reviews, facilitating knowledge transfer, and maintaining detailed technical documentation.
Education
B.Tech - IT
DR Mahalingam College of Engineering and Technology, Anna University
01-2016
Skills
Programming Languages: Python, Scala
Query Languages: SQL, Spark SQL
Big Data Technologies: Apache Spark, PySpark, Hadoop, Hive
Cloud Technologies: AWS S3, AWS EMR, AWS Athena, AWS Step Functions
Data Processing & Storage: HDFS, Hive Tables
Data migration
SQL expertise
Certification
Talend DI and Bigdata, 2017, Chennai
Personal Dossier
- Date of Birth: 03 Dec 1994
- Nationality: Indian
- Passport Details: N8677820 valid till May 2026
Timeline
Data Engineer
Tata Consultancy Services
01.2025 - Current
Data Engineer
NTT Data
06.2024 - 12.2024
Data Engineer
LTIMindtree
09.2022 - 06.2024
Data Engineer
Virtusa Consulting Services and Pvt. Ltd.
03.2017 - 09.2022
B.Tech - IT
DR Mahalingam College of Engineering and Technology, Anna University