RAJESH KUMAR G
Chennai,TN
Summary
Dynamic Data Engineer with 3 years of experience in designing and implementing robust data pipelines that enhance real-time processing capabilities. Expertise in optimizing data storage solutions to drive efficiency and cost-effectiveness, while ensuring the highest standards of data quality and integrity. Proficient in leveraging tools such as PySpark, Python, and Apache Airflow to develop seamless ETL processes and facilitate collaboration within cross-functional teams. Committed to continuous improvement and innovation in data engineering practices.
Overview
3
3
years of professional experience
1
1
Certification
Work History
Data Engineer
Cognizant Technology Solutions
07.2022 - Current
- Developed end-to-end ETL pipelines in PySpark and Python to ingest, clean, and transform high-volume datasets, supporting reliable downstream analytics and reporting.
- Optimized Spark jobs by tuning partitions, caching, and broadcast joins, reducing runtime by 30%.
- Built automated CI/CD pipelines using Jenkins and Git for PySpark and Python projects, incorporating code reviews, unit testing, and deployment monitoring, which improved collaboration and reduced production issues.
- Developed and tested PySpark modules in IntelliJ IDE with proper logging and error handling.
- Implemented unit tests and integration tests for Spark jobs, improving data quality and reliability.
- Configured Airflow DAGs for orchestrating PySpark batch pipelines and monitoring job status.
- Facilitated on-time project delivery, ensured 100% compliance and actively contributing to sprint planning and promoting effective team collaboration.
- Have 3 years of experience in an Agile development environment.
Education
Bachelor of Engineering - Electrical and Electronics Engineering
Panimalar Engineering College
04-2022
Skills
- Languages/Framework: Python, PySpark
- Data Engineering: Apache Airflow, ETL, Data Pipelines
- IDEs: IntelliJ, AWS Code Server
- Cloud: AWS (S3)
- Tools & Platforms: Jenkins, Jira, GitHub/GitLab, Jupyter Notebook
- Databases: MySQL
- Data pipeline design
Certification
- Big Data Environment & Analytics – Udemy
- Data Warehousing and ETL Fundamentals – Udemy
Timeline
Data Engineer
Cognizant Technology Solutions
07.2022 - Current
Bachelor of Engineering - Electrical and Electronics Engineering
Panimalar Engineering College