Damodaran J
Bigdata Engineer
Chennai
Summary
Innovative Data Engineer with 3 years of experience in developing scalable ETL pipelines and implementing integrated security systems. Reduced ETL time by 80% through partition-level extraction and loading, and designed efficient Apache Airflow DAGs to ensure seamless data workflow automation. Seeking a Bigdata Engineer role to advance data-driven strategies through expertise in Hadoop, Pyspark, and Kafka.
Overview
4
4
years of professional experience
5
5
years of post-secondary education
6
6
Certifications
Work History
Data Engineer
Sunoida Solutions Pvt Ltd
Chennai
04.2023 - Current
- Developed scalable ETL pipelines using Scala and Apache Spark, implementing partition-level extraction and loading, which reduced ETL time by 80%.
- Integrated a security mechanism using HashiCorp Vault to manage secrets and sensitive credentials securely.
- Designed and implemented Apache Airflow DAGs to orchestrate and automate data workflows, ensuring reliable data pipeline execution.
- Built a Data Lake POC for the flagship product Vision Lake-House, enabling efficient storage and retrieval of both structured and unstructured data at scale, which improved data accessibility and reduced data retrieval time by 40%.
- Conducted social media data analysis, deriving actionable insights from platforms to support business intelligence and marketing strategy.
Junior Programmer
CHAIN-SYS SOFTWARE EXPORTS PRIVATE
Chennai
10.2021 - 08.2022
- Implemented Azure SAML SSO and Ping SSO from scratch into existing enterprise applications built on Spring Boot and Java Servlets, enabling secure single sign-on across the platform.
- Collaborated with DevOps teams to manage SSO metadata, handle certificate management, and configure secure endpoints for production deployment, ensuring seamless integration and security for 5+ applications in the organization.
- Gained strong hands-on experience in core data structures and algorithms, enhancing the performance and scalability of backend modules.
- Contributed to debugging and resolving complex authentication issues in multi-tenant environments.
- Played a key role in debugging and resolving complex authentication issues within a multi-tenant environment, leading to huge reduction in support tickets related to authentication errors.
Education
Bachelor in Architecture - Architecture
Hindustan University
Chennai, India 08.2016 - 04.2021
Skills
ETL development
Data pipeline design
Data warehousing
Big data processing
Spark framework
Hadoop ecosystem
Certification
Big Data Hadoop
Timeline
Python for Data Science
09-2023
Machine Learning
09-2023
Linux training
09-2023
APACHE SPARK AND SCALA
07-2023
Data Engineer
Sunoida Solutions Pvt Ltd
04.2023 - Current
Big Data Hadoop
02-2023
Kafka
01-2023
Junior Programmer
CHAIN-SYS SOFTWARE EXPORTS PRIVATE
10.2021 - 08.2022
Bachelor in Architecture - Architecture
Hindustan University
08.2016 - 04.2021
Similar Profiles
Aravinth ChithiraikumarAravinth Chithiraikumar
Senior Software Engineer - Front End Developer at Sunoida Solutions Private LimitedSenior Software Engineer - Front End Developer at Sunoida Solutions Private Limited
R. AjaiR. Ajai
Software Developer at SunoidaSoftware Developer at Sunoida
Jissin AntonyJissin Antony
Assistant Manager - Financial Shared Services at Thinksynq Solutions Pvt Ltd (Outsourced to Servion Global Solutions Pvt Ltd)Assistant Manager - Financial Shared Services at Thinksynq Solutions Pvt Ltd (Outsourced to Servion Global Solutions Pvt Ltd)