Data Engineering professional with 9 years of proven track record in developing robust data pipelines, database management, and cloud-based solutions.
Skilled in SQL, Python, and Spark, with strong focus on team collaboration and delivering impactful results. Adaptable and reliable, consistently meeting evolving project needs while fostering productive team environment.
- Rewrote metadata files using predefined procedures to align with the new data location.
- Registered copied tables with the AWS Glue Catalog and refactored Spark transformations to integrate with the new architecture.
- Enabled table optimization in Glue using Airflow DAGs for enhanced performance.
- Ensured data integrity by implementing and executing a comparison DAG to validate data consistency between the source (Tabular) and target (Glue) databases post-migration.
Traffic Management in Cloud Data Centers, Anjali Ajayan, G. Anusooya, International Journal of Engineering and Future Technology, Vol.7, 2016, 51-57
Advanced Java Technologies,
C-DAC (Centre for Development of Advanced Computing)