Ezhilarasan Selvaraj

• Data Migration: Successful Migration of existing spark workload (2000+ jobs) from Amazon Redshift to Databricks for optimizing cost, data storage and processing.
• Programming and Scripting: Migrated shell scripts to Scala and Python script for data manipulation, transformation, and automation tasks.
• Workflow Automation: Designed and implemented end-to-end data workflow using Stonebranch.
• Job Scheduling: Expertise in job scheduling and management using Stonebranch.
• Job Monitoring : Scheduled data pipelines, managed dependencies, and monitored job execution to ensure timely and reliable data processing.
• Data Pipeline Development: Proficient in developing and optimizing ETL processes to extract data from Amazon S3, transform it to meet business requirements, and load it into target systems (Databricks) or data warehouses (Redshift).
• Data Mapping : Experienced in data mapping, data validation, and error handling.
• Data Modeling: Expertise in creating and implementing dimensional modeling (In Databricks and Redshift), and schema design for efficient data storage and retrieval.
• Agile Methodologies: Experience working in an agile environment, employing iterative and incremental approaches to project delivery.
• Demonstrated ability to manage multiple tasks simultaneously and deliver high-quality results within the specified time frame.

Databricks Migration (2022 May - 2022 September) : Data migration and integration project for a global healthcare provider, transitioning data from legacy systems to a modernized data infrastructure.

Role: Worked independently as one of the ETL testers for a global healthcare provider, remediated and executed ETL processes, ensured data integrity and quality throughout the migration.

Worked on basic SQL and Excel to generate reports.

Generated Monthly report for customers through sales and Scope Of Work given by Business.

Effective Management of cooling Systems databases through SQL.

I worked as a Design and Site Engineer in 16 floor high rise buildings in Perambur Chennai.