Parath Kumar Sabesan

• Leading the team on migrating the existing customer central warehouse to the cloud warehouse.
• Involved in designing the cloud migration process to replicate the exact process that was running in legacy systems.
• Developed Spark codes in cloud environment (Databricks)to replicate the legacy process.

• Developed batch jobs using spark-scala to extract data from Postgre and write as business required files to deliver it to customers.
• Involved in various bug fixes on Wave 1 jobs to make sure it to go smoothly during production release.
• Involved in the discussion with legacy team to understand the process to define the root cause of the issues.

• Gathered the business requirements from the business partners and subject matter experts.
• Identified the CCPA law-influenced tables and respective columns from the data lake to design the metadata.
• Developed Scala code to read the request JSON files, load them to the tables based on the request type, and generate response JSON.
• Designed and created metadata for the identified tables that need to mask or delete the data in them.
• Developed Scala code to mask the data in the tables that have customer details.
• Designed Scala code to validate all the tables, whether the masking process completed without any issues.

Designed and scheduled an end-to-end process flow in order to run without any manual intervention.

• Developed shell scripts to pull and push files to the SFTP server.

· Importing and exporting data into HDFS using SQOOP

· Performed Data enrichments such as filtering, pivoting, format modeling, sorting and aggregation using Hive and Pig tools

· Installed,configured Hive and also written Hive Aggregate Functions

· Importing and exporting data into HDFS and Hive using Sqoop

· Developing the Pig scripts for pre-processing the data

· Transforming Hive data into HBase and querying the data using Phoenix