Mayank Kandpal

Demerger process for the BI and Data ecosystem of the prominent Australian Gaming and Wagering organization.

• Contributed to various modules in support of the corporate demerger initiative, focusing on provisioning the KENO Data Warehouse and Lotteries Data Warehouse.
• Designed and developed new code in collaboration with business and technical stakeholders using PySpark, Postgres, Teradata, Talend, and Control-M.
• Collaborated with the team to design a model for capturing various slowly changing dimensions within different Dimension and Fact tables.
• Analyzed and upgraded existing code to facilitate integration with the future state environment.
• Conducted an analysis and updated the internal IDs from the old Salesforce system to the new ones in the newly provision Data Warehouse environment using Talend, Salesforce, Teradata, Teradata SP.
• Analyzed and revised the surrogate key generation mechanism in the DDL statements for Fact and Dimension tables to eliminate any duplication of surrogate keys after provisioning to the new DWH environment. The implementation utilized Teradata, YAML, and Git actions.
• Supported various testing phases(UT, SIT, UAT) and code Go-Live.

Cloud Migration for the one of the largest US pharmaceutical.

• Migrated the ingestion pipelines from Oracle DB to HDFS, transitioning from Sqoop to PySpark for enhanced performance.
• Migrated the ETL pipelines, originally running on Talend, to PySpark and worked on Spark performance tuning to optimize overall system performance.
• Utilized Apache Hive for in-depth data analysis during UT & SIT phases.