Shashank Anupam

Driven Data Engineer with 8 years of expertise in Hadoop, Spark, Scala, and reporting tools within the Data analytics domain. Proficient in architecting, developing, and optimizing data pipelines for actionable insights. Seeking a challenging role where I can apply my proven skills to propel data-driven decision-making and make significant contributions to innovative projects.

IMS(Inventory Management System) :

• Designed, created, and maintained data pipelines for Inventory data Management.
• Consolidated Inventory reporting table to create a consolidated data view and reduce data redundancy. It saved around 60% of cloud costs on a daily basis to run the pipeline.
• Created Inventory pipeline for promotional items and shelf management which generates yearly GMV of $300M.
• Created Data validation and Data quality framework to ensure data completeness and correctness.

LIMO - Item Path Eligiblity :

• Designed, created, and maintained NRT data pipelines to load BQ and Hudi tables.
• Handled Upsert data volumes in TBs with the final table having 2 Trillion records and approx 120 TB storage volume. Business value generated by this project is $150M.

Cloud cost optimisation :

• Spark tuning of jobs to reduce cluster size and job run time.
• Created compression utility to store raw data in datalake compressed format. Enabled around 70% raw data compression and 60% storage cost saving.
• Worked on data asset consolidation to reduce data redundancy in the system. Enabled autoscaling in cluster creation to optimize resource memory allocation in the data pipelines.

• Received Badgify Badges - Customer champion, Collaborator, Impact driver, Fire fighter.
• Receives Bravo Award for managing Inventory Mart.