Juhi Josh

Project - (MCT): December 2022 - August 2024, Ongoing:

FMCG Manufacturing Domain - Application for all Global PepsiCo Manufacturing plants.

• Designed and implemented a robust medallion architecture using SQL Server as the Bronze layer, leveraging ADF for efficient ETL processes for large datasets, Azure Databricks PySpark notebooks for data transformations, and Azure Synapse as the Gold layer for downstream data science operations.
• Collaborated with data scientists, business analysts, and software engineers across cross-functional teams to understand their data requirements and then transform data based on business logic using Azure Databricks notebooks, facilitating data cleansing, normalization, aggregation, and feature engineering.
• Optimized query performance to make the ingestion process 40x faster using techniques like data partitioning, vacuuming, and optimization. Implemented functions to handle repetitive code blocks and data checks to automatically alert and resolve data gaps in the gold layer.
• Built complex stored procedures on Azure PaaS server to aggregate 50+ SQL tables for dynamic transformations and utilized Azure Data Factory's CDC to meet real-time data flow business requirements and reduce cost.

Project - Customer Rewards Program Application, Jan 2024 - July 2024

• Built and maintained ETL pipelines using Couchbase Server source, ingesting JSON data into Azure Blob Storage. Further, used Azure Databricks to prepare data for downstream software development and seamlessly moved all data to the gold layer using techniques like inserts and updates, following a high watermark level template.

Project - Consumer Domain Data Engineering Team - August 2024, Ongoing

• Responsible for meeting a 24-hour SLA to ingest data from Nielsen IQ in bronze and silver layers for all sectors and markets while resolving data quality issues on the go.
• Successfully migrated data pipelines from Snowflake to new API source by leading the utilization of Azure Data Factory (ADF), Databricks, and Blob storage. Implemented Bronze and Silver layers for seamless and efficient data transfer of critical consumer datasets.