RAHUL Yalladas GADDAM

Data Ingestion and Transformation Using Medallion Architecture and Applying Present SCD Type Implementation.

• Implemented Medallion Architecture in Databricks, transforming raw data into refined datasets across Bronze, Silver, and Gold layers, enhancing data quality and analytics performance.
• Built and optimized data pipelines in Databricks using Medallion Architecture, automating data processing to generate clean, ready-touse datasets for business insights.
• Copied data from Blob Storage to Azure SQL and Lakehouse (One Lake) for efficient data integration.
• Developed Slowly Changing Dimension (SCD) logic with PySpark, handling both Type 1 and Type 2 updates to maintain historical data integrity.
• Transferred data across Blob Storage locations using Data Factory (Data Flow) in Microsoft Fabric .
• Monitored data pipelines and storage solutions to ensure efficient data flow and system performance

ETL Process Development

• Developed and optimized ETL processes to ensure data accessibility and high-quality outputs for business systems .
• Designed, documented, and automated data workflows using SQL, Power BI, and Python.