HITHESWAR REDDY MACHUPALLI

Project 1:

• Spearheaded the creation of an end-to-end ETL pipeline, establishing a Golden Layer Repository for historical data.
• Overhauled and optimized the monthly dashboard refresh process using the Golden Layer as a central foundation, reducing delivery time from 72 hours to 24 hours.
• Leveraged Azure tools, including Databricks, Data Factory, and Data Lake Storage, to streamline workflows, enhance data consistency, and fully automate the process.
• Integrated automated QC/QA checks within the pipeline using Python, cutting down manual QC efforts for dashboard owners, and ensuring data accuracy.
• Enabled a robust, scalable process that improved client satisfaction and unlocked opportunities for additional data-driven products.

Project 2:

• Drove optimization of the yearly dashboard refresh, cutting down processing time by over 58% (72 hours to 30 hours), thereby boosting client's competitive edge.
• Flagged the errors in legacy codes and corrected legacy code flaws, ensuring two years of historical data is accurately aligned with current-year data.
• Created scripts to extract additional client-requested information from raw files, embedding it within the existing process for more comprehensive reporting.
• Enabled timely and precise data access during the Annual Enrollment Period (AEP), crucial for US Medicare providers.

• Leveraged Azure Data Lake Storage, Azure Data Factory, Databricks, and SQL Data Warehouse to streamline large dataset management, supporting efficient data processing and ETL tasks.
• Developed SQL and Python scripts in Databricks to support data ingestion and execute ETL processes seamlessly.
• Designed and managed data pipelines to automate code execution and transfer data across SQL servers and storage accounts.
• Gained proficiency in ClickHouse database and workflow orchestration with Airflow through hands-on project experience.