Shubham Khadake

Project Name: Emergency Respond Data Warehouse Analysis

• Designed and implemented a complete data pipeline, integrating various data sources and leveraging technologies such as Apache Spark, Hive, Kafka, MySQL, HBase, NIFI, and Python.
• Enhanced data processing efficiency by 30% through Spark-Hive and Spark-HBase integrations, reducing latency in real-time and batch processing scenarios.
• Implemented daily jobs using Databricks to refresh data and maintain up-to-date information. I worked on the latest Azure Databricks feature, the Delta Live Table.
• Secured a 40% increase in Kafka data ingestion speed, optimizing the streaming pipeline, and reducing overall processing time by 25% across various data pipeline components.
• Increased data processing efficiency by 40% through parallel processing and Spark jobs, and provided solutions for production issues.

Project Name — Telecom Infra Data Analysis

• ETL from different sources to different destinations, and transformed using SQL stored procedures and scheduled triggered jobs.
• Proficient in writing complex SQL queries with multiple table joins and stored procedures, depending on requirements.
• Have a good understanding of SQL functions for data preparation and transformation.
• Created SQL jobs, tested, and monitored the same.
• Transformed SQL complex queries into a readable format for better analysis.