Pratham Patil

Dual Role Leadership

• Operating in a hybrid role (Scrum Master ~30% / Data Engineer ~70%), combining leadership with hands-on engineering.
• Leading Agile delivery for a team of 8 engineers across globally distributed stakeholders
• Improved sprint predictability and reduced spillovers by enforcing Agile best practices and structured planning

Data Engineering & Cloud Migration

• Designed and maintained large-scale Spark/Databricks pipelines processing multi-terabyte airline datasets
• Led migration of multiple critical pipelines to Azure Databricks, improving scalability and reliability
• Contributed to full decommissioning of legacy (L-Box) pipelines, reducing technical debt and operational overhead

Performance Optimisation and FinOps (High Impact).

• Assisted in achieving annual cost savings of over €61,000 through effective strategies. Facilitated €50,000 yearly savings by implementing a spot VM strategy for non-SLA workloads. Removed redundant curves and columns from datasets, leading to €11,000 in additional savings.
• Achieved approximately 35% storage reduction by implementing ZSTD compression across datasets, saving approximately €167,000.

• Built PySpark pipelines for Model History Data (MHD) and Model Ready Data (MRD)
• Conducted thorough data quality checks, rectifying inconsistencies and ensuring reliability of information.
• Processed large structured datasets (millions of records) with complex transformations
• Implemented data quality and monitoring solutions, improving reliability
• Managed deployments across DEV, DR, and Production environments

• Developed end-to-end ingestion pipelines using Sqoop, Hive, and Spark
• Processed structured & semi-structured data (JSON, RDBMS)
• Designed optimised Hive data models improving analytics performance
• Delivered clean, enriched datasets for downstream consumption