Salman Shaik

Project: Supply Chain Management

• Architected and implemented scalable ETL pipelines using Python, PySpark, and Spark SQL, processing high-volume datasets across batch and streaming workflows, handling millions of records daily.

• Spearheaded end-to-end migration of legacy ETL pipelines from Cloudera to IOMete, standardizing codebases, reducing technical debt, and improving system maintainability by 40%.

• Integrated Apache Kafka into ETL frameworks to enable real-time data ingestion and event-driven processing, reducing downstream data latency by 60%.

• Engineered comprehensive unit testing frameworks, boosting code reliability and test coverage to 90%+, ensuring seamless CI/CD pipeline deployments with zero downtime.

• Executed full-scale migration of analytical and operational tables from SingleStore to Lakehouse architecture, incorporating end-to-end data hydration, reconciliation, and validation ensuring 100% data integrity.

• Optimized Spark job performance by 50% through refined partition strategies, query logic restructuring, and systematic elimination of processing bottlenecks across critical data pipelines.

Project: SCM - Data Agent Project

• Designed and developed the RCA Analyzer Agent, an AI-powered module automating root cause analysis of production incidents, accelerating issue diagnosis and reducing Mean Time to Resolution (MTTR) by 70%.

• Built the ProdOps Observer Agent, a real-time observability engine that continuously monitors production environments, detects anomalies using intelligent alerting mechanisms, and surfaces actionable insights to operations teams, improving incident response efficiency by 50%.