Shahid Haider
Sr Data Platform Engineer
Bengaluru
Summary
Results-driven Senior Platform Engineer with expertise in cloud infrastructure, observability, and data engineering. Adept at designing and implementing scalable solutions on Azure and Databricks, optimizing monitoring and analytics workflows, and integrating enterprise platforms. Passionate about problem-solving, research, and automation to enhance system visibility and operational efficiency. Aspiring Solution Architect with a strong foundation in cloud technologies, infrastructure as code, and observability practices.
Overview
5
5
years of professional experience
5
5
years of post-secondary education
6
6
Certifications
Work History
Manager - Platform Engineering
Landmark Group
03.2021 - Current
- Developed a Databricks job monitoring tool in Python by leveraging the Databricks REST API for automated job tracking and alerting.(https://github.com/snhaider9977/monitoring-databricks-jobs)
- Developed Observability dashboards to monitor Azure spend by integrating with the Cost Management API for cost visibility and analysis.
- Utilized PySpark to develop data ingestion pipelines from sources like Branch, MoEngage, etc and managed daily Databricks platform operations including schema management
- Developed an observability dashboard using Databricks system tables to track DBU consumption and monitor cluster performance.
- Automated Delta table maintenance to Monitor changes to Delta Lake tables (e.g., version history, number of files, data size)..
- Developed custom tools and scripts to automate routine tasks, boosting team productivity.
- Implemented tiered storage strategy reducing Data Lake costs by 40%.
- Established team priorities, maintained schedules and monitored performance.
- Developed a real-time data pipeline POC ingesting data from GTM to Event Hub.
- Developing a backup and disaster recovery (DR) strategy using Delta Lake clone functionality on Databricks.
- Reimplemented Azure Data Factory (ADF) Devops pipeline deployment using linked ARM templates, overcoming size limitations.
- Architected and designed a Power BI solution including PBI Datagateways on the Azure stack.
- Optimized cost savings by reserving Databricks services and with reserved Azure VM, achieving up to 30% savings.
- Led annual cloud budget planning for FY25 with 95% forecast accuracy
- Contributed to the design and implementation of Cloud Data Platform (CDP) on Databricks.
- Developed CI/CD pipelines for Databricks using Azure DevOps, incorporating Git strategies.
- Collaborated with Microsoft FastTrack Team to design an Azure Landing Zone with Data Mesh Architecture.
- Projected a one-year cloud budget for analytics workloads.
- Used Terraform to migrate Azure services from the South India region to Central India, ensuring infrastructure consistency and minimal downtime.
- Designed a fully private enterprise-grade Azure Analysis Services solution using a hub-and-spoke network architecture
- Deployed a complete analytics infrastructure on Azure (Databricks, Synapse, ML Workspace, AKS).
- Integrated Power BI across cloud and on-prem environments, including licensing planning.
- Explored Oracle Content Management and trained teams on campaign management tools.
- Collaborated on cross-functional teams to drive platform improvements and deliver business value.
Intern - Cloud & Infrastructure
Softway Solutions
08.2020 - 11.2020
- Worked with Architect on observability-focused network infrastructure, contributing to the design of a secure VPC using subnets, NACLs, NAT gateways, and security groups to ensure reliable monitoring and access control.
- Designed and implemented end-to-end observability solutions using AWS CloudWatch, including custom metrics, dashboards, and alerts to enhance system visibility and performance monitoring.
- Configured and tuned Nginx and Apache web servers with a focus on logging, monitoring, and uptime tracking to support performance diagnostics and incident response.
- Earned AWS Solutions Architect Certification, demonstrating a strong understanding of scalable and observable cloud architecture
Education
Post Graduate Diploma - Computer Applications
Dr. C.V. Raman University
01.2016 - 01.2017
Bachelor of Engineering - Mechanical Engineering
Rungta College of Engineering and Technology
01.2008 - 01.2012
Skills
- Azure platform expertise
- GCP
- Infrastructure as code with Terraform
- Azure Resource Manager templates
- Azure DevOps
- Git
- Jenkins
- Pyspark
- Docker
- Databricks proficiency
- Azure Synapse
- Power BI
- Python
- sql
- Cost management api
- Datafactory
Websites
Certification
AWS Certified Solutions Architect - Associate
Accomplishments
- Cleared RRB ALP/TECH CEN-2018 (Top 50, Score: 83.5/100)
- Cleared GATE Exam twice
Timeline
Manager - Platform Engineering
Landmark Group
03.2021 - Current
Intern - Cloud & Infrastructure
Softway Solutions
08.2020 - 11.2020
Post Graduate Diploma - Computer Applications
Dr. C.V. Raman University
01.2016 - 01.2017
Bachelor of Engineering - Mechanical Engineering
Rungta College of Engineering and Technology
01.2008 - 01.2012
Similar Profiles
Ramya ShuklaRamya Shukla
Buyer at Landmark GroupBuyer at Landmark Group
Kislaya Kislaya null
Senior Software Development Engineer at Landmark GroupSenior Software Development Engineer at Landmark Group
SATISH PUNDIRSATISH PUNDIR
Retail business manager at Landmark GroupRetail business manager at Landmark Group
Faiez NoorFaiez Noor
Key Accounts Lead at Landmark GroupKey Accounts Lead at Landmark Group
Ankit RajAnkit Raj
Technology Account Manager at IBMTechnology Account Manager at IBM