M. Atif
DevOps Engineer
Pune
Summary
Data & Platform Engineer with 4+ years experience, specializing in batch and stream data processing. Proficient in Kubernetes and Databricks for efficient pipeline orchestration and hybrid-cloud infrastructure management.
Overview
4
4
years of professional experience
1
1
Certification
Work History
Development Engineer 3
Comcast
12.2021 - Current
- Engineered and executed large-scale data migration from legacy CDH/HDP clusters to modern AWS S3 and MinIO object storage platforms.
- Hands-on experience configuring and maintaining core AWS services, including S3, IAM, Route 53, and deploying event-driven solutions with Lambda.
- Automated the deployment and configuration of hybrid infrastructure (Hadoop, Kubernetes, MinIO) and cloud resources using Terraform and Ansible.
- Managed the full lifecycle (deployment, maintenance, and troubleshooting) of data infrastructure, including Hadoop and MinIO clusters, with expertise in both on-premises Kubernetes and AWS EKS deployments.
- Implemented continuous integration and deployment (CI/CD) pipelines using Jenkins for automating minIO lifecycle policies.
- Ensured operational stability by applying strong network fundamentals to diagnose and resolve complex network-related issues affecting cluster communication and data transfer.
- Implemented robust data monitoring (validation checks, data counts, processing lag) and created performance dashboards using AWS CloudWatch.
- Conducted root cause analysis on data anomalies (e.g., unexpected nulls, schema drift) to resolve upstream data quality issues and prevent recurrence.
- Leveraged Spark/PySpark/Scala to build streaming and batch workflows for schema enforcement, data enrichment, and reliable delivery to Delta Lake and Kafka/Kinesis.
Education
Bachelor of Engineering -
Skills
Certification
Kubernetes Administrator by linux foundation
Timeline
Development Engineer 3 - Comcast
12.2021 - Current
Pune University - Bachelor of Engineering,