M. Atif
DevOps Engineer
Pune
Summary
Data & Platform Engineer with 4+ years experience, specializing in batch and stream data processing. Proficient in Kubernetes and Databricks for efficient pipeline orchestration and hybrid-cloud infrastructure management.
Overview
4
4
years of professional experience
1
1
Certification
Work History
Development Engineer 3
Comcast
12.2021 - Current
- Engineered and executed large-scale data migration from legacy CDH/HDP clusters to modern AWS S3 and MinIO object storage platforms.
- Hands-on experience configuring and maintaining core AWS services, including S3, IAM, Route 53, and deploying event-driven solutions with Lambda.
- Automated the deployment and configuration of hybrid infrastructure (Hadoop, Kubernetes, MinIO) and cloud resources using Terraform and Ansible.
- Managed the full lifecycle (deployment, maintenance, and troubleshooting) of data infrastructure, including Hadoop and MinIO clusters, with expertise in both on-premises Kubernetes and AWS EKS deployments.
- Implemented continuous integration and deployment (CI/CD) pipelines using Jenkins for automating minIO lifecycle policies.
- Ensured operational stability by applying strong network fundamentals to diagnose and resolve complex network-related issues affecting cluster communication and data transfer.
- Implemented robust data monitoring (validation checks, data counts, processing lag) and created performance dashboards using AWS CloudWatch.
- Conducted root cause analysis on data anomalies (e.g., unexpected nulls, schema drift) to resolve upstream data quality issues and prevent recurrence.
- Leveraged Spark/PySpark/Scala to build streaming and batch workflows for schema enforcement, data enrichment, and reliable delivery to Delta Lake and Kafka/Kinesis.
Education
Bachelor of Engineering -
Skills
Spark
SQL
Python
Terraform
System administration
Monitoring and logging
Infrastructure automation
Containerization technologies
Configuration management
Maintenance and troubleshooting
Continuous integration
Continuous deployment
Certification
Kubernetes Administrator by linux foundation
Timeline
Development Engineer 3 - Comcast
12.2021 - Current
Pune University - Bachelor of Engineering,