Abhishek S
Senior Hadoop Admin
Bengaluru,KA
Summary
Passionate and self-motivated Hadoop Administrator with over 7.5 years of experience in managing, configuring, and supporting large-scale Hadoop clusters across Pharma and Banking domains. Extensive experience with Cloudera and MapR distributions, along with a strong background in Hadoop components, data replication, and security management. Proven ability to optimize system performance, collaborate with cross-functional teams, and handle challenging assignments with a positive attitude. Detail-oriented, organized, and meticulous employee. Works at fast pace to meet tight deadlines. Enthusiastic team player ready to contribute to company success.
Overview
8
8
years of professional experience
4
4
years of post-secondary education
1
1
Certification
Work History
Senior Hadoop Engineer
Altimetrik
12.2023 - Current
- Provide end-to-end platform support for a Hadoop ecosystem spanning 18 clusters with over 15,000 servers
- Lead the administration of Hadoop clusters, including installation, configuration, and monitoring of HDFS, YARN, Spark, Hive, Kafka, and other Hadoop components
- Implement and manage data replication, backup, and disaster recovery strategies to ensure high availability
- Administer access control and security policies using Kerberos and Ranger, ensuring compliance with industry standards
- Optimize YARN resource management for efficient job scheduling and resource allocation
- Conduct post-mortem analyses of major incidents, identifying root causes, and implementing preventative measures
- Troubleshoot and resolve issues in Kubernetes-based environments, ensuring smooth operation of containerized Hadoop jobs and other microservices
- Collaborate with developers to optimize Hive, Spark, and MapReduce jobs, reducing execution time
- Manage and troubleshoot Tableau and Machine Learning Platform (MLP) jobs
- Conduct regular health checks and audits of Hadoop clusters to ensure data integrity and security.
Customer Operations Engineer
Cloudera
05.2022 - 11.2023
- Served as a frontline engineer for Cloudera products, focusing on HDP and CDP data processing components like Kafka, NiFi, Spark, YARN, and Schema Registry
- Assisted customers with resolving issues related to HDP and CDP components, leveraging extensive hands-on experience
- Replicated customer issues in lab clusters and executed troubleshooting procedures
- Managed Cloudera Manager, HDFS, Zookeeper, Hive, and MapReduce, with a focus on maintaining cluster health
- Led upgrade projects for customers transitioning from HDP to CDP or updating CDP to newer versions
- Integrated services with LDAP and ensured seamless operation
- Analyzed logs and service configurations to troubleshoot and resolve cases
- Utilized various ticketing tools, including JIRA, for incident management.
Hadoop Administrator
Tata Consultancy Services
11.2020 - 04.2022
- Set up Cloudera clusters hosted on Azure, managing end-to-end implementation
- Installed, configured, maintained, and troubleshot Hadoop clusters, ensuring high availability and performance
- Maintained cluster health via Cloudera Manager, addressing disk-related issues, and managing resource allocation
- Commissioned and decommissioned servers, optimizing resource utilization
- Troubleshot and fine-tuned MapReduce, Hive, Spark, and Oozie jobs for optimal performance
- Managed server memory, CPU, OS, and networking issues, ensuring a stable environment
- Performed changes in the Hadoop environment using CLI, ensuring compliance and stability
- Set up alerts from Cloudera Manager as part of proactive cluster monitoring.
Hadoop Administrator
Atos Syntel
01.2017 - 05.2020
- Analyzed log files for Hadoop ecosystem services to identify and resolve issues
- Supported HDFS data storage and managed MapReduce jobs
- Optimized Hive, MapReduce, Spark, Oozie, and Datameer jobs to improve performance
- Built single-node and multi-node clusters, installing and configuring various services
- Documented resolutions and created a Knowledge Database (KEDB) for future reference
- Configured backup and recovery from NameNode failures, ensuring data integrity
- Managed incidents using ServiceNow, resolving within specified SLAs.
Education
Bachelor of Engineering in Computer Science and Engineering -
Sri Eshwar College of Engineering
01.2012 - 04.2016
Skills
Hadoop Technologies: HDFS, YARN, Hive, Spark, Kafka, MapReduce, Zookeeper, HBase, NiFi
Hadoop Distributions: HDP, CDP, MapR
Containerization & Orchestration: Kubernetes, Docker
Machine Learning Platforms: MLP (Machine Learning Platform)
Cloud Platforms: Azure
Security: Kerberos, Ranger
Tools: ServiceNow, JIRA , Nagios , Ganglia , Splunk
Programming/Scripting: Shell Scripting
Certification
Azure Fundamentals AZ-900
Personal Information
- Date of Birth: 01/07/94
- Gender: Male
- Nationality: Indian
Languages
English
Malayalam
Tamil
Hindi
Timeline
Senior Hadoop Engineer
Altimetrik
12.2023 - Current
Customer Operations Engineer
Cloudera
05.2022 - 11.2023
Hadoop Administrator
Tata Consultancy Services
11.2020 - 04.2022
Hadoop Administrator
Atos Syntel
01.2017 - 05.2020
Bachelor of Engineering in Computer Science and Engineering -
Sri Eshwar College of Engineering
01.2012 - 04.2016