Venkatesan V
Bangalore
Summary
Overall 8+ years of deep expertise in Big Data infrastructure, on-premise data center management, and Linux administration
Overview
8
8
years of professional experience
1
1
Certification
Work History
Site Reliability Engineer 3
PhonePe
BANGALORE
10.2021 - Current
- Kafka Version Migration: Orchestrated a seamless major version upgrade from Kafka 2.x to 3.x for clusters supporting 100+ active consumers, ensuring zero data loss and maintaining production SLAs.
- Infrastructure Automation: Developed comprehensive SaltStack formulas to automate the end-to-end installation and lifecycle management of Ambari, Hadoop, and Kafka clusters, eliminating manual configuration.
- Operating System Modernization: Led the fleet-wide migration from Ubuntu 16.04 (Xenial) to 18.04 (Bionic), managing package compatibility and performance tuning for distributed data services.
- Kafka Migration Architecture: Designed and implemented MirrorMaker strategies for large-scale data migration between Kafka clusters, facilitating cross-region data consistency.
- Disaster Recovery (DR): Engineered and managed HBase replication for DR sites, ensuring high availability and business continuity for mission-critical datasets.
- Advanced Tooling: Integrated and configured Cruise Control to automate Kafka partition rebalancing and resource utilization, and authored Python scripts to automate repetitive manual tasks.
- Platform Reliability (BAU): Managed daily operations for Kafka, HBase, HDFS, Elasticsearch, and MySQL, including node commissioning/decommissioning, index lifecycle management, and health monitoring.
- Incident Management: Served in a 24/7 on-call capacity, performing root cause analysis (RCA) for complex production issues across the distributed stack to minimize MTTR and improve system stability.
Hadoop DevOps Engineer
Organization: Infosys Client: World Bank
Chennai
02.2019 - 10.2021
- Involved in start to end process of Hadoop cluster deployment on Physical Platform including installation, configuration and monitoring the cluster.
- Experience in High Availability in cluster for existing Name Node/Resource Manager and experience in Name Node backup and restore
- Experience in Application/User On-boarding, HDFS space management, Queue Management, Workload Management, ACL implementation and Platform optimization
- Involved in Hardware replacement and upgrades (Disk, Cache Module replacement, Firmware and Network Upgrade) of hadoop cluster
- Experience in Installing, Configuring, Supporting and Managing of HDP 3.0 and Hadoop platform
- NIFI and Kafka Administration on HDF 3.0 hadoop platform.
- Creating home directories for users and providing permissions.
- Hadoop Security Kerberos, SSL and Ranger.
- Commission and Decommission nodes or services.
- Automated cluster tasks using ansible playbooks and roles.
- Collaborating with application teams to deploy applications on DEV, UAT and PROD environment.
- Design and configure Topics in new Kafka cluster and backup for all the instances for Kafka.
- Extensively used shell scripting for various administrative activities
- Deploy, Configure, Maintain compute on Azure Cloud
- Troubleshoot Azure related issues and engage internal teams for issue resolutions
- Infra-Structure support on Azure using various services like Virtual Machine, Storage, Azure CDN, Virtual Network, Resource Group.
Production Support Engineer
Organization: Altimetrik Client: PayPal
Chennai
11.2017 - 02.2019
- Migration of Teradata applications to Hadoop using Sqoop and Hive via Internal Framework
- Worked on data migration between Teradata and Hadoop
- Experienced in developing Hive Queries which converts as MapReduce programs on different data formats like Text file, CSV files etc
- Hands on experience using Teradata utilities (TPT, B-TEQ, Fast Load, Multiload, Fast Export).
- Migration of Hadoop applications to different cluster
- Worked extensively with Sqoop for importing and exporting data from Teradata and SAP Hana database to HDFS and HIVE
- Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance
- Experience working independently and as part of a team to debug application issues working with configuration files, databases and application log files
- Good Knowledge in debugging batch jobs running in Sqoop, Hive, Spark and Teradata
Education
Bachelor of Engineering With 72% -
AMS College Of Engineering
Chennai2014
Skills
- Hadoop and HBase
- HDFS and Kafka
- Elasticsearch and MySQL
- Linux administration
- Shell scripting
- Ansible/SaltStack
- Python programming
- Docker and Kubernetes
- Cloud services (Azure, AWS)
- Version control (GitLab)
Certification
RedHat Certified System Administrator
RedHat Certified Engineer
AZ 103 Microsoft Azure Administrator
Confluent Certified Kafka Administrator
Certified Kubernetes Administrator
RedHat Certified Engineer
AZ 103 Microsoft Azure Administrator
Confluent Certified Kafka Administrator
Certified Kubernetes Administrator
Additional Information
LinkedIn: https://www.linkedin.com/in/sai-venkatesh-5275a856
GitHub: https://github.com/saivenkatg5
Timeline
Site Reliability Engineer 3
PhonePe
10.2021 - Current
Hadoop DevOps Engineer
Organization: Infosys Client: World Bank
02.2019 - 10.2021
Production Support Engineer
Organization: Altimetrik Client: PayPal
11.2017 - 02.2019
Bachelor of Engineering With 72% -
AMS College Of Engineering