Somanath Vijay Naik
Big Data Engineer
Pune
Summary
Around 12+ years of IT experience including 7 years in Big Data Technologies.
Well versed with Hadoop Map Reduce, HDFS, Hive, Impala, Sqoop, Flume, Yarn, Zookeeper, and Oozie
In depth understanding/knowledge of Hadoop Architecture and various components.
Experience in installation, configuration, support and management of a Hadoop Cluster.
Worked on both Hadoop distributions: Cloudera and Hortonworks.
Strong Knowledge in Hadoop Cluster Capacity Planning, Cluster Monitoring.
Capability to configure scalable infrastructures for HA (High Availability)
Experience in balancing the cluster after adding/removing nodes.
Experience in Setting up Data Ingestion tools like Flume, Sqoop and Kafka
Experience in configuring Zookeeper to coordinate the servers in clusters.
Experience in setting up Name Node high availability for major production cluster
Experience in designing Automatic failover control using zookeeper and quorum journal node
Experience in creating, building and managing public and private cloud Infrastructure
Experience in working with different file formats and compression techniques in Hadoop
Experience in analyzing existing Hadoop cluster, Understanding the performance bottlenecks and providing the performance tuning solutions accordingly.
Strong experience in System Administration, Installation, Upgrading, Patches, Migration, Configuration, Troubleshooting, Security, Backup, on Linux (RHEL) systems.
Experience in working large environments and leading the infrastructure support and operations.
Migrating applications from existing systems like MySQL, oracle, db2 and Teradata to Hadoop.
Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster.
Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster.
Knowledge in AWS cloud
Possess good knowledge in creating and launching EC2 instances using AMI’s of Linux, Ubuntu, RHEL, and Windows
Overview
12
12
years of professional experience
7
7
years of post-secondary education
Work History
Lead - Big Data Platform Engineer
India Pvt Ltd
04.2021 - 12.2022
- Involved in maintaining Hadoop cluster in Production, Test, Development, UAT environments
- Configured and Installed Hive and Impala in Hadoop cluster
- Good knowledge in using ticketing tool JIRA
- Experience in automating tasks through Ansible
- Performance tuning for Impala and Hive jobs
- Performance tuning for Spark jobs
- Analyse query profile and troubleshoot
- Configuring Impala pools
- Looking into impala issues
- Verifying stats and advising user to collect the stats
- Increasing MEM LIMIT at pool level based on the requirement
- Working access level issues for DBs, users and groups
- POC for customers, attending client call to discuss about the issues and future changes
- Managing Incidents, RFCs till closure
- Experience in troubleshooting errors and finding RCA in the Hadoop cluster
- Configured Resource management in Hadoop through dynamic resource allocation in Cloudera Manager
- Experience in Upgrading Cloudera manager and CDH to the latest released versions using Ansible
- Worked on CDP issues and fixed
- Experience in GIT using bit backet tool to build the objects
- Used shell script to automate the deployment process
- Data ingestion using Kafka
- Data pipelining
Senior Hadoop Administrator
phData Solutions Pvt Ltd
10.2019 - 04.2021
- Managing and making team to learn process and procedures
- POC for customers, attending client call to discuss about the issues and future changes
- Configured Resource management in Hadoop through dynamic resource allocation in Cloudera Manager
- Experience in Upgrading Cloudera manager and CDH to the latest released versions using Ansible
- Experience creating Kafka topics and running producer consumer
- Experience in securing Hadoop clusters using Kerberos
- Experience in automating tasks through Ansible
- Experience in automating of upgrading Cloudera manager server and setting file permissions, files deployments, starting and stopping services using Ansible
- Installed Archway workspace creation tool on customer environments
- Experience in Upgrading Oracle BDA
- Setting up and configuring Kafka Environment in Windows from the scratch and monitoring it
- Created a data pipeline through Kafka Connecting two different clients Applications
- Worked on setting up 3 Instances in UAT/STAGING environment and in Production environment
- Hands-on experience in standing up and administrating on-premise Kafka platform
- Experience managing Kafka clusters both on Linux environment
- Configuring Impala pools
- Looking into impala issues
- Verifying stats and advising user to collect the stats
- Increasing MEM LIMIT at pool level based on the requirement
- Designed and implemented by configuring Topics in new Kafka cluster in all environments
- Integrated Apache Kafka for data ingestion
- Successfully Generated consumer group lags from Kafka using their API Kafka- Used for building real-time data pipelines between clusters
- Experience configuring and setting up Kudu on existing cluster
- KUDU Tablet rebalancing
- Adding new Tablet servers to the cluster
- Looking into KUDU issues
- Recovering Tablet replica from disk failure or disk full
- Backup and restore KUDU tables
- NTP sync issues
- Setting up Rack awareness
- Monitoring cluster health
- Changing directory configuration
- Rebuilding KUDU Filesystem Layout
- Migrating KUDU data.
Hadoop Administrator
Cisco Systems, VARITE India Pvt Ltd
02.2019 - 10.2019
- Install, configure and administer Hdfs, Hive, Sentry, Pig, HBase, Oozie, Sqoop, Spark and Yarn
- Worked on the installation and configuration of Hadoop HA Cluster
- Involved in capacity planning and design of Hadoop clusters
- Setting up alerts in Cloudera Manager for the monitoring of Hadoop Clusters
- Setting up security authentication using Kerberos security
- Create directories and setup appropriate permissions for different applications
- Involved in planning and implementation of Hadoop cluster Upgrade
- Installation, Configuration and administration of CDH on Red Hat Enterprise Linux 7.6
- Setting up MongoDB cluster
- Setting up Postgres cluster and OpenLDAP as mirror mode
- Configuring and setting up WSO2 API and IS
- Integrating Hadoop cluster with other internal components
- Working with development team to fix the issue
- Installed and configured RabbitMQ.
Hadoop Administrator
Aricent Technologies, ALTRAN
05.2016 - 08.2018
- Installed, configured, upgraded, and applied patches and bug fixes for Prod, Test and Dev Servers
- Installed/Configured/Maintained Hadoop clusters in dev/test/Prod environments
- Install, configure and administer Hdfs, Hive, Sentry, Pig, HBase, Oozie, Sqoop, Spark and Yarn
- Worked on the installation and configuration of Hadoop HA Cluster
- Involved in capacity planning and design of Hadoop clusters
- Setting up alerts in Cloudera Manager for the monitoring of Hadoop Clusters
- Setting up security authentication using Kerberos security
- Commission and decommission the data nodes from cluster
- Write and modify UNIX shell scripts to manage CDH environments
- Installed and configured Flume, Hive, Sqoop and Oozie on the Hadoop cluster
- Create directories and setup appropriate permissions for different applications
- Involved in planning and implementation of Hadoop cluster Upgrade
- Installation, Configuration and administration of CDH on Red Hat Enterprise Linux 6.6
- Used Sqoop to import data into HDFS from Oracle database
- Detailed analysis of system and application architecture components per functional requirements
- Review and monitor system and instance resources to insure continuous operations (i.e., database storage, memory, CPU, network usage, and I/O contention)
- On call support for 24x7 Production job failures and resolve the issue in timely manner.
Hadoop Administrator
LG Soft India Pvt Ltd
09.2014 - 04.2016
- Responsible for Cluster maintenance, Adding and removing cluster nodes, Cluster Monitoring and Troubleshooting, Manage and review data backups, Manage and review Hadoop log files
- Played responsible role for deciding the hardware configurations for the cluster along with other teams in the company
- Resolving tickets submitted, P1 issues, troubleshoot the error documenting, resolving the errors
- Adding new Data Nodes when needed and running balancer
- Responsible for building scalable distributed data solutions using Hadoop
- Continuous monitoring and managing the Hadoop cluster through Ganglia and Nagios
- Done major and minor upgrades to the Hadoop cluster
- Done stress and performance testing, benchmark for the cluster
- Working closely with both internal and external cyber security customers
- Research effort to tightly integrate Hadoop and HPC systems
- Responsible for Cluster maintenance, Adding and removing cluster nodes, Cluster Monitoring and Troubleshooting, Manage and review data backups, Manage and review Hadoop log files
- Deployed, and administered Hadoop clusters
- Compared Hadoop to commercial big-data appliances from Netezza, XtremeData, and LexisNexis
- Published and presented results
- Worked on developing Linux scripts for Job Automation
- Resolving tickets submitted by users, P1 issues, troubleshoot the error documenting, resolving the errors
- Developing machine-learning capability via Apache Mahout.
Linux Administrator
Wipro Infotech, Consulting
08.2012 - 09.2014
- Providing the specific solution/technology/product consulting and technical support as part of a team doing the final system support and
- Implementation for a customer activities include
- Configuring and troubleshooting network installations
- Understand business requirements for customer base and be able to translate them into technical requirements
- Creating, present and document technical solutions
- Providing consultative support and mentor other less experienced Systems Engineers in assigned major task
- Working effectively as a team member and assume a leadership role for the team
- Understanding the end user requirement on that basic dividing team work to give the solution within end user deadline
- Handling daily base IT related issue of end user and production system issue
- Configuration & management of Linux OS in Chassis server and Rack server based on end user requirement
- Installation of Red hat Enterprise operating System( CD,NFS & Kickstart)
- Experience installing, administering Experience administering Server/Data Center
- Administration of LINUX services
- Troubleshooting server performance issues & Mail Configuration
- Restoring the server data
- Maintaining Server inventory list
- Addition, Deletion of the servers, scheduling the jobs using crontab
- Hardening the LINUX servers
- Linux (Red hat, CentOS, Fedora, Ubuntu)
- Proven experience in Web/Enterprise/Telecom Server Domain (IBM, HP,DELL and Cisco)
- Strong configuration management knowledge in complex systems involving multiple versions of hardware and OS
- Some experience with database MySQL,
- Experience assembling server hardware to bring up new sever / workstation systems
- Configuring NTP and NFS servers
- Configuring log rotation for log files
- Implementing password policy
- Network administration activities
- Installation MySQL DB
- Installing and Configuring VMware ESXi
- Knowledge of KVM virtualization
- Server building as per client requirement.
Technical Support Engineer
IBM India Pvt Ltd
09.2010 - 08.2012
- Administration of LINUX services
- Troubleshooting server performance issues & Mail Configuration
- Restoring the server data
- Maintaining Server inventory list
- Addition, Deletion of the servers, scheduling the jobs using crontab
- Hardening the LINUX servers
- Linux (Red hat, CentOS, Fedora, Ubuntu)
- Proven experience in Web/Enterprise/Telecom Server Domain (IBM, HP,DELL and Cisco)
- Strong configuration management knowledge in complex systems involving multiple versions of hardware and OS
- Some experience with database MySQL,
- Experience assembling server hardware to bring up new sever / workstation systems
- Configuring NTP and NFS servers
- Configuring log rotation for log files.
Education
Bachelor of Science - Information Technology
Karnataka State Open University
08.2010 - 05.2013
Diploma - Computer Science & Engineering
Board of Technical Education
07.2000 - 05.2004
SSLC -
Athani Vidyavardhaka Sounsthe Athani
Athani, Belgaum, Karnataka, India 06.1999 - 04.2000
Skills
TECHNICAL SKILLSundefined
Additional Information
- Experience in working large environments and leading the infrastructure support and operations. Migrating applications from existing systems like MySQL, oracle, db2 and Teradata to Hadoop. Experience in administering the Linux systems to deploy Hadoop cluster and monitoring the cluster. Experience on Commissioning, Decommissioning, Balancing, and Managing Nodes and tuning server for optimal performance of the cluster. Knowledge in AWS cloud Possess good knowledge in creating and launching EC2 instances using AMI’s of Linux, Ubuntu, RHEL, and Windows
Timeline
Lead - Big Data Platform Engineer
India Pvt Ltd
04.2021 - 12.2022
Senior Hadoop Administrator
phData Solutions Pvt Ltd
10.2019 - 04.2021
Hadoop Administrator
Cisco Systems, VARITE India Pvt Ltd
02.2019 - 10.2019
Hadoop Administrator
Aricent Technologies, ALTRAN
05.2016 - 08.2018
Hadoop Administrator
LG Soft India Pvt Ltd
09.2014 - 04.2016
Linux Administrator
Wipro Infotech, Consulting
08.2012 - 09.2014
Technical Support Engineer
IBM India Pvt Ltd
09.2010 - 08.2012
Bachelor of Science - Information Technology
Karnataka State Open University
08.2010 - 05.2013
Diploma - Computer Science & Engineering
Board of Technical Education
07.2000 - 05.2004
SSLC -
Athani Vidyavardhaka Sounsthe Athani
06.1999 - 04.2000
Similar Profiles
Randhir Kumar PrasadRandhir Kumar Prasad
Stacker at Engineering and Allied Services Pvt. Ltd, Lift & Shift India Pvt. Ltd, Roadwings International Pvt. Ltd, Western Carriers India Pvt. Ltd, Tractor India Pvt. Ltd, Transworld Terminal Pvt. LtdStacker at Engineering and Allied Services Pvt. Ltd, Lift & Shift India Pvt. Ltd, Roadwings International Pvt. Ltd, Western Carriers India Pvt. Ltd, Tractor India Pvt. Ltd, Transworld Terminal Pvt. Ltd
Akhila ShilpaAkhila Shilpa
Senior Project Manager at IBM India Pvt Ltd/ Kyndryl India Pvt LtdSenior Project Manager at IBM India Pvt Ltd/ Kyndryl India Pvt Ltd
Soumya SinghSoumya Singh
Sales Analyst 3 at NetApp India Pvt. LTD, Varite India Pvt LtdSales Analyst 3 at NetApp India Pvt. LTD, Varite India Pvt Ltd
Aditya SawantAditya Sawant
Sr. Manager Network Operations at Vitech Systems Asia Pvt Ltd.Sr. Manager Network Operations at Vitech Systems Asia Pvt Ltd.
Pranjal SharmaPranjal Sharma
Advanced Software Developer at AmdocsAdvanced Software Developer at Amdocs