Rahul Shrivastava
Bangalore
Summary
Overall 13.5 years of experience with strong domain knowledge of IT Operations & Big Data Technologies, Data Engineering that includes-Data operations, AWS- S3, EC2, EMR, Redshift, Athena, Glue, Lambda, IAMSQS, SNS, Cloud Data watch, Administration, Monitoring and Management of Network systems, Data Center, Application & Production Support, and end-to-end IT Operations Big Data enthusiast with 5+ years expertise in Bigdata- Hadoop Cloudera, Snowflake, Data Engineering, data warehousing and broad knowledge of ETL tools. Proven history of optimizing database efficiency and dataquality. Certified Snowflake SnowPro Core & AWS solution architect Associate Certification with comprehensive handling of its architecture. Specializes in data migration to cloud-based platforms and incorporates strategic use of Snowflake's unique features to improve data accessibility and security. Experienced in leading both small and large sized teams and managed their daily operations and team’s objectives. Experienced in the area of maintaining Hadoop-Big Data, ETL, Snowflake and Unix/Linux History of successfully managing multiple Hadoop-Big Data. Demonstrates a consistently strong work ethic and capable of delivering timely results, possesses strong analytic and problem- solving skills. On-call production support and Activities, profound knowledge of Hadoop/ Big Data technology. Keen customer centric approach with skills that addresses client priorities, resolving escalations within prescribed turnaround time, thereby attaining client satisfaction and high compliance scores Worked with a diverse set of Stakeholders [Business Units, Trading Partners, Functional Groups, Consultants, Vendors, External Agencies (Software/Hardware Suppliers)] on matters relating to systems for establishing, implementing & improving Infrastructure Solutions. An innovative, creative professional with strong communication, analytical, and interpersonal skill.
Overview
14
14
years of professional experience
2011
2011
years of post-secondary education
5
5
Certifications
Work History
Manager (ETL)
Commonwealth Bank of Australia
02.2021 - Current
- Heading a project size of 15+ with direct reports
- Experience on Application and Process Monitoring
- Implementing ETL pipelines within and outside of a data warehouse using Snowflakes Snow SQL
- Assist with production issues in Data Warehouses like reloading data, transformations, and translations
- Experienced on AWS - S3, EC2, EMR, RedShift, Athena, Glue, Lambda, Athena, IAM Roles, Lake formation Services
- Develop a Database Design and Reporting Design based on Business Intelligence and Reporting requirements
- Technically sophisticated & astute Big data Professional & ETL, Snowflake, Redshift as Manager (Big data/ Snowflake)
- Experience on Job Monitoring utility (Autosys)
- Supporting approx
- 1.5 Lakh Production jobs in Hadoop, Abinitio, AWS
- Solid understanding of software development principles and strong knowledge of big-data programming paradigms
- Experience in designing and building a scalable infrastructure platforms to collect and process very large amounts of structured and unstructured data
- Experienced on Hadoop Technology and its eco system components Sqoop, Hive, HBase, Oozie, Impala, Spark, Pyspark, Zookeeper, Hue, Journal nodes
- Experienced on Cloudera Manager
- Experienced on Hadoop administration and its eco system components Sqoop, Hive, HBase, Oozie, Impala, Spark, ZooKeeper, Hue, Journal nodes
- Experience in commissioning and decommissioning of nodes, keeping track of jobs, monitoring critical alerts, Configuring high availability, configuring data backups and data purge
- Analysing Log files for Hadoop ecosystem services and finding root cause
- Supporting & Monitoring Omnia Jobs & Abinitio Jobs by Control centre
- Experience on Job Scheduler & Monitoring utility (Autosys & Control center)
- Provided governance over the outcomes of the operational delivery of applications and services within project
- Responsible for hiring, on-boarding, training, transition, deploy in production and support operational continuity
- Built the project from planning, develop, hire to go-live
- Accountable for ensuring operational standards and processes are met by the team
- Responsible for end-to-end operational delivery management of relevant IT Services including maintaining Service Level Agreements and Operational Level Agreements for data services, incidents and problem tickets, operational service life-cycle management, continuous service improvement, service integration, governance over quality control from projects
- Drove the operational delivery such that the services are provided in a consistent and reliable manner to the Bank and its customers within agreed service levels
- Overseen the integration of services from multiple application service providers, monitor service provider compliance with operational policies and processes, and escalate issues of non-compliance as appropriate
- Responsible to provide advice and support to Service Managers/Owners and IT Commercial Managers to ensure SLAs, SLOs and financial targets and results are achievable and adequate
- Provided strategic advice and recommendations to leadership in the development, implementation
- Established and implemented short- and long-range goals, objectives, strategic plans, policies, and operating procedures; monitored and evaluated programmatic and operational effectiveness
- Responsible for employee management from setting up individual objectives, goals, development actions till employee separation
- Responsible to define, manage, review and consolidate KPIs related to respective area of responsibility to ensure compliance and optimal performance
- Responsible for employee performance appraisal(mid-year/year-end) includes performance review, individual rating, 1:1 feedback, calibration sessions, final discussion
- Developed team leadership skills including team motivation, process reviews and performance management
- Responsible for all training needs such as to design, coordinate, and implement training programs for the team
- Managed and provided day-to-day leadership to various technical, professional, and administrative project activities
Senior Engineer – Platform
FIS Global Business Solution
08.2019 - 02.2021
- Used Cloudera distribution in Hadoop Bigdata Environment & AWS
- Highly skilled on Hadoop Cluster Setup and Hadoop- Bigdata
- Experience in commissioning and decommissioning of nodes, keeping track of jobs, monitoring critical alerts, configuring high availability, configuring data backup and data purge using by ETL & AWS
- Worked with Development team for deployment of the new code or script in Production environment
- Worked on Cloudera Manager
- Having experience on Hadoop administration and its eco system components Sqoop, Pig, Hive, Impala, Spark, HBase, Oozie, Zookeeper, Hue, Journal nodes
- Worked on R-studio tool
- Worked on Unravel for Jobs level optimization and performance improvement for Jobs
- Worked on CDH upgrade & Maintenance activities
- Performance tuning of Hadoop Cluster and Hadoop Jobs
- Supporting the IT Infrastructure that meets all service level agreements and a 24x7 availability, response time parameters for the incident management and resolution
- Design, develop, implement and continuously improve the Incident, Problem, Change Management processes, Procedures, and methodologies based upon ITIL principles, and governing policies
- Organize, facilitate, and lead technical teams to work together to resolve major incidents
- Worked cross-functional with other support teams/ towers but not limited to Change management, Service asset and configuration management, Knowledge management processes
- Responsible for completion of assigned tasks within SLA
- Acted as a further escalation point for all critical issues
- Represented the team in service/process owners meeting and reviewed recent outages & action plans
- Accurate and timely communications are sent to internal and approved external stakeholders including management
- Take an end-to-end view of the services and seeks to be proactive in the continuous improvement of service delivery
- Responsible for Transition planning and support, Release and deployment management, Service validation & testing, Change evaluation
- Review, control and manage Change requests as per ITIL standards
- Reviews all implemented changes to ensure that they have been carried out properly and to understand what went well and what did not
- Provide input, document requirements and support the design and delivery of training programs
Associate –Line of Business - GTI (Global technology Infrastructure)
JP Morgan Service India Pvt. Ltd
12.2014 - 08.2019
- Total 6 Cluster using in Hadoop environment with Cloudera and Hortonworks distribution
- Solid understanding of software development principles and strong knowledge of big- data programming paradigms and techniques with strong leadership, planning and organization skills
- Experience in designing and building a scalable infrastructure and platforms to collect and process very large amounts of structured and unstructured data
- Experience in commissioning and decommissioning of nodes, keeping track of jobs, monitoring critical alerts, configuring high availability, configuring data backups and data purge
- Performing maintenance, backups and system upgrades including service packs, patches, hot fixes and security Configuration issues raised by clients
- Starting/Stopping the Hadoop services during OS patching and during hardware failures on the data nodes
- Worked & Support all Linux Red-hat RHEL 6.x problems
- & HP-ILO, IDRAC, for physical server’s console
- Worked on HP Service Manager ticketing tool I was using, Netcool Tool for monitoring of the all Hadoop Server
- I have experienced of SLA response and resolution times and ensure cases are regularly updated
- I have knowledge of Escalation of problems in a timely manner as per the escalation matrix
- Responsible for driving the Tasks for completion of assigned tasks within SLA
- Acted as a further escalation point for all critical issues
Associate Engineer
Collabera Technology Pvt. Ltd
09.2014 - 12.2014
Linux Administrator
Accenture service Pvt. Ltd
09.2013 - 08.2014
- Installation and configuration of Linux server in VMware Esxi 4x/5x
- Managing & maintaining Red Hat Enterprise Linux & SUSE Linux Platform, VMware ESXi
- Day to day server management
- File System Management LVM NFS Samba
- User Management User creation/deletion, User authentication administration
- Software installations (Installation through “RPM”, “YUM”)
- Job scheduled by (CRONTAB and AT)
- House Keeping the File System when there is a space issue
- Installation and configuration of infrastructure Servers DNS, DHCP, FTP, NFS & SAMBA
- Installing and Configuring Linux OS and Nagios monitoring on Linux
- Reporting and communicating the status of Infrastructure Servers on Daily basis to the management
- Maintain and update the documentation regularly
- Ensuring system security compliances and standards for Red Hat servers
- Installed and configure Nagios server in order to monitor the remote server and services
- Fixing system monitoring compliances and issues
Data Center Operator – Linux Admin
HP Service India Pvt. Ltd
10.2011 - 03.2013
- Ensuring system security compliances and standards for Red Hat servers
- Installed and configure Nagios server in order to monitor the remote server and services
- Fixing system monitoring compliances and issues
- Monitored Server health status
- Maintained necessary back-up of the important files, scheduling the job to take the backup of configurations files
- Package management of all Linux Servers
- Also Using LVM for extending Logical volume, volume group etc
- Worked Logical Volume Manager (LVM) Approach for creating physical volume, volume group, and logical volume
- I have created of file system
- Fixing system monitoring compliances and issues
- Maintained necessary back-up of the important files, Scheduling the job to take the backup of configurations files
Education
B.E - Electronics & Communication Engineering
Rajiv Gandhi Proudyogiki Vishwavidyalaya
Bhopal, Madhya PradeshSkills
- Cloud - AWS
- S3
- VPC
- ELB
- EBS
- Cloud Front
- Cloud Watch
- Appdynamic
- EC2
- EMR
- RedShift
- Athena
- Glue
- Lambda
- IAM Roles
- Lake Formation Services
- Big Data
- Hadoop
- Cloudera
- Yarn
- Hive
- Spark
- H Base
- SQL
- ETL - Abinitio
- IT Infrastructure Management
- IT Operations Management
- ITIL Process Governance
- Incident Management
- Problem Management
- Change Management
- ITSM Tools
- Service Now
- Myservice
- HP Service Manager
- SMIS
- Project Management
- MS Project
- Snowflake
- Databricks
- Software - Hadoop Big-data
- OS Platform - Linux
- Unix
- Windows
- Network Monitoring Tools
- App Monitoring Tools
- SCOM
- Kibana
- Unravel
- Autosys
- Grafana
- MS Office
- Power Point
- Excel
- Visio
Certification
Certified Snowflake - SnowPro Core Certification, S0027560
Tools And Technical Proficiency
- ITSM Tools - Service Now / Myservice / HP Service Manager / SMIS
- Project Management – MS Project
- Cloud – AWS - S3, RedShift, Athena, Glue, Lambda
- ETL – Abinitio/ Control Center
- Snowflake
- Databricks
- SQL
- Software – Hadoop Big-data
- OS Platform – Linux, Unix, Windows
- Network/ App/ Monitoring Tools - SCOM, Kibana, Unravel, Autosys, Grafana
- MS Office – Power Point, Excel, Visio etc.
Timeline
Manager (ETL)
Commonwealth Bank of Australia
02.2021 - Current
Senior Engineer – Platform
FIS Global Business Solution
08.2019 - 02.2021
Associate –Line of Business - GTI (Global technology Infrastructure)
JP Morgan Service India Pvt. Ltd
12.2014 - 08.2019
Associate Engineer
Collabera Technology Pvt. Ltd
09.2014 - 12.2014
Linux Administrator
Accenture service Pvt. Ltd
09.2013 - 08.2014
Data Center Operator – Linux Admin
HP Service India Pvt. Ltd
10.2011 - 03.2013
B.E - Electronics & Communication Engineering
Rajiv Gandhi Proudyogiki Vishwavidyalaya
Similar Profiles
Aanchal AroraAanchal Arora
Insights Analytics Manager, Advanced Analytics at Commonwealth Bank of AustraliaInsights Analytics Manager, Advanced Analytics at Commonwealth Bank of Australia
Anupama ShenoyAnupama Shenoy
Chapter Lead at Commonwealth Bank Of AustraliaChapter Lead at Commonwealth Bank Of Australia
William SinWilliam Sin
Customer Service Specialist at Commonwealth Bank of Australia, CBACustomer Service Specialist at Commonwealth Bank of Australia, CBA
JULIAN NARAYANJULIAN NARAYAN
Customer Relations Specialist at Commonwealth Bank Of Australia, CBACustomer Relations Specialist at Commonwealth Bank Of Australia, CBA
Sijith VJSijith VJ
Principal Software Engineer at OraclePrincipal Software Engineer at Oracle