PRADEEP MISHRA

Dynamic Big Data Platform Engineer and team lead at TESCO HSC, adept at designing scalable solutions and enhancing system performance. Proven expertise in Hadoop administration and effective team leadership, driving successful cross-functional collaborations. Recognized for implementing innovative data strategies that significantly improved operational efficiency.

Advance certification program for cloud computing and DevOps by Intellipat

Cleared MCSA: 70-462 certification

• Big data cluster planning and implementation
• DR setup for big data cluster
• Backup setup for big data cluster.
• Integrated Wandisco Fusion and LDM with HDP Cluster and Object store
• Dremio Setup on K8 with object store
• K8 Support for multiple application with end-to-end integration
• Completed the Cluster Migration activity from one DC to another
• Done successful poc to migrate data using WANdisco, Ranger policies, Hive DB and Kafka Topics from one cluster to another
• Kafka mirroring POC
• POC for on-prem to cloud s3 filesystem compatibility, started with AWS and done final migration to CEPH.
• Handled Multiple Project for data Ingestion on HDP cluster with success.
• Participated in automation project for Build Hadoop, Spark, and Kafka clusters from Scratch both manually as well as Automation, along with security features like Kerberos & SSL.
• Worked on Horizontal scaling up and scaling down of clusters as and when the situations demanded and applied Automation wherever necessary and possible.
• Formed Hadoop cluster using Apache rpms and using HDP Ambari
• Set up SNMP Traps to propagate Ambari alerts to one of the monitoring and alerting tools called Splunk and send out alerts via X-matters to the on-call device.
• Have set up mail alerts for customized scripts monitoring the jobs for their running hours, resource utilization and cluster utilization.
• Set up Prometheus server tool to monitor some of the services by capturing the JMX metrics exposed by the Prometheus agent.
• Worked on Small file reductions or Zero-byte files reduction in the cluster and worked with application teams to constantly reduce empty partitions.
• Ansible automation for upgrades and builds

• Manage Team Members run sprint planning and standups to manage day to day tasks
• Managing Spark/Dremio/Kyuubi application on K8 deployment and production run and life cycle management
• Understand behavior of application tool with orchestration tool
• Managing Hadoop HDP cluster and jobs
• Capacity requirement for K8 cluster
• Planning of the Hadoop cluster requirement based on the available data and expected growth rate.
• Setting up DR (Data/metadata jobs etc for entire big data estate)
• Setup and managed HDP/Confluent Kafka for consuming real time events from producer and put them in HDFS/Object Store/S3
• Participated in development activities and provided solutions and assistance to developers and writing Oozie workflows and tuning spark jobs
• Day to day management of Airflow jobs
• Managing Ranger, Knox, IAM Roles, Kerberos to manage security of prod and test clusters
• Participating in code deployments and coordinating with developers
• Troubleshooting issues with jobs and coordinating dev team to fix them
• Participating on call with Onshore to update the status of jobs and cluster
• Life cycle management and Vulnerabilities management for different component of big data stack.
• Automate regular tasks using Ansible scripts to automate tasks to manage cluster etc.
• Involved in using the benchmarking tools such as Teragen, Terasort to determine the performance evaluation.
• Set up encryption zone to stored hashed and PII data and other information that is critical and adhered according to the GDPR compliance rules.

Date of Birth : 5th FEB 1987 Gender : Male Marital Status : Married I hereby declare that all the details given above are true to the best of my knowledge and belief. Place: Bangalore (Pradeep Kumar Mishra)