Abid Basha Shaik

Motivated professional eager to embrace challenging opportunities and contribute to organizational growth and success. Committed to delivering results through efficient, innovative approaches and fostering a collaborative environment that drives team success. Passionate about continuous personal and professional development, with a strong desire to evolve alongside the company. Ready to leverage skills and experience to make a meaningful impact in a dynamic workplace.

• Got A+ certificate for C programming
• Got raising Start award in Project in 2018-2019
• I was school pupil leader for two consecutive years in School.

Performance & Optimization
• Optimized complex Hive queries, reducing data processing time by 40% through partitioning, bucketing, and query refactoring.
• Improved ETL pipeline efficiency by rewriting heavy joins, cutting query runtime from 25 mins to 8 mins.
• Implemented Hive table partitioning strategy that reduced daily load failures by 90%.
• Built automated ETL pipelines using HiveQL and SQL, processing.
• Designed Hive schemas and external tables to support a new analytics pipeline.

Project 1:

Client: E.ON

Team Size: 30

Roll: Python and Pyspark developer

Project: Analytical platform

Description:

This project provides an all-in-one central repository for converged Data Platform to analyze the customer segmentation. As a python developer at E.ONE.ON I will be responsible for developing, implementing, and maintaining python-based applications that support our project goals. Using the pyspark we are a processing data and cleansing and loading the data into the hive table by using the standard metrics and existing business rules and KPI and storing into hive for reporting and analysis.

I will collaborate with a team of developers, designers and project managers to deliver high quality solutions that meet the project requirements.

Roles and Responsibilities:

• Develop python applications based on project requirements.

• Write clean, efficient, and maintainable code.

• using the pyspark we are a processing data and cleansing and loading the data into the hive table

• Involved in developing Spark Jobs and Generating the report using hive with spark Engine.

• Involved in the design discussion and implement Bigdata salutations and implementation.

• Participate in code review and provide constructive feedback.

• collaborate with team members to design and implement features.

• Troubleshoot and debug applications to ensure optimal performance.

• Problem solving and productions issues of the existing job which is in production.

Project 2:

Client: Centrica

Team Size: 30

Roll: Spark, Scala and Hadoop Developer

Project: Analytical platform

Description:

This project provides an all-in-one central repository for converged Data Platform to analyze the customer segmentation. This is a migration project from the existing systems (SAP) to Hadoop platform and proving single repository for the analysis, reporting, storage and backup. Capturing the data from the different sources using the Sqoop and Kafka connectors load the data into the staging area. using the spark and Hadoop we are a processing data and cleansing and loading the data into the hive table by using the standard metrics and existing business rules and KPI and storing into hive for reporting and analysis.

Roles and Responsibilities:

• Implanted the ETL process using the Spark and hive using internal framework.

• Involved in developing Spark Jobs and Generating the report using hive with spark Engine.

• Performed Data enrichments such as filtering, format modelling, sorting and aggregation using Hive.

• Worked on performance optimization of various ecosystems such as hive, Spark and HBase.

• Implemented and Reporting tools infrastructure on Hadoop and spark.

• Preparing the documents for the user guide and Design documents.

• Involved in the design discussion and implement Bigdata salutations and implementation.

• Involved in the CICD process for deployed into dev and SIT environment.

• Problem solving and productions issues of the existing job which is in production.