Summary
Overview
Work History
Education
Skills
Timeline
Generic
Shreshth Shrivastava

Shreshth Shrivastava

Noida

Summary

  • 2 years of overall experience with Wipro and Brandwidth Technologies pvt ltd.
  • With 2 years of Relevant experience in Big Data, Data analyst technologies.
  • In-depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, MapReduce and YARN concepts. Strong knowledge of Hive's analytical functions(with Structure and query-level optimization).
  • Experience in Python programming , Sql, Ms-Excel
  • Strong knowledge of Spark structured API. Experience in importing and exporting terabytes of data between HDFS and Relational Database System using Sqoop.
  • Strong knowledge of Software Development Life Cycle and expertise in detailed design documentation.
  • Strong knowledge of HBase and its integration with Hive and Spark.
  • Experience in working with the Business Intelligence team for requirement analysis and delivering data as per requirement. Experience in creating and modifying ETL packages as per the business need following the practices.

Overview

2
2
years of professional experience

Work History

Data Engineer

Brandwidth Technologies pvt ltd.
07.2022 - Current
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Designed compliance frameworks for multi-site data warehousing efforts to verify conformity with state and federal data security guidelines.
  • Generated detailed studies on potential third-party data handling solutions, verifying compliance with internal needs and stakeholder requirements.
  • Built databases and table structures for web applications.
  • Analyzed complex data and identified anomalies, trends, and risks to provide useful insights to improve internal controls.
  • Analyze customer purchase data to better understand customer behavior and preferences. Processed millions of transactions in batches.
  • Created Hive tables [External/Internal & Partitioned Bucketed]. Created shell scripts for Sqoop jobs. Monitoring Sqoop jobs which are scheduled on daily basis. Verifying incremental loaded data.
  • Involvement in writing Hive queries with join optimization [Map-Side and SMB].
  • Performed cleaning and filtering on imported data using Apache Spark[Structured API] & store the data in NoSQL DB(HBase).

Associate Analyst

Wipro
07.2021 - 06.2022
  • Responsibilities: To do Data Aggregation and Data analytics on top of the raw dataset which contains millions of records about the purchase products from multiple sources.
  • We wrote several Hive scripts to do ETL and data analysis. Worked on a live 30 nodes Hadoop cluster running on Apache Hadoop 2.4.1.
  • Extracted the data from MySQL into HDFS using SQOOP. Created Hive [Partitioned + Bucketed] table. Data aggregation using the windowing function in Hive.
  • Experience in using Sequence files, ORC & Parquet file formats. Helps in configuring NiFi to check for new data from time to time. Tuning existing Spark codes from time to time based on the business requirement Completely involved in the requirement analysis and design phase. Solved performance issues in Spark and Hive scripts with an understanding of Joins, Groups, and Aggregation.
  • Worked with business intelligence software and various reports to glean insights into trends and prospects.
  • Developed complex dashboard and reporting tools to track business performance metrics.
  • Identified patterns and trends in large data sets and provided actionable insights.
  • Created various Excel documents to assist with pulling metrics data and presenting information to stakeholders for concise explanations of best placement for needed resources.
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Designed compliance frameworks for multi-site data warehousing efforts to verify conformity with state and federal data security guidelines.
  • Built databases and table structures for web applications.
  • Analyzed complex data and identified anomalies, trends, and risks to provide useful insights to improve internal controls.

Education

B.Tech - Electronics And Communication Engineering

AKTU
06.2018

ISC BOARD - SCIENCE (PCM)

St.Francis School
05.2014

ICSE -

St.Francis School
05.2012

Skills

    Sql

    python

    MS-Excel

    ETL

    pySpark

    Sql Server

    Oracle Sql

    Apache Spark

    Hive

    HiveQL

    Apache Sqoop

    Scala

    NoSQl

    Azure

    Hadoop

    Azure Data Factory

    Azure Databricks

    Presentation and communication skill

Timeline

Data Engineer

Brandwidth Technologies pvt ltd.
07.2022 - Current

Associate Analyst

Wipro
07.2021 - 06.2022

B.Tech - Electronics And Communication Engineering

AKTU

ISC BOARD - SCIENCE (PCM)

St.Francis School

ICSE -

St.Francis School

Similar Profiles

CREATE PROFILE
Shreshth Shrivastava