Summary
Overview
Work History
Education
Accomplishments
Timeline
Tools & Technology
Tools & Technology
CustomerServiceRepresentative

PRIYANK SINGH

Big Data Engineer
Kolkata,WB

Summary

● More than 6.6 years of IT experience in Big data
● Approx. 3 years of experience as a Big Data Developer
● Experience in Data Warehousing within the environment of Spark, Scala, Hive, Sqoop, HDFS
● Basic Knowledge of Databricks
● Hands-on experience in tools like Putty, Control-M, Hue, Winscp, and GIT.
● Experience in importing and exporting data from a relational database into Hadoop cluster using sqoop.
● Experience writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
● Extensively worked on data extraction, Transformation, and loading data from Flat Files and MYSQL using sqoop
● Involved in preparing test data for testing flows to validate and prove positive and negative cases
● Experienced in working with large datasets using Partitions, Spark in Memory capabilities, effective & efficient Joins, and optimizations during the ingestion process.
● Experience in resolving Defects, presenting the Defect Status reports, resolving requirements, observing design inconsistencies, and Preparing documentation for some of the recurring defects and resolutions and business comments for those defects.
● Coordinated with all the teams, representatives, and downstream users involved in testing for sign-off approvals.
● Implemented Test logic by writing SQL/HQL queries.
● Experience in Data Analysis, Data Validation, Data Cleansing, Data Verification, and identifying data mismatch.

Overview

7
7
years of professional experience
4
4
years of post-secondary education

Work History

Big Data Developer

Cognizant Technology Solutions
Pune
12.2020 - Current

CLIENT : NOVARTIS

PROJECT : ATLAS ENHANCEMENT

RESPONSIBILITIES :

Understand new enhancement\requirement requested by the client and create Spark/Scala-based code with Hive as data storage on the HDFS platform.Atlas4A program is set to interface via evico platform to provide the enriched HCP and HCO information to different consumers of Novartis. Utilize the BRR framework(Metadata Driven Tool) for the implementation of different inbound and outbound interfaces.
· Key Role in Analyzing requirements of the alignment data interface.
· Understanding the overall functionality of the project and interaction with different downstream.
· Implementing the incremental logic in the ODS layer.
· Scheduling jobs in Control M.
· Actively involved in code deployment in a higher environment.
· Providing hyper care support post-production.
· Imported the data in CSV file format into hive table and data validation
· Created multiple tables and views in the hive, updated parquet files, DDL files, and Scala code, and pushed it into Git.
· Implementation of Adhoc new requirements following SDLC.

Big Data Engineer

Cognizant Technology Solutions
Pune
08.2019 - 12.2020

CLIENT : NOVARTIS

PROJECT : VDW REBUILD

RESPONSIBILITIES :

The objective was to migrate the existing VDW (Vantage Data Warehouse) system to a new big data platform EVICO. ODS will house data captured in the Customer Relationship Management application (CRM) namely ‘Vantage’. End users/Sales Representatives log their call/sampling and all other activities in ‘Vantage’ and ODS (though DICE) Extract that data and store it in a Database (EVICO Platform) in Novartis Network. This data is then provisioned to various other systems within Novartis for easy access.

  • Develop and implement code that loads data into an information product that helps to inform the organization in reaching strategic goals.
  • Involved in creating Hive tables, and then applied HiveQL on those tables for data validation.
  • Processing the data by applying the business rules using SparkSQL.
  • Work on ingesting, storing, processing, and analyzing large data sets.
  • Translate complex technical and functional requirements into simple optimized code.
  • Investigate and analyze alternative solutions to data storing, processing, etc. to ensure the most streamlined approaches are implemented if any Data Quality issues occur.
  • Running the spark jobs to load the data automatically using wrapper scripts.
  • Running the jobs in Control-M for loading the data to different business layers.
  • Analyzing and fixing the failure scripts.
  • Used Git for version control.
  • Helped the QA team in testing a few code changes in the QA environment.

Big Data Tester

Cognizant Technology Solutions
Pune
06.2018 - 07.2019

CLIENT : NOVARTIS

PROJECT : COGNOS MIGRATION

RESPONSIBILITIES :

The objective was to compare 150+ tables containing various drugs information in MYSQL. Data were loaded in HDFS system from RDBMS, which were pushed from DEV to QA and finally to Production. We had to validate Structure,Delimiter,Data of every file.

· Prepared test cases by understanding the business requirements, Data Mapping documents

· Tested Sqoop scripts used to transfer data between RDBMS and HDFS.

· Tested Hive QL queries with Partitioning, Dynamic Partitions and Buckets.

· Compared metadata,data,structure of each file in each environment.

· Raised Bugs in JIRA\Proton as per Defect Lifecycle Concept

· Executed various Test Cases in HP Proton and captured necessary test evidences.

Big Data Tester

Cognizant Technology Solutions
Pune
12.2015 - 05.2018

CLIENT : Union Bank of Switzerland

PROJECT : UBS DataLake

RESPONSIBILITIES :

The objective was to validate data and functional logic in tables containing customer related data.

· Go through the report requirements and make sure that all clarifications are done with the respective BAs.

· Complete Test Scenario/Case writing activities during Development stage within the assigned timeline.

· Actively follow up with Leads/BAs/Developers with respect to test case review and sign off, defect closure, etc.

· Ensure complete coverage of functional and non-function requirements of reports in test cases and they are mapped to the requirements through RTM for the assigned modules

· Provide regular status updates to the Test Lead in the agreed format.

· Reporting issues in JIRA, tracking of issues and interacting with the Developers for bug fixes.

· Re-test resolved defects and bring the defects to a logical closure within the agreed timelines.

· Report work progress and any problems faced to the Test Lead.

· Provide KT to the new joiners in the testing team as well as guide them technically.

· Execute the Test Cases in HP proton and capture test evidences.

Education

Bachelor of Technology - Electrical, Electronics And Communications Engineering

Symbiosis Institute of Technology
Pune
06.2011 - 06.2015

Accomplishments

  • Got Appreciation from the client for doing multiple optimizations in scala & SQL codes that run in spark. It has decreased the overall execution time by 3.5 hours.
  • Got Top Rating for constantly performing well in developing complex business logic, enhancement changes, and delivering it on time.
  • Got Top Rating for testing around 200+ code changes in the QA environment and successfully deploying it to production without any defect. It saved overall project cost and was completed before the estimated time.

Timeline

Big Data Developer

Cognizant Technology Solutions
12.2020 - Current

Big Data Engineer

Cognizant Technology Solutions
08.2019 - 12.2020

Big Data Tester

Cognizant Technology Solutions
06.2018 - 07.2019

Big Data Tester

Cognizant Technology Solutions
12.2015 - 05.2018

Bachelor of Technology - Electrical, Electronics And Communications Engineering

Symbiosis Institute of Technology
06.2011 - 06.2015

Tools & Technology

  • Big Data : Spark, Scala, Hive HQL, Sqoop, Impala, HDFS,Databricks
  • IDE : Intellij
  • Programming Lang. : SQL,Spark-SQL,HQL,Scala
  • Databases : MySQL
  • Tools : Control-M,Jira,PL/SQL Developer,HP Proton,Service Now,Sharepoint
  • OS : UNIX

Tools & Technology

  • Big Data : Spark, Scala, Hive HQL, Sqoop, Impala, HDFS,Databricks
  • IDE : Intellij
  • Programming Lang. : SQL,Spark-SQL,HQL,Scala
  • Databases : MySQL
  • Tools : Control-M,Jira,PL/SQL Developer,HP Proton,Service Now,Sharepoint
  • OS : UNIX
PRIYANK SINGHBig Data Engineer