Summary

Overview

Work History

Education

Skills

Timeline

HARMANJEET SINGH

Data Engineer

New Delhi,Delhi

Summary

Working as a Senior Data Engineer at Rakuten with a total experience of 5 years. Job involves develop, construct, test and maintain architecture for various big data & data science projects.

Overview

5

5

years of professional experience

Work History

Senior Data Engineer

Rakuten

Bengaluru, Karnataka

12.2019 - Current

Built spark streaming pipeline to ingest kafka data from Viki (OTT platform) and provide real time analytics.
Built Sleeper Cell Detection by analyzing time series data of cell towers to ensure zero down time. Performed spark & hive optimization to bring job completion time to 15 mins from an hour. And helped improving model accuracy from 85% to 95%.
Developed MNO Data Insight pipeline to enrich data for customer segmentation and targeted ad campaigning. Data involve transactional data from Rakuten e-commerce platform.
Built dashboard for data utilization of hadoop resources by analyzing ranger logs for auditing.
Setup a data quality control pipeline to ensure data correctness and freshness for various business units. Pipeline is built to handle TBs of data and run every hour.

SENIOR ENGINEER | DATA ENGINEER

CSS CORP PRIVATE LIMITED

Bengaluru, Karnataka

10.2018 - 12.2019

Built Smart Case Manager for call centre clients to provide real time data insights and help them better serve their clients.
Discovering data acquisition opportunities and building web crawling processes to scrap data from social media sites and various forums.
Worked on chatbot POC for Hindustan Unilever in Google DialogFlow, REST services and NodeJS.
Automation & scheduling jobs to support data science and analytics projects.
Built tableau dashboard for data visualization and analytics.

DATA ENGINEER

TIMES INTERNET LIMITED

New Delhi, Delhi

02.2017 - 10.2018

Built Recommendation Engine for The Economics Times to provide personalized content to its user. Ensemble learning technique were used for better predictions.
Hands on experience in Hadoop cluster management and configuration.
Created Data Lake by extracting user data from various data sources into HDFS. This include data from RDBMS and MongoDB.
Involved in design and development of Data Transformation Framework components to support ETL process.
Developed an ingestion module to ingest data into HDFS from Kafka Stream.
Built distributed in-memory applications using Apache Spark and Spark ML to do processing on huge data sets.
These applications were built using Spark Scala & Java API and used YARN as resource manager.
Efficiently used spark transformations and action to build simple/quick and complex ETL applications.
Used Oozie workflow scheduler system and Linux crontab job scheduler to manage Apache Hadoop jobs.
Involved in performance optimization of Spark jobs and designed efficient queries to query large data sets.

SOFTWARE ENGINEER

TIMES INTERNET LIMITED

New Delhi, Delhi

06.2016 - 02.2017

Involved in profiling user data coming to The Economics Times site.
Co Built system to retrieve user activity on website and track engagement.
Developed web services for various contest and surveys using restful services. Spring Web MVC, Spring Boot, Spring Data frameworks were used in building these applications.

Education

B.Tech - Computer Science Engineering

G. B. Pant Govt. Engineering College

08.2012 - 06.2016

GPA: 82.98

Senior Secondary Education

Mata Gujri Public School

04.2011 - 03.2012

GPA: 93.25

Secondary Education

Mata Gujri Public School

04.2009 - 03.2010

GPA: 90

Skills

Spark

undefined

Timeline

Senior Data Engineer - Rakuten

12.2019 - Current

SENIOR ENGINEER | DATA ENGINEER - CSS CORP PRIVATE LIMITED

10.2018 - 12.2019

DATA ENGINEER - TIMES INTERNET LIMITED

02.2017 - 10.2018

SOFTWARE ENGINEER - TIMES INTERNET LIMITED

06.2016 - 02.2017

G. B. Pant Govt. Engineering College - B.Tech, Computer Science Engineering

08.2012 - 06.2016

Mata Gujri Public School - , Senior Secondary Education

04.2011 - 03.2012

Mata Gujri Public School - , Secondary Education

04.2009 - 03.2010