Summary
Overview
Work History
Education
Skills
Current Title
Timeline
Generic

Richard Kashyap Deka

Bengaluru

Summary

Seasoned Big Data Engineer with 10+ years of experience in designing, building, and optimizing scalable data solutions across diverse domains including finance, retail, and healthcare. Proven expertise in modern big data technologies such as Spark, Hive, Kafka, and HBase, with strong hands-on experience on cloud platforms like Google Cloud Platform (GCP) and Microsoft Azure. Adept at real-time and batch data processing, customer data platform implementation, and end-to-end data pipeline development. Skilled in Java, Scala, PySpark, and SQL, with a strong focus on performance, reliability, and business value. Known for delivering complex data projects on time and driving impactful insights through robust data engineering solutions.

Overview

10
10
years of professional experience

Work History

Senior Data Engineer L2

Publicis Sapient
Bengaluru
07.2023 - Current
  • Client: Lloyds Bank, Project: New Payment Architecture - Executed architecture implementation using BigQuery, DBT, Kafka, Kubernetes, Terraform, and Helm charts. Migrated existing payments process to GCP platform for enhanced performance and feasibility. Processed both real-time data via Kafka and batch data with scheduled curation. Curated data utilized by reporting layer to populate tables for reconciliation.

Data Engineer III

Walmart India Pvt Lmtd
Bengaluru
01.2022 - 07.2023
  • Client: In-house Walmart - Processed and maintained Walmart's online data, integrating third-party websites for comprehensive customer insights. Consolidated data to create 360-degree views of customers, enhancing decision-making capabilities. Utilized Scala, Spark, and Kafka to streamline data processing workflows. Employed BigQuery and Dataproc to efficiently manage large datasets and improve performance.

Senior Data Engineer L1

Publicis Sapient
Bengaluru
09.2019 - 12.2021
  • Client: Falabella, Project: CDP - Customer Data Platform - Implemented data pipelines utilizing GCP, BigQuery, Bigtable, and Apache Beam. Executed ingestion, transformation, and aggregation of diverse business unit data. Developed customer identification process using deterministic matching logic. Created APIs to provide end users with comprehensive views of aggregated data.
  • Client: KPMG, Project: CDP - Customer Data Platform - Executed implementation of Scala, Spark, Microsoft Azure, Databricks, DeltaLake, ADF, and Logic Apps. Processed and transformed Adobe Analytics, Customer Profile PPC, and Google Search Data for various member firms. Developed customer identification process utilizing deterministic matching logic for new and existing customers. Utilized transformed data in HVA dashboard reporting for enhanced insights.

Data Engineer

ClusterFoundry Technologies
Bengaluru
12.2018 - 09.2019
  • Client: LBrands, Project: DOMS-Domestic Order Management System - Developed pipelines for order processing and report generation within DOMS. Created JSONs for Kafka topics to facilitate data streaming. Designed Java microservices to consume Kafka topics, transforming and storing data in MAPRDB/HBase. Engineered Spark jobs to read from HBase tables, executing transformations and aggregations. Wrote processed data into Hive tables to support analytical reporting. Leveraged technologies including Java, Hive, Kafka, Spark, Spring Boot, and HBase.

Software Engineer

Zaloni Tech India Pvt. Ltd
Guwahati
07.2015 - 10.2018
  • Client: Bank of America; Project: De-Dup - Executed REST API calls from Zaloni-based product to generate profile metrics. Aggregated REST API results for accurate computation of deduplicated records percentage. Implemented Java, Hive, and Spark technologies to enhance data processing capabilities.
  • Client: UHG, Project: Payment Integrity - Executed implementation of Hive, Sqoop, Shell scripting, and SQLite for data processing. Contributed significantly to planning and processing multiple data sources. Optimized and modified various modules to enhance system performance. Conducted regression testing to ensure system reliability and functionality.
  • Client: Cisco; Project: Cisco-ETL - Developed automation scripts for data discovery and job scheduling using Hive, Java, and Shell Scripting. Created ingestion programs for Cisco data to enhance processing efficiency. Tuned Hive queries to improve performance metrics and reduce execution time.Conducted performance testing to ensure optimal functionality of data processing systems.

Education

BE - Computer Science & Engineering

Jorhat Engineering College
Jorhat, Assam
07.2015

Skills

  • Technologies - Hadoop, Mapreduce, Hive, Sqoop, Spark, Hbase, Microsoft Azure, Databricks, Azure Data Factory, GCP Bigquery, DBT
  • Programming Language - Java, Scala, Hive-QL, Shell Scripting, Pyspark

Current Title

Senior Associate Data Engineering L2

Timeline

Senior Data Engineer L2

Publicis Sapient
07.2023 - Current

Data Engineer III

Walmart India Pvt Lmtd
01.2022 - 07.2023

Senior Data Engineer L1

Publicis Sapient
09.2019 - 12.2021

Data Engineer

ClusterFoundry Technologies
12.2018 - 09.2019

Software Engineer

Zaloni Tech India Pvt. Ltd
07.2015 - 10.2018

BE - Computer Science & Engineering

Jorhat Engineering College

Similar Profiles

CREATE PROFILE
Richard Kashyap Deka