RAJESH DASH

Meteor Pipeline

• Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
• Analyzed complex data and identified anomalies, trends and risks to provide useful insights to improve internal controls.
• Capturing of data from sim capture database and doing various transformation over the data.
• Extensively used DataFrame operations.
• Implemented various rules in Spark to meet business requirements.
• Implemented raw, bronze and silver layer for the pipeline.
• Responsible for writing scala unit test for testing custom function.
• Responsible for writing open source library for anonymization of data.
• Started building some part this project from scratch which includes POC as well.
• Involved in writing data-quality test cases and show it on sql analytics dashboard
• Technologies used: Apache Spark(Batch), Scala, Databricks , Postgres,DMS,Git, ,SQS,SNS,Lambda,S3

Adaptive Learning

• Developed API to collect user behavior event data to capture data from 7 products .
• Involved in writing pipeline for cleansing the data and flattening the complex columns to create easy query able data for datascientist
• Involved in writing data quality tests for different test cases.
• Developed, implemented and maintained data analytics protocols, standards and documentation.
• Involved in storing profiles (Derived from event-store data) in mongo-db database for querying.
• Implemented simple lambda based api with api gateway for querying data from mongo-db.
• Technologies used: Fast API , Lambda,S3,FireHose,AWS Glue,Apache Spark (Batch),Scala, Mongo-db ,Git

CISO

• Built ETL pipeline for scaling up data processing flow to meet the rapid data growth by exploring different technologies like NIFI,Elastic Search.
• Pulling the data from islon share location and deriving the end to end flow of modifying the customer data, processing the data and store unique results in Hive.
• Worked on ingestion using Apache NIFI and scheduling the spark jobs through it.
• Responsible for writing spark job which check different file hashes and reporting if it comes under malicious using spark and crowdstrike .
• Responsible for creating NIFI flow which runs query from elastic search and generate report by storing unique records in hive whenever there is any attack .
• Technologies used: Scala,Spark,Elastic Search Hive, Shell, Nifi ,Bitbucket, Crowdstrike,Siem

N-Viva

• Built ETL pipeline for scaling up data processing flow to meet the rapid data growth by exploring Spark and improving the performance of the existing algorithm in Hadoop using Spark-Context, Spark- SQL, Data Frame, Pair RDD’s and Spark YARN
• Pulling the data from mediation and deriving the end to end flow of modifying the customer data, processing the data and rewarding eligible customer.
• Worked on rewarding process to store the reward details through the reward loader in cassandra
• Worked on logging the status of each of the job in to dynamo db using aws sdk.
• Defined parameters for API and data acquisitions.
• Technologies used: Apache Spark, S3,AWS ,Scala ,Casandra, Dynamo-Db, GitHub, Jenkins, Scala test

HCL Sametime

• Technologies used: Apache Hadoop, HDFS, Java, Mongo DB, React-js, GitHub

EDA (Electronic Design Automation)

• Worked with software development and testing team members to design and develop robust solutions to meet client requirements for functionality, scalability and performance.
• Tested methodology with writing and execution of test plans, debugging and testing scripts and tools.
• Technologies used: Apache Hadoop,HDFS ,Java , Hive ,GitHub, Jenkins, Ant,Maven

CDK (Configuration Development Kit)

• Technologies used:Java1.8 , Equinox,OSGI,Javafx,Matlab,jaxb,Sonar Qube,Black Duck.NSIS,Jenkins,EMF,Xtext

IREC (Intelligent Radio Configuration Tool)

• Technologies used: Java1.7 , Equinox,OSGI,Jface,C,EMF ,Sonar Qube,Black Duck.NSIS,Jenkins