Rakesh C Patil

PAYROLL PANEL:

Payroll panel identifies payroll transaction from ~5000 employers using Payroll Identification Engine (PIE). Apart from payroll specific rules, PIE also internally invokes recurring algorithm (ARIS) to identify recurring patterns, payroll cadence and few other enrichments like anomalous payroll transactions, etc. PIE module also consumes PRISM model to identify transactions to SMB accounts in the data.

• Designed and Developed a Spark Based Batch Application to implement the Payroll Product.
• Developed a Data Pipeline solution to orchestrate a multiple Spark SQL based jobs by utilizing AWS EMR Technologies.
• Technology Used: Apache Spark, Scala, AWS EMR , Shell Scripting, AWS Redshift.

FALCON:

• Designed and developed a Universal Securities Collection repository, data extracted from all over major Stock Exchanges across the globe and enriched with features like Security Type, Security Style and OSI's.
• Implemented a complete Enriched and Validation Layer to validate the Checksum logic of all Global identifiers to mark as Public or Private Security.
• Designed and developed a mid‐level Data Science solution for mapping investment holdings with its proper Global Identifiers in order to overcome the errors in net value profit at application level using the Enriched Global Identifiers and Price model of Securities at which they are being traded in different Stock Exchanges.
• Designed and developed a Spark and Hadoop Batch pipeline to enrich a billion holdings dataset on FALCON Data science model.
• Technology used: Scala, Python, Redis , MongoDB, Shell Scripting, Hadoop, Hive and Spark.

RECOMMENDATION SYSTEM & INTELLIGENT CREDIT ENRICHMENT

• Designed and developed a real time event driven micro service based application using Confluent Kafka and MongoDB to recommend views on Historical Card/Bank transactions given a set of rules.
• Designed a Drool based Event Driven Microservice application and also Spark Batch Pipeline to granularize the transaction category given a set of credit transactions for a given user.
• Technology Used: Confluent Kafka, MongoDB, Java , Drool , Apache Spark, Scala and Kubernetes.

CLOUD MIGRATION:

• Capturing the Analytical Services dependency with other platform Integration points. Architecting the AWS Based cloud solution for migrating existing in house Batch App and Micro Service based Analytics Service.
• Developed a AWS Step Function based solution for orchestrating Batch Pipeline by utilizing AWS Batch Technologies
• Developed a Helm based Kubernetes solution for deploying Analytics micro service using AWS EKS Technologies.
• Developed a CI/CD Architecture for Continuous Build and deployment into AWS EKS using Gitlab Runner and Docker.
• Developed Terraform templates for provisioning various infrastructure components required for Analytics Services.
• Technology Used: AWS , Terraform, Gitlab, Kubernetes, Helm and Shell Scripting
  

RCIS Data Ingestion and Active‐Copy Framework:
Scope of this project was to rollout Attunity Replicate tool as a new Data Integration Platform, leverage its capability to capture real time data (cdc) and finally to integrate the tool with the existing Enterprise Hadoop Data Lake.

Secondly, the project was also about developing a new framework to handle incremental data changes and to create hive process to make the data in Enterpise Data Lake in‐sync with the Source Data.

• Developed Powershell scripts to integrate Attunity Replicate tool with Enterprise Hadoop Data Lake to ingest data into HDFS and also with Autosys Scheduler.
• Developed Java Standalone Code to detect and generate report/hqls for the incremental data changes by taking Masterschema as the benchmark.
• Designed Hive and Pig Scripts to handle and load incremental Data.
• Designed Hive Process to handle Active Copy Framework operations in Enterprise Hadoop Data Lake.

Predictive Analytics Data Architecture:

The scope of this project was to setup Enterprise Data Lake and Integrated Analytics platform in Hadoop and to provide support for various uses cases including Analytics, processing, storing and Reporting of voluminous, rapidly changing, structured, semi structured and unstructured data.

The Data Platform provided end to end Data Lineage, transformation Lineage, Metadata management throughout the Data Lifecycle such as Ingestion, ETL, Curation and Models.

Multiple data sources like Relational Databases/Tables, Mainframe Off‐Loads, Structured Data Files (ASCII/Binary), Semi Structured Data Files (XML/JSON), and Un‐Structured Data Files (Raster/Shape Files/PDF/Images) were integrated with Enterprise Hadoop Data Lake.

• Involved in Requirements gathering, Analysis, Design, Development, Testing and Deployment phases of the project life cycle.
• Designed and developed Sqoop scripts to import the data from various RDBMS sources to Hadoop Lake in an optimized way by selecting the right number of Mappers, Split‐By column, Applying compression, etc.
• Designed and developed Pig scripts and pig UDFʼs to process and transform the Ingested data by applying level‐1 Curation and transformation rules like Trim of String fields, Formatting theSigned Integers, Convert EBCDIC to ASCII, Format Data/Timestamp fields, Apply Masking on Protected and Confidential data, Flattening of Flat files, Removal of Non‐printable characters.
• Designed data model on Hive to store structured and Semi‐Structured data files.
• Applied Partitioning, Bucketing, MapJoin, Vectorization and CBO (Cost based optimization) techniques in hive to improve the performance of queries involving joins, aggregations, filters, etc.
• Designed and developed Work Flow Management to schedule Big Data jobs using Autosys Scheduler.
• Driving initiative to automate the recurring manual activities for monitoring and operations using Unix Scripting.