Harish Pranami

Data Sniff : - I have written self-service generic platform to data validation between two dataset across Data sources like, Snowflake, Teradata, Kafka, Hive.

➢ It is written on of Drop wizard, Spark, Java and deployed in Kubernetes.

➢ Various generics data validators written i.e. For count, Sample , datatype. ➢ It generatereports in Excel form so easier to work with reports.

ACD/Keystone Pipelines:- I have written applications Keystone Data Platform, where data comes from GBI/ACI Kafka and Spark get persist into multiple Teradata & Snowflake simultaneously

➢ Create and Deploy pipeline for ACD team

➢ I have done troubleshooting of pipeline when it fails because issues like format, environment,Retry, Scheduling

➢ Performance analysis and tuning of pipelines do benchmarking based on requirement

 Evija TM - Build data lake software to cater data services where data get replicated and processed to/fro AWS S3/Kafka/Teradata and Spark

➢ Design & Development of Framework where user can create pipeline to transfer data into S3 Storage

➢ Run periodic query and send notification when it complete

➢ Enable encryption of data on the fly and data at Rest

➢ Do Performance analysis based on format of source and find out optimal performance

Risk Control System project is develop detect fraud done by customer or consultant on Realtime and

report to concern authority. Get Realtime Tax pro feed and analyse exchange info raise ticket if any

violation found Used Technologies: Kafka, spark, Salesforce

➢ Development of Spring-boot micro services that REST asynchronous request and process into spark

➢ It built on cutting edge technologies like: Alluxio, ActiveMQ, Spark, Spring-boot, Cassandra ➢ Horizontally scalable & distributed using Spring eureka, Config, Zuul, admin

Snapdeal Replication Engine TM - Framework is does replication of data from different data sources. It Design in such a way it moves data from source to sink in incremental load fashion and Framework is plug-able enough to accept new source and sink Vertica, MySQL, mongo, HBase, HDFS etc. Responsibilities –

➢ Fully responsible for End to End design and development of framework.
➢ It is in production and doing replication about 30 databases and 1600 table with huge data from

Vertica/ MySQL to HDFS in 3 time in day. Destination data size about 3 TB.

A Big Data product built using Hadoop technology like MapReduce 1, Hadoop, HBase, Hive, Oozie, Sqoop and Pig, to provide a platform for big data analytics and also includes a set of big data solutions built on top of it.

Responsibilities –

• ➢ Developed “Social Media Master” Framework is core framework in “AllSight” to extract hiddencustomer information in social media data like twitter and Facebook on top of (Oozie, pig, HDFS) ➢Originated “Social Media Toolkit” to collect Facebook feed on top of HBase, H2 database.
• ➢ WorkedondevelopingTwitterandFacebookdatacollectionjobsfor“Social-Media-Master(SMM)” module. SMM is implemented in java and invoked through pig UDF for collecting Facebook, Twitter data. The collected data is then transformed into JSON format. HBase and RDBMS (MySQL/H2) usedfor Data storage.