ABHAS UPADHAYAY

Results-driven Software Engineer with over 4 years experience in building large scale distributed computing systems. Recognized consistently for performance excellency and understanding complex business requirements while helping companies to collect, collate and exploit digital assets.

Big Data with Hadoop and Spark, CloudxLab

PROJECT-1: Concept Generator

• Built a common framework using Pyspark and Airflow which helped Data scientists to index their data to Apache Solr/Elastic Search easily and independenlty.
• Built client facing APIs for our product using fastapi to query and combine data from MySQL, Elastic Search, Solr and power the concept generator feature which was demoed in Future Food-Tech event,London and received great appreciation. Used best practices for sharding, replication to scale the APIs to handle traffic.

PROJECT -2: Finance Fixed Assets in Hadoop3

• Built framework using Akka library in Spark(Scala) to ingest data from REST-API.
• Built data pipeline to store and process 500million+ rows of semi-structured,structured and unstructured data using Spark, Hive, Apache Oozie, GitHub, Unix Shell Scripting, Kafka, Grafana, PostgreSQL to support 5 finance reports.

PROJECT-3: Inventory Sub Locations core history in Hadoop

• Build data pipeline to store and process 200million+ rows of JSON data using Spark structured streaming to give latest state of store's stock inventory after every 6 hours.

PROJECT-4 ( Self-project): BankCustomer Churn Prediction

• Performed univariate, bivariate analysis on the input dataset and cleansed it using Pandas. Trained the Random Forest model on cleansed data and predicted churn propensity with 90% accuracy.