Prakash Balasubramani

Enterprise Data Platform: We developed EDP (Enterprise Data Platform) to create a Common Data Lake for all microservices. I have been involved in this product from inception through deployment, maintenance, and upgrades.

Key phases include:

· Data Acquisition: Leveraged Apache NiFi, an open-source data flow web tool, to develop a common template for extracting and loading data from various sources (CSV, Text, JSON, RDBMS, Kafka). Supports streaming, bulk, and incremental loads.

· Data Transformation: Utilized Apache Spark, a distributed in-memory computing engine, to build a common Spark codebase for reading datasets from diverse sources (RDBMS, NoSQL, HDFS, Kafka, Event Bus, flat files). Implemented transformations such as Select, Filter, Join, Group, Aggregate, Sort, Data type conversion, Timestamp format change, drop, distinct, etc. Also enabled users to write SQL query transformations for both batch and stream jobs.

· Data Presentation: Employed PowerBI to present data from our MongoDB Data Lake. Users configure metrics or dashboards as needed.

· EDP Engine: Built using Play Framework, responsible for validating, submitting, status tracking, and scheduling EDP jobs.

• Responsible for creating data services job across different layers, in order to load the
data.
• Took the responsibility of managing the team in the critical situation.
• Responsible for the build release across different boxes like testing, Pre_prod, Prod.
• Responsible for Internal service audit.
• Prepared unit test cases and support hand over document.
• Involved in production support.
• Conducted many KT sessions to expose my knowledge to my peers.
• Involved in creating automated tool in order to simplify our manual efforts.