Prashant Kumar

Roles and Responsibilities: 1. Data Pipelines Development: 2. Backend Scala Microservices Development:

• Designed and developed multiple Spark-based production data pipelines from scratch using Scala, Spark, Hadoop, MongoDB , deployed on AWS and in-house clusters to efficiently process and optimize geospatial data .

key data pipelines:

• Missing Street Centerline Data Pipeline: Identifies missing or incomplete street-level map data and updates the map accordingly to ensure accuracy and completeness.
• Related Cluster Stats Data Pipeline: Groups related To-Do tickets based on predefined clustering rules, enabling efficient bulk actions and improving resolution speed.
• Edit Automator Stats Pipeline: Provides a holistic analysis of various map-related issues using customizable filters, enabling better insights and improving automation efficiency.
• Missing Trail Data Pipeline: Detects missing or incomplete trail-level map data and updates the map to enhance coverage and reliability.
  
• Delivered multiple enhancements and features in backend microservices, significantly improving the automation rate for resolving RAP (Report a Problem) tickets.

Key Contributions:

• Ticket Clustering: Implemented related clustering of To-Do tickets based on street, address, and routing rules, enabling faster resolution of related issues, which increased automation efficiency and reduced resource overhead.
• Automated Ticket Closure: Developed an auto-closing system using Face ML models to analyze user-submitted photos and comments, introducing a scoring mechanism to filter out irrelevant tickets efficiently.
• Customer Outreach Features: Implemented automated user notification systems to request additional information from users in cases of incomplete or ambiguous reports.
• REST API Development: Built and maintained multiple RESTful APIs using the Play Framework, facilitating seamless UI communication and integration.
• ML Model Integration: Integrated machine learning models and multiple data sources to enable data-driven decision-making for resolving customer issues more effectively.
• Mentorship and Training: Provided mentorship and guidance to junior team members, accelerating their learning curve and enhancing team productivity.
  

Projects: Roles and Responsibilities:

• Developed Spark/Scala-Based ETL Pipelines: Built and managed ETL data pipelines using Spark and Scala, enabling seamless data ingestion, transformation, and storage across cloud platforms such as Google Cloud Platform (GCP) (BigQuery) for researchers, Azure Data Lake (ADL) for modeling teams, and Sybase for trading desk analytics.
• Python-Based Data Pipelines: Designed and implemented Python-based data pipelines to ingest data asynchronously from APIs and databases, process it efficiently, and serve refined data to Climate TA (Technical Analysis) models.
• Data Integration Across Multiple Sources: Integrated data from diverse sources, ensuring consistency, completeness, and accuracy across all downstream systems.
• Generalized QC Tool Development: Created a Quality Control (QC) Tool governed by a QC Data Dictionary to validate data through sanity checks, comparisons, and consistency rules.
  Automated report generation to flag anomalies before feeding data into Climate TA models.
• CI/CD Pipeline Implementation: Built CI/CD pipelines in Azure, enabling automated code packaging and deployment to production environments for streamlined releases.
• Monthly Data Pipeline Maintenance: Managed and maintained multiple monthly data pipelines running on distributed clusters, ensuring smooth execution and resolving production issues and bugs promptly.
• Job Scheduling with Nite Jobs: Set up and maintained Nite Jobs for scheduling and orchestrating Spark data pipeline workflows, ensuring timely and reliable execution.
  

Work Experience Projects Polaris: Aprgloss:

• Scala Developer: Worked as Scala developer and experience of working with 15+ distributed microservices based on event driven architecture(Akka) along with the team.
• Hands-on experience with high-throughput Pub/Sub messaging systems like Kafka and AMPS for seamless communication across microservices.
• Proactively automated manual tasks using Shell scripting, leading to increased efficiency and reduced operational overhead.
• Demonstrated expertise in end-to-end SDLC processes, including requirement gathering, development, testing, deployment, and production support.
• Skilled in SQL Server and MySQL database technologies for efficient data handling and storage solutions.
• Experience with CI/CD pipelines using tools such as Jenkins, Gradle, SBT, and Bitbucket to streamline development workflows.
• Proficient in writing unit tests, integration tests, performance tests, and BDD tests using frameworks like Cucumber.
• Adept at debugging production issues and implementing quick bug fixes to ensure uninterrupted system performance.
• Comfortable working in Linux environments, with extensive experience using job scheduling tools (Autosys) and monitoring tools (ITRS).
• Created and managed release documentation, secured necessary approvals, and executed complex production releases seamlessly
• Trained juniors about our application stack.
  Well adjusted in working in Agile environment with Global Team.
  
• Developed an optimized algorithm for unified bookmark calculation for messages across three different AMPS topics, including extensive performance testing.
• Created a Disaster Recovery Automation Tool (DRAT) in Scala and Shell Script to process exchange-provided files during connection failures.
• Developed Trade booking in downstream application using RIC instrument in Scala.
• Built a standalone Scala application to fetch data from APIs in JSON format, convert it to CSV format, and trigger Shell scripts for bulk order execution.
• Contributed to the development of JMS-AMPS Bridge Applications for communication between SGX Exchange and clearing applications using Scala.
• Created Deploy Application for deploying Scala application using Ansible.
• Wrote unit tests, integration tests, and Cucumber feature files to ensure robust and reliable software functionality.
• Developed and optimized SQL stored procedures, improving database efficiency and query performance.
• Delivered enhancements to the Trade Exception Management System, including automated email job creation and improvements to the .NET-based UI.

Wrote automated test cases to validate various business scenarios, including those related to the creation of new business entities in London post-Brexit.