Nirmal Jha

• Led the migration of on-premise solutions to EMR on EKS for a leading bank's lending platform. Implemented real-time data ingestion with Spark Streaming and managed Kafka for high-throughput data streaming. Resulted in data availability within 10 minutes as per required business SLA.
• Designed and implemented end-to-end batch processing pipelines for complex business transformations, handling large volumes of data efficiently.
• Established robust data quality frameworks and governance practices to ensure data accuracy, consistency, and regulatory compliance.
• Hands on experience with Apache Superset for building dashboards, charts, and analytical queries, enabling end users to generate insights and perform self-service analytics.
• Applied Test-Driven Development (TDD) and Extreme Programming (XP) methodologies to deliver high-quality, maintainable code.
• Integrated Open Telemetry framework for observability, enabling comprehensive monitoring, tracing, and metrics collection to enhance system reliability and performance.

• Led the creation of an advanced car platform by integrating IoT sensor data and telecom data. This platform enabled enhanced real-time analytics and connectivity features for automotive applications.
• Engineered a real-time data streaming solution to handle high-volume data from IoT sensors and telecom sources. Utilized Spark, Hadoop, and HBase for efficient data ingestion, processing, and storage.
• Developed a dynamic rule engine using Spark to analyze streaming data and generate real-time alerts based on predefined conditions, enhancing the platform's responsiveness to critical events.

• Worked on a data lake solution for a major utility company serving more than 2 millions customers across six states. The project involved capturing data from diverse sources such as traditional RDBMS (Oracle), data warehouses (Informatica), and various file formats (CSV, DAT, XML) to handle GBs of data inflow daily.
• Developed Spark programs in Python for creating data frames, performing data transformations, applying User Defined Functions (UDFs), integrating with Hive tables, and executing Spark SQL queries.
• Created and managed Hive tables, optimized data loading from various sources, and integrated these tables with Spark for efficient analytics. Focused on Hive optimizations to enhance performance.
• Contributed to the development of solutions for bill generation, energy usage visualizations, and machine learning models, providing valuable insights to the utility company and its customers.

• Developed Android applications with a focus on seamless user experience and robust functionality.
• Created RESTful Java web services for a payment system client, ensuring secure and efficient transaction processing.
• Integrated various Clover devices with the First Data payment system, enabling smooth and reliable payment solutions.

Created a comprehensive school management platform utilizing Java and Spring MVC for backend development. Integrated Google Maps API for real-time school bus tracking and employed MySQL for database management. Implemented front-end interfaces using HTML5 and CSS to deliver a user-friendly experience.