Detail-oriented, organized, and meticulous employee. Works at fast pace to meet tight deadlines. Enthusiastic team player ready to contribute to company success.
Overview
4
4
years of professional experience
2
2
Certifications
Work History
Software Engineer
Microsoft
1 2024 - Current
Part of Fabric Live Table Team (Development Stage), whose work is to orchestrate Spark Jobs That support Materialized Views/Live Tables in a microservice architecture
Handling, Debugging and Improving C# API implementation for spark Jobs Dag(s) generation & Execution, Dag Execution History for UI across and error handling.
Adding complete UT coverage for the same
Implementing Client interfaces to leverage other fabric services.
Software Engineer
Microsoft
06.2022 - 01.2024
Responsible for developing, building, testing, debugging, deploying HA, fault-tolerant Flink Clusters on Kubernetes
Played major role in 5+ Customer Onboarding, mostly that are to do with Flink SQL/formats, HA availability, Integration with other Azure offerings
Integrated Flink SQL Service into existing systems, increasing AAD security capability and improving overall performance.
Flink was integrated with HMS, Hive, and was made to support delta, Iceberg and Hudi Format
Integrated SQL Gateway service that support third party client like Beeline.
Actively engaged with OSS Issues on integration bugs
Enabled common Platform Service Script Action for HILO clusters and was particularly useful for Flink class loading issues.
Cloud Data Engineer
Intel Technologies
08.2020 - 05.2022
Developed rule-based predictive analysis of compute/memory requirements for a spark function (resource utilization metrics), and optimize, if possible, spark SQL queries by reducing exchange and sort exercises in physical plan
Benchmarked AWS EMR spark flavor on Intel, AMD and Graviton using TPC-DS
Improvement in spark SQL perf by 37% for 1 TB TPCDS run by tuning spark conf parameters
Benchmarked Intel Gazelle spark Engine against TPC-DS
Presented best practices from Spark cluster configuration, DW Storage Layout, spark config settings and physical plan POV at Intel Bootcamps to 100+ APJ Customers (single-handedly by me).
Technical Intern
Intel Technologies
- 05.2020
Curated Intel OpenVINO applications, for Intel EII (Intel edge software)
Enabled TCS Customers to build/deploy their Data Analytics models on EII and extending the scope of EII for Time-Series use case.
AI and Compute Performance Intern
Intel Technologies
- 12.2019
Trained UNET Model for Tumor segmentation in 3D Brain MRI images using Distributed Deep Learning Framework Horovod on top of Open MPI / Intel MPI to reduce the training Time without compromising the accuracy of the model RESULT: Training time for each epoch reduced from 3 hours to 125 seconds with 82% dice-coefficient
Optimized the NN model to minimize False-Positive/True-Negative cases by playing with loss function.