Niraj Sonar

Project 1: Data Migration – Banking Sector

• Led data migration efforts from Hadoop to Azure Data Lake Storage (ADLS), ensuring seamless transition and improved performance.
• Re-engineered legacy Python scripts to be compatible with ADLS, replacing Hadoop commands with a new framework optimized for ADLS.
• Transformed Hive-based systems into modern Apache Spark applications, ensuring compatibility with current technology stacks.
• Converted Hive create table scripts into efficient SQL-based table creation scripts, facilitating smooth data handling and management.
• Collaborated closely with cross-functional teams to ensure successful deployment and testing of the new data architecture.

Project 2: KPI Web Application – Mining Sector

• Developed a comprehensive web-based system to track the company's Key Performance Indicators (KPIs), delivering real-time insights through SQL stored procedures.
• Created SQL-based logic to generate detailed KPI reports, covering daily, weekly, monthly, quarterly, and yearly performance metrics, along with visually rich graphs and a key-issues section.
• Spearheaded performance optimization, reducing report load times from 6 seconds to 2 seconds by shifting complex logic from the UI to the SQL backend.
• Optimized a critical stored procedure, reducing its processing time from 12 seconds to 4 seconds.
• Built dynamic calculation logic to calculate KPIs using a formula provided in the master data for specific KPIs.
• Implemented a data validation framework using Azure Data Factory and Databricks notebooks, comparing curated layer data with warehouse fact tables for automated KPIs.
• Streamlined master data management by handling create, update, and delete operations for dimension tables, ensuring data integrity.
• Played a key role in Azure DevOps for managing code deployments, resolving merge conflicts, and ensuring smooth releases through effective pull request management.