MANISH NIGAM

• Designed and built Azure ADF pipelines to ingest data into ADLS from SQL Server.
• Implemented activities: Copy activity, Execute Pipeline, Get Metadata, If Condition, Lookup, Set Variable, Filter, For Each pipeline activities for on-cloud ETL processing.
• Developed a Databricks notebook to read ingested data (ESV, Parquet) from ADLS, perform ETL transformations in PySpark, and load it to a Delta table.

Role & Contribution

• Developed a Databricks notebook to read data from SQL Server, Synapse Server incrementally, and perform ETT. Transformations in PySpark and load to Databricks Delta table.
• I have completed the code migration from Synapse and SAS code to Databricks PySpark code, which includes the ETL process.
• Performed data validation by comparing existing data with migrated code.
• Performed unit testing and collaborated with the QA team.
• I have completed the migration of complex SQL stored procedures into PySpark code.

Technology and Tools

• Created workflows and orchestrated Databricks notebooks.
• Implement SCD Type 2 in ADB notebooks using merge.

• Followed TradeEdge Process model
• Prepared and updated module specifications as per customer requirement
• Development of new functionality and enhancement of existing functionality as per design specifications
• Prepared SSIS ETL scripting for input files using SQL
• Prepared requirement gathering document by communicating with customer
• Provided Level 1 to Level 3 support to customer
• Transform Client Data using Stored Procedure and function
• Involved in activities related to important extracts for customer
• Develop, design, and administer database functionalities using T-SQL
• Design, develop, and optimize database solutions using SQL Server and other relevant technologies
• Actively monitor databases to troubleshoot production issues and identify optimization opportunities
• Independently analyze, solve, and correct issues in real-time, providing end-to-end problem resolution
• Troubleshoot database issues and perform root cause analysis
• Refine and automate regular processes, track issues, and write documentation
• Interact with customers and participate in business and system requirements sessions.

• Development BFSI module with BANK and Insurance Company
• Prepared and updated design specifications as per customer requirement
• Development of new functionality and enhancement of existing functionality as per design specifications
• Prepared requirement gathering document by communicating with customer
• Patch deployment Activities
• Follow SSIS tool to Transform and Integrate data
• Involved in activities related to important extracts for customer
• Handling group of three subordinates
• Overshadowing their tasks and challenges
• Involved and helping in functional configurations along with technical configurations
• Involved and understanding integrations with third system with the help of product feature
• Provide database support for critical business operations and resolve reported issues within SLAs
• Actively monitor databases to troubleshoot production issues and identify optimization opportunities
• Review pull requests and optimize SQL code submitted by developers.

• Implementing and developing MS SQL, MySQL, and ASP.NET.
• Active Directory and related services.
• Understanding the knowledge of backup and restore tools on the MS platform.
• Data Center Visit for Maintenance.
• Ticket resolution process and provide quality Level 1 support.