Vishal More

Banking Data Integration and ETL Pipeline Optimization:

• Designed and optimized Azure Data Factory (ADF) pipelines for seamless ingestion, processing, and transformation of diverse datasets into Azure Data Lake Storage (ADLS).
• Participated in agile development processes, contributing to sprint planning, stand-ups, and reviews to ensure timely delivery of data projects.
• Improved pipeline execution by 20% and optimized resources for handling high-volume data.
• Developed high-performing Databricks workflows to process JSON, Parquet, and Delta Lake data, with scalability in mind.
• Created and maintained database objects in Azure SQL, enhancing analytics and query efficiency.
• Automated end-to-end workflows with ADF and Databricks, reducing manual interventions by 30%.

Azure-Based Enterprise Data Integration Framework

Description:

• Designed and developed Azure Data Factory (ADF) pipelines to integrate and transform data from diverse on-premises and cloud sources into Azure Data Lake Storage (ADLS).
• Migrated databases from on-premises SQL servers to Azure SQL Database and Azure Managed Instances, ensuring minimal downtime, and data integrity during the transition.
• Implemented Delta Lake and Lakehouse Architecture on Azure Databricks to enable scalable big data processing and analytics.
• Collaborated with cross-functional teams to develop database objects, including schemas, views, and stored procedures, to meet analytics requirements.

• Azure Data Factory: Proficient in Linked Services, Data Sets, Pipelines, and Data Flows.
• Azure Setup: Skilled in setting up Azure Data Factory and Azure Data Lake Storage (ADLS).
• Database Migration: Experience migrating databases from SQL Server to Azure Managed Instance using Azure tools.
• Azure Databricks: Knowledge of its architecture.
• Data Migration Logic: Use of 'For' and 'If' conditions to migrate tables from on-premises to Azure SQL and ADLS.
• High Availability and Disaster Recovery (HA/DR): Expertise in clustering, Always On Availability Groups (AG), log shipping, mirroring, and replication.
• Backup and Restore: Proficient in troubleshooting and database restoration.
• SQL Automation: Automation of SQL processes, migrations, and installations.
• Performance Tuning: Handling performance issues, and optimizing queries.
• Cloud Setup: Skilled in SQL, Azure, and Azure Virtual Machine (VM) configurations.

• Installing, configuring, and managing MS SQL Server on production site servers.
• Having a good knowledge of isolation levels (dirty read, non-repeatable reads, phantom read).
• Configure Always-on, log shipping, and mirroring in production servers.
• Troubleshooting Always On/Log Shipping Errors.
• Rebuild the Master, Model, and MSDB databases.
• Creating Index (Type of Index, page splitting, fill factor, and fragmentation).
• Migrate/upgrade from lower versions to higher versions for the banking client.
• Taking backups, restoring backups, and scheduling backup jobs.
• Monitoring these servers remotely, and managing the databases of sizes ranging from 100 GB to 18 TB.