ASHISH BARWAL

Project: Data Migration.

Client: World Bank Group.

• Design and implement scalable data ingestion pipelines using Azure Data Factory (ADF) to extract data from SAP ECC legacy systems and load it into Azure Data Lake Storage Gen2 (ADLS Gen2) in CSV format.
• Develop and maintain medallion architecture layers (bronze, silver, gold) in Databricks SQL Warehouse, supporting efficient data staging, transformation, and analytics.
• Build and optimize data transformation and modeling workflows using dbt, including data cleaning, deduplication, and business logic implementation to deliver reliable silver-layer tables and gold-layer views.
• Manage data governance, access control, and metadata using Databricks Unity Catalog to enforce security policies and compliance standards.
• Integrate Azure Key Vault for secure management of credentials and sensitive configurations across data pipelines and Databricks environments.
• Implement CI/CD pipelines using Azure DevOps to automate build, test, and deployment processes for Data Factory, dbt models, and Databricks resources, ensuring consistent and reliable releases across development, test, and production environments.
• Configure Azure Logic Apps to send automated email notifications for pipeline status updates (success, failure, start) in ADF, enabling real-time monitoring and rapid responses to data pipeline events.
• Collaborate with cross-functional teams to align data models and pipelines with business requirements and reporting needs.
• Monitor and troubleshoot data pipeline performance and data quality issues, ensuring high availability and accuracy of migrated data.
• Automate the deployment and version control of dbt models and data pipelines using CI/CD best practices to improve release cycles and reduce errors.

Project: Integrated Enterprise Data Platform on Azure.

Client: Cigna Health Care.

• Collaborated with cross-functional teams to gather and understand data requirements for various business units.
• Designed and implemented end-to-end data pipelines using Azure Data Factory, ensuring efficient data flow and processing.
• Developed and maintained data models in Azure Synapse Analytics for optimal performance and scalability.
• Conducted performance tuning and optimization of SQL queries for improved data retrieval times.
• Developed and maintained ETL processes to extract, transform, and load data from various sources into Azure SQL Data Warehouse for analytical purposes.
• Implemented data quality checks and monitoring mechanisms to ensure the accuracy and integrity of the data.
• Utilized Azure Databricks for big data processing and analytics, optimizing performance and resource utilization.

Project: Centralized Data Management Platform

• Responsible for building scalable distributed data solutions using Hadoop.
• Developed Spark scripts by using PySpark shell commands as per the requirement.
• Optimizing existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames, and Pair RDD.
• Experienced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective and efficient Joins, Transformations, and others during the ingestion process itself.
• Designed, developed, and did maintenance of data integration programs in a Hadoop and RDBMS environment with both traditional and non-traditional source systems as well as RDBMS and NoSQL data stores for data access and analysis.
• Analyzed the SQL scripts and designed the solution to implement using Pyspark.
• Implemented Partitioning, Dynamic Partitions, and Buckets.
• Used Reporting tools like Tableau to connect with Hive to generate daily reports.

Project: Centralized Data Management Platform

• Responsible for building scalable distributed data solutions using Hadoop.
• Developed Spark scripts by using PySpark shell commands as per the requirement.
• Optimizing existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames, and Pair RDD.
• Experienced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective and efficient Joins, Transformations, and others during the ingestion process itself.
• Designed, developed, and did maintenance of data integration programs in a Hadoop and RDBMS environment with both traditional and non-traditional source systems as well as RDBMS and NoSQL data stores for data access and analysis.
• Analyzed the SQL scripts and designed the solution to implement using Pyspark.
• Implemented Partitioning, Dynamic Partitions, and Buckets.
• Used Reporting tools like Tableau to connect with Hive to generate daily reports.

• Supervision of 2 production shops (Header Shop and Coil Shop) and production processes.
• Fulfilled Responsibility including solving technical issues, providing advice, Manpower Management and preparing daily Production reports.
• Creation and maintaining bill of materials (BOM).
• Planning & generation of the reports for the production Flow/Machining process for components.
• Coordination with other team members to enhance process and performance.
• Supervised the work & employees for the timely completion of the project.
• Coordination with quality control staff to complete inspections and Supported quality team members during Corrective action updates.