Mohammad Ali

Etl pipeline using ADF

• Ingesting data into Adls Gen 2
• Data sources File Systems, WebAPIs, Sql and No-Sql DBs from both on premise and cloud
• Transformed ingested data using Databricks
• Alert system using Logic App
• Resource Monitoring system using Azure Workspace
  

End to End data pipeline on Synapse

• Migrated data from on premise to azure synapse using pipeline component, stored raw format in Adls Gen 2
• Enrich and transformed data using Spark pool
• Performed Transactional Upsert and SCDType2 using sql pool
• Transferred data in Dedicated Sql Pool by dividing into facts and dimensions for further analysis