Kajol Patira

Data Migration Project:

• Developed Azure Data Factory pipelines for incremental data load that extracts data from SFTP Server.
• Developed scripts using Python, SQL, Docket to load data from ADLS Gen2 containers to the tables in the Snowflake data warehouse.
• Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
• Used Azure DevOps to maintain the code and docker file and to build the docker images in the Azure Container Registry.
• Build data wrangling scripts using python in Azure Databricks Notebooks to transform the data.

Market Trends Data Corpus:

• Designed solution architecture and implemented low-code no-code automated ADF pipeline to extract data from RSS feeds on stocks indexes from multiple news websites at various time frequencies.
• Scraped news articles using python scripts and azure functions.
• Performed Sentiment Analysis and Named Entity Recognition on the news articles using Azure Cognitive Services.
• Designed and implemented Error Handling and Pipeline Maintenance from user's point of view.

Covid-19 Trends in Europe:

• Ingested Covid-19 data from European center for disease prevention and control (ECDC) using HTTP connector in ADF pipeline to fetch data to ADLS gen 2, Also used Azure Blob Storage to store Population Data.
• Designed Pipeline with data flow activities for data transformation and stored data into Azure SQL DB.
• Created Reports on the Transformation of Data using Microsoft Power BI.