N.S.D. Parvathy Ch

Results-driven ETL professional with almost 2 years of experience in Azure, Database technologies. Proficient in crafting robust data pipelines using Azure Data Factory, facilitating seamless data transfer across diverse sources and destinations. Skilled in architecting intricate SQL database structures to meet dynamic business needs.

Expertise in designing scalable data pipelines using PySpark. Developed end-to-end data processing solutions on Azure, utilizing Azure Data Factory for data cataloging, ETL orchestration, and schema inference, alongside Azure data lake for scalable storage of raw and processed data.

Project 1:

Azure Data Integration Project 6/2022-9/2022

• Orchestrated end-to-end data integration between Azure Blob Storage and Snowflake using Azure Data Factory.
• Managed Azure resource provisioning using Terraform for optimized infrastructure.
• Automated Azure and Snowflake resource creation via Azure DevOps Pipeline, ensuring streamlined deployment.
• Executed data transformations within Azure Data Factory to enhance data quality.
• Implemented secure authentication using Azure Key Vault for enhanced data security and access control.

Project : Movie Lens Recommendation System.

Environment: 3/2023-7/2023

Azure Portal,ADF(Azure Data Factory),Databricks Key Responsibilities:

⋄ Developed and deployed scalable data pipelines and workflows using Azure Databricks notebooks and jobs, orchestrating data ingestion, transformation, and loading processes.

⋄ Implemented data transformation and manipulation logic using PySpark RDDs (Resilient Distributed Datasets) and DataFrames, ensuring data quality and integrity.

⋄ Implemented batch and streaming data processing solutions using PySpark Streaming and Azure Databricks Structured Streaming, enabling real-time insights and decision-making.

SnowIOT:

Automotive IoT Data Analytics Project 9/2023-1/2024

Project Description:
Implemented a comprehensive IoT data analytics solution, SnowIOT, focused on real-time vehicle telemetry data analysis. Leveraged Microsoft Azure and Snowflake to store and analyze data, enabling insights into driver behavior and mileage.

Key Responsibilities:

• Provisioned and configured Kafka Setup to enable real-time data ingestion from Azure Event Hub to Snowflake.
• Transferred and transformed data from Azure Event Hub to Snowflake using Kafka, ensuring data integrity and reliability.
• Flattened complex data structures in Snowflake for streamlined analysis and reporting.
• Developed and implemented scripts in Python to generate random data for testing and simulation purposes.

Technologies Used:

• Microsoft Azure (Azure Event Hub, Azure Functions, Azure Virtual Machine),Kafka,Snowflake,Python

• Azure Data Fundamentals
• Azure Fundamentals
• Snowflake Snow Pro Core