PALLAVI C N

Experienced Data Engineer with over 3 years of expertise in designing, developing, and optimizing ETL pipelines. Skilled in data integration, transformation, and storage solutions, with a strong focus on efficiency and performance. Seeking an opportunity to apply my technical skills and problem-solving abilities to contribute to data-driven strategies and innovative solutions in a dynamic and challenging environment.

• Microsoft Azure Data Fundamentals (DP 900)
• Databricks Accredited Lakehouse Fundamentals

ODS 

Summary : 

Integrated data from different source like ITSM, Chat, Device Metrics, projects from SharePoint and other ticketing tools and Rest API. Using Azure data Factory Azure Databricks and SSIS Packages. Responsible for data discovery, data mapping and data engineering activities. 

• Created Data mapping sheets with standard format and mapped data with Power BI dashboards data received from Client to Actual datasets available in Blob Storage.
• Established robust connections to various data sources to acquire data in formats including JSON and CSV file Implemented data pipelines for efficient data loading into Azure Blob Storage and Server.
• Leveraged SQL Server stored procedures and views to streamline data transformation and load processes.
• Proactively identified potential issues in the ETL process and deployed preemptive activities and code to prevent failures.
• Ensured continuous integration, maintaining high data pipeline efficiency throughout development lifecycle.

Unilever 

Summary : 

Design an Azure Data Factory pipeline for transferring data from Sharepoint to Blob and from blob to SQL server database. The data files are received monthly.

• Created an Azure App Registration to facilitate connectivity between SharePoint and Azure Blob Storage. Created Required Linked Servers and Datasets to Proceed with ADF pipeline.
• Built an Azure Data Factory (ADF) pipelines to Ingest raw data from Sharepoint into BLOB and From BLOB to SLQ server Tables.
• Create a Stored Procedure responsible for transforming and loading data from the Staging tables into the User access layer, ensuring data accuracy and consistency.
• Design and set up triggers to schedule the execution of jobs on a weekly basis, automating the data processing tasks efficiently.
• Develop separate stored procedures for handling configuration tables and log table These tables serve to manage lookup data and log pipeline details, contributing to the overall monitoring and maintenance of the data pipeline system.