SAMRAAT UDAYAGIRI

Client : Nestle

Role : Azure Data Engineer

• Orchestrated complex data workflows using Azure Data Factory pipelines and triggers, resulting in a 45% increase in process automation.
• Implemented dynamic pipeline parameters to support multi-environment deployments, leading to a 30% reduction in deployment time.
• Created custom connectors and integration runtime configurations within Azure Data Factory, enhancing data source connectivity by 50%.
• Utilized Azure Data Factory monitoring and logging features to proactively identify and resolve data pipeline issues, leading to a 20% decrease in system downtime.
• Established Delta Lake transactions and ACID compliance on Azure Databricks Delta Tables, resulting in a 40% increase in data reliability.
• Engineered Azure Data Lake Storage solutions with dynamic data partitioning and lifecycle management, resulting in a 35% cost reduction in data storage.
• Designed and implemented historical data versioning using Delta Lake's time travel feature, leading to a 30% improvement in auditability and compliance.
• Leveraged Delta Lake schema evolution to adapt to changing data structures without downtime, achieving a 25% increase in data integration flexibility.
• Optimized Delta Lake table partitions to improve query performance and manage large datasets efficiently, resulting in a 35% reduction in query latency.
• Developed Delta Lake-based streaming analytics solutions using Azure Databricks Structured Streaming, yielding a 50% increase in real-time data processing capabilities.
• Engineered scalable data migration solutions into Azure Data Lake, integrating 7 primary data sources using Oracle Tools, effectively accommodating a 50% increase in processing to support growth strategies.
• Implemented data encryption, achieving 62% data breach reductions and full compliance with Indian banking standards, bolstering data protection and regulatory adherence through effective communication & collaboration with different teams.
• Established robust data governance models on Azure Databricks using the Unity Catalog, achieving a 100% improvement in data management and promoting data governance practices.
• Implemented robust data security measures by leveraging Azure Key Vault and role-based access controls, achieving compliance with GDPR and HIPAA regulations.
• Deployed data pipelines into different environments using Azure DevOps Git, ensuring version control and streamlined deployments.

Client: Marriott Hotels, Role: Azure Data Engineer.

• Designed, developed, and deployed end-to-end data pipelines on Microsoft Azure using Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics.
• Implemented data integration and ETL processes to ingest structured and unstructured data into Azure Data Lake and data warehouses.
• Optimized data pipelines for performance, scalability, and cost-efficiency to meet SLAs and business requirements.
• Ensured data security and compliance by applying role-based access control and data encryption within Azure services.
• Monitored pipeline health and performance, proactively troubleshooting and resolving issues to minimize downtime.
• Integrated Azure Databricks with Azure Synapse Analytics to enable scalable and high-performance data processing.
• Collaborated with data scientists and business analysts to translate data requirements into efficient workflows using ADF and Databricks.
• Implemented continuous data ingestion pipelines using Spark Streaming, enabling real-time processing in micro-batches.
• Worked with data architects to design schemas and define ingestion strategies for Spark Streaming applications.
• Built parameterized ADF pipelines to ensure flexibility and reusability across multiple environments and data sources.
• Collaborated with Azure DevOps teams to implement CI/CD pipelines for ADF and Databricks notebooks, automating deployment, and version control.

Project: Nordea Bank, Azure Data Engineer.

• Implemented end-to-end data integration and orchestration using Azure Data Factory (ADF) to extract, transform, and load data from multiple sources into Azure Data Lakes and Data Warehouses.
• Designed and developed complex ADF pipelines to process large volumes of data, improving throughput, and reducing latency.
• Optimized ADF pipelines by parallelizing tasks and leveraging ADF Data Flows, reducing overall processing time by 30%.
• Enhanced real-time data ingestion using Azure Stream Analytics and Event Hubs, achieving a 50% reduction in data latency.
• Integrated Azure Databricks with Azure Data Lake Storage Gen2 for scalable storage, and seamless analytical data access.
• Built high-performance data transformation pipelines in Azure Databricks using PySpark to process large datasets efficiently.
• Implemented automated data refresh and monitoring for ADF using Azure Monitor and Log Analytics, improving reliability.
• Reduced pipeline failures by 25% through error handling and retry mechanisms in Azure Data Factory workflows.
• Migrated on-premises data warehouses to Azure Synapse Analytics, reducing operational costs by 40%, and improving scalability.
• Established data governance frameworks using Azure Purview for cataloging and lineage tracking, enhancing data quality and accessibility by 50%.