Summary
Overview
Work History
Skills
Timeline
Generic
SAI KRISHNA

SAI KRISHNA

Hyderabad

Summary

  • Senior Data Engineer with over 7 years of professional experience in the field. Demonstrated expertise in a wide range of technologies, including Python, MySQL, Azure Data Factory, Azure Databricks, Azure Synapse, PySpark and RPA.
  • Demonstrated expertise in building scalable data pipelines and modern data warehouse solutions to support increasing data volume and complexity. Proven ability to collaborate effectively with cross-functional teams to deliver business-driven analytics solutions.
  • Reliable Data Engineer with a strong passion for designing, developing, and implementing data pipelines in Azure Data Factory. Used Azure Data Factory to move and transform data from various sources like databases, files, APIs, and cloud services. Skilled at delivering data to destinations such as Data Lakes, Data Warehouses, and databases.
  • Demonstrated expertise in creating and maintaining database objects such as Tables, Views, Indexes, Triggers, and Sequences, ensuring efficient data storage and retrieval mechanisms. Implemented best practices for database design and optimization to enhance performance and scalability.
  • Proficient in transforming structured and semi-structured data from various sources using Mapping Dataflow Transformations within Azure Data Factory. Utilized a range of transformations including Join, Conditional Split, Lookup, Union All, Sort, Aggregation, Derived Column, Pivot, Parse, Rank, Window, etc., to standardize and enrich data for downstream analytics and reporting purposes.
  • Created pipelines in ADF using linked services/Datasets/Pipeline/to Extract, Transform and load data from different sources like Azure SQL, Blob storage,Azure SQL Data warehouse.
  • Implemented email feature using Azure Logic Apps to automate pipeline notifications, ensuring timely communication and efficient monitoring of data workflows.
  • Experience in developing My SQL Stored Procedures, Functions, Cursors.
  • Demonstrated proficiency in managing key vaults services, ensuring secure storage and management of sensitive data assets in accordance with industry best practices.
  • Led the implementation of CI/CD pipelines for Azure Data Factory (ADF), streamlining the deployment process and enabling continuous integration and delivery of data pipelines.
  • Applied expertise in implementing Incremental pipeline with watermark, optimizing data processing efficiency and minimizing resource utilization for enhanced performance.
  • Managed the publication, scheduling, and triggering of pipelines on a daily and weekly basis, aligning with the specific business requirements of clients. This involved orchestrating data workflows to ensure timely execution and delivery in accordance with client expectations.
  • Demonstrated experience in scheduling Databricks notebooks within Azure Databricks and Azure Data Factory, optimising data processing workflows for efficiency and reliability.
  • Proficient in utilizing Azure Data Factory for ETL processes, including delta loads and insert-update loads, streamlining data integration and transformation tasks. Successfully automated these processes to enhance productivity and accuracy.
  • Leveraged Azure DevOps for effective management of Azure Data Factory and Azure Databricks across multiple environments. Implemented best practices for version control, deployment, and monitoring, ensuring seamless collaboration and smooth operations.

Overview

8
8
years of professional experience

Work History

Azure Data Engineer

Quadrant Technologies
Hyderabad
02.2024 - Current

Project Overview: Data Modernization for Haldiram’s Supply Chain and Sales Analytics

Haldiram’s, a leading snack and sweets manufacturer, aimed to modernize its data infrastructure to support real-time analytics for supply chain optimization, sales trends analysis, and customer behavior insights. The project involved migrating on-premises legacy systems to Azure and implementing a robust data platform to enhance decision-making and operational efficiency.

Key Responsibilities:

  • Designed and developed end-to-end data pipelines using Azure Data Factory (ADF) to extract, transform, and load (ETL) data from various sources into Azure Data Lake Storage (ADLS).
  • Implemented data ingestion workflows to integrate structured and unstructured data from ERP, CRM, and POS systems.
  • Set up and optimized Azure Synapse Analytics for data modelling and querying to support business intelligence reports.
  • Defined schema and created Azure SQL tables in Azure using Sql queries.
  • Deployed Azure Databricks to process large volumes of data for predictive analytics, including demand forecasting and customer segmentation.
  • Ensured data security and compliance by implementing role-based access controls (RBAC), encryption, and Azure Key Vault for sensitive data.
  • Collaborated with data scientists and analysts to develop machine learning models using Azure Machine Learning integrated with Databricks.
  • Automated pipeline monitoring and failure notifications using Azure Monitor and Logic Apps.

Azure Data Engineer

FIS Global
Bangalore
10.2021 - 09.2023

Project Overview: Healthcare Data Integration and Analytics Platform
A leading healthcare provider sought to modernize its data infrastructure to improve patient care, streamline operations, and ensure compliance with healthcare regulations. The project involved creating a unified data platform to integrate patient records, clinical data, and operational metrics while enabling advanced analytics and real-time reporting.

Key Responsibilities:

  • Designed and developed ETL pipelines using Azure Data Factory to ingest data from electronic health records (EHRs), IoT devices, and operational databases into Azure Data Lake.
  • Built a healthcare data model in Azure Synapse Analytics to support queries for clinical insights and patient history analysis.
  • Processed unstructured data, such as doctor’s notes and diagnostic images, using Azure Databricks and integrated them into the analytics pipeline.
  • Ensured data security and compliance by implementing Azure Policy, data masking, and encryption in transit and at rest.
  • Automated anomaly detection for patient vitals and operational data using machine learning workflows in Azure Machine Learning.
  • Monitored data pipeline performance using Azure Monitor and set up alerts for failures or bottlenecks.

Application Support Engineer

HCL Technologies
Hyderabad
03.2017 - 09.2021

Project Overview: Azure-Based Application Support for a Python Web Application

A logistics company deployed a custom Python-based web application for real-time shipment tracking and order management. The application required ongoing support to ensure high availability, optimize performance, and resolve issues. The infrastructure was hosted on Azure, with integration for monitoring, scaling, and security.

Key Responsibilities:

  • ● Monitored application health and performance using Azure Monitor and Application Insights, ensuring uptime and quick incident resolution.
  • Automated log analysis and error detection using Python scripts integrated with Azure Log Analytics.
  • Responded to user-reported issues, identified root causes, and implemented hotfixes in collaboration with the development team.
  • Managed Azure App Service configurations, including scaling, SSL certificate updates, and custom domain settings.
  • Set up and monitored Azure Functions for background tasks, such as periodic data syncing and email notifications.
  • Wrote PowerShell and Python scripts to automate tasks like VM snapshots, resource tagging, and cost optimization.
  • Ensured data security by managing Azure Key Vault for API keys and sensitive configurations.
  • Documented troubleshooting guides, standard operating procedures (SOPs), and user guides for the application.

Skills

  • Python
  • PySpark
  • Azure Data Factory
  • Azure Databricks
  • Azure Data Lake
  • Azure blob storage Gen 2
  • Azure Synapse
  • Azure Functions
  • Azure logic
  • Azure
  • JIRA
  • Bitbucket
  • Git
  • GitHub
  • Azure Monitoring
  • Azure Log Analytics
  • Azure SQL database
  • Azure SQL DW
  • Microsoft SQL Server

Timeline

Azure Data Engineer

Quadrant Technologies
02.2024 - Current

Azure Data Engineer

FIS Global
10.2021 - 09.2023

Application Support Engineer

HCL Technologies
03.2017 - 09.2021
SAI KRISHNA