Murugan Elumalai

Ulundurpet

Summary

Results-driven Data Engineer with expertise in designing and optimizing scalable data pipelines using Azure Databricks, Spark, SQL, and Python. Specialized in building robust ETL workflows, enforcing strong data governance practices, and managing Unity Catalog within cloud data platforms. Proficient in implementing CI/CD pipelines using Azure DevOps to automate data deployment and testing workflows. Adept at collaborating with cross-functional teams to deliver reliable, secure, and high-performance data solutions in dynamic environments.

Overview

years of professional experience

Certification

Work History

Data Engineer

iLink Digital

Chennai

09.2021 - Current

Designed and implemented ETL processes based on detailed business requirements for various subject areas.
Developed scripts to automate data ingestion from SFTP and REST APIs into Azure Data Lake Storage (ADLS) and Databricks.
Executed data ingestion into multiple Azure services, including Azure Data Lake, Azure SQL, and Azure Blob Storage.
Built robust pipelines following the Medallion Architecture (Bronze, Silver, Gold) using various file formats (CSV, TXT, etc.).
Migrated data from Databricks to SQL Server, creating stored procedures for seamless transformation and loading into target tables.
Built scripts to enable bi-directional data movement between SQL Server and Databricks.
Developed and configured CI/CD pipelines using Azure DevOps to automate deployment and improve development workflow efficiency.

Education

Bachelor of Engineering - Electrical And Electronics Engineering

Saranathan College of Engineering

Trichy

04-2021

Skills

Languages/Tools: Python,PySpark, SQL, T-SQL, Databricks, Microsoft Fabric
Databases: SQL Server, Azure Data Lake

Data Engineering: ETL Development, Data Warehousing, Data Pipeline Optimization