NANTHINI SELVARAJ

• Architected a metadata-driven ELT pipeline on Databricks/Delta Lake using a custom Python framework, implementing a Data Vault 2.0 model across raw, curated, silver, and gold layers

• Built core Data Vault migration components (custom Hub/Link/Satellite classes) in PySpark, enabling accurate timestamp normalization and event-driven data loading

• Established YAML-based metadata configuration standards governing entity loading, surrogate key resolution, and datamart table definitions across the pipeline

• Resolved Delta Lake constraint violations by redesigning entity load ordering across hub, link, and satellite dependency chains, eliminating recurring pipeline failures

• Designed and implemented a surrogate key null-handling strategy (quarantine-and-drop pattern) to enforce referential integrity across fact and dimension tables, backed by automated unit tests

• Root-caused and resolved data integrity defects, including timestamp misalignment across silver-layer entities that was collapsing rows in historical join logic

• Managed end-to-end pipeline orchestration through Azure Data Factory and Azure DevOps, including parameter propagation across multi-stage pipeline chains and secrets management via Databricks

• Administered Unity Catalog access controls, diagnosing and resolving schema-level governance issues blocking data definition operations

• Maintained Git workflows across a multi-developer codebase, resolving complex rebase conflicts and branch synchronization issues

• Refactored legacy metadata conventions in response to peer code review, replacing hardcoded logic with typed constants and structured configuration formats

• Collaborated in code review cycles, iterating on YAML schema design and data normalization logic based on reviewer feedback

• Designed and developed data pipelines using Azure Data Factory and AWS Glue for ingesting structured and semi-structured data
• Built scalable ETL solutions using PySpark on AWS EMR and Azure Databricks
• Integrated AWS S3 and Azure Data Lake for secure, performant data lake implementations
• Worked on Ingesting data by going through cleansing and transformations and leveraging AWS Lambda, AWS Glue and Step Functions
• Developed Spark applications using Pyspark and Spark – SQL for data extraction, transformation and aggregation from multiple file formats
• Automated pipeline orchestration using AWS Step Functions and Azure Data Factory triggers
• Created Glue jobs for transformation, schema inference, and dynamic partitioning
• Worked with Athena and Redshift for analytics-ready data querying
• Developed fault-tolerant and performance-optimized Spark jobs with Spark SQL
• Built S3 buckets and managed policies for S3 buckets and used S3 bucket for storage and backup on AWS
• Ensured data lineage and compliance through logging and monitoring via CloudWatch and Azure Monitor

• Collected, cleaned, and standardized large datasets across multiple sources
• Migrated on-premise data warehouse systems to cloud and streamlined ETL pipelines using SSIS
• Designed and maintained Power BI dashboards for KPI monitoring across telecom business units
• Tuned complex SQL queries and stored procedures to enhance report generation performance by 50%
• Worked on integrating early cloud services with existing systems, including report automation and job scheduling

• Designed and delivered data models, reports, and visualizations using SSRS, SQL, and Excel for healthcare clients
• Developed stored procedures, T-SQL scripts, and validation routines for business-critical reporting
• Provided analytics support to various departments, ensuring accuracy and consistency in reports and insights
• Validated business logic and transformation rules applied during ETL processes