Karthik K
Coimbatore
Summary
Senior Data Engineer with 4+ years of experience specializing in Data Quality Engineering within Azure-based data lakehouse environments. Strong expertise in validating end-to-end data pipelines across ADLS layers, performing large-scale source-to-target reconciliation, aggregation validation, and set-based integrity checks using Databricks SQL. Solid foundation in data warehousing concepts including fact/dimension modeling, SCD logic, and KPI validation. Experienced in supporting enterprise retail analytics platforms with high data accuracy standards.
Overview
5
5
years of professional experience
1
1
Certification
Work History
Senior Data Engineer – Data Quality
LTM (formerly LTIMindtree / Mindtree)
Coimbatore, Tamil Nadu
08.2021 - Current
- Performed end-to-end validation of Azure-based data lakehouse pipelines across RAW, Preprocess, Valid, Configured, and Processed layers.
- Executed source-to-target reconciliation using aggregation checks and bidirectional set-based validations (EXCEPT/MINUS queries).
- Validated incremental and full-load data processing using ingestion timestamp and latest-modified logic.
- Tested Databricks SQL transformations, KPI calculations, and Spark-based aggregations within Delta Lake architecture.
- Conducted functional, regression, and sanity testing for enhancement requirements impacting existing datasets and reporting outputs.
- Verified fact–dimension relationships and validated star schema models including Slowly Changing Dimension (SCD) logic.
- Performed backend-to-reporting reconciliation by validating curated data layers against Power BI semantic models.
- Validated Azure Data Factory (ADF) pipelines, orchestration logic, triggers, and integration with Databricks workflows.
- Conducted data profiling to identify anomalies, duplicates, null violations, and referential integrity issues.
- Logged and tracked defects in Jira with detailed reproduction steps and root-cause documentation.
- Designed reusable SQL-based validation patterns to improve testing efficiency and reduce manual reconciliation effort.
- Formerly LTIMindtree / Mindtree
Education
Master of Technology (M.Tech) - Software Engineering
Birla Institute of Technology and Science
Pilani01-2025
Bachelor of Science (B.Sc.) - Electronics and Communication Systems
Sri Krishna Arts and Science College
2021
Skills
Technical Skills
- Azure data platform
- Azure Data Lake Storage (ADLS Gen2)
- Azure Data Factory (ADF)
- Azure Databricks
- Delta Lake architecture
Data Quality and Validation
- Data quality and validation
- End-to-end pipeline validation
- Source-to-target reconciliation
- Aggregation and set-based validation
- Incremental and full load testing
- Data profiling and anomaly detection
- Referential integrity validation
- Data warehousing strategies
Data Warehousing
- Star schema validation
- Fact and dimension modeling
- Slowly changing dimensions (SCD)
- KPI and business rule validation
Programming & Querying
- SQL (Advanced – Joins, Aggregations, Window Functions, Set Operations)
- Scala programming skills
- Basic PySpark knowledge
- Spark-based transformations understanding
Tools
- Jira for defect management
- Power BI reporting validation
- Excel for data analysis
Certification
- Microsoft Certified: Azure Data Fundamentals (DP-900), 2023-01-01
- Microsoft Certified: Azure Fundamentals (AZ-900), 2022-08-01
Awards & Recognition
Excellence in Action Award – LTM
Recognized by leadership for critical contributions in data quality validation and ensuring high delivery standards during a key project milestone. Award presented during client-facing event.
Timeline
Senior Data Engineer – Data Quality
LTM (formerly LTIMindtree / Mindtree)
08.2021 - Current
Master of Technology (M.Tech) - Software Engineering
Birla Institute of Technology and Science
Bachelor of Science (B.Sc.) - Electronics and Communication Systems
Sri Krishna Arts and Science College