ABHIJITH S H

Accomplished Data Engineer at QBurst, adept in designing high-performance ETL pipelines and cloud data platforms. Proficient in Apache Airflow and AWS Glue, I excel in data transformation and quality assurance. My strong analytical skills and attention to detail drive successful project outcomes, ensuring data integrity and usability for stakeholders.

LegalZoom – Business Profile Platform (Single Source of Truth)

• Designed and maintained a Single Source of Truth (SSOT) platform consolidating enterprise business data from 45+ U.S. state systems.
• Orchestrated complex Apache Airflow DAGs to process multi-GB structured and semi-structured datasets, improving workflow reliability.
• Implemented pre-processing validations (schema checks, column count, data integrity), reducing downstream data issues by ~60%.
• Optimized transformation workloads using Snowflake’s high-performance processing, improving pipeline execution time by ~35–40%.
• Eliminated manual validation by leveraging Snowflake’s native error-handling, reducing reprocessing effort by ~30%.
• Automated file ingestion using Airflow sensors (including SFTP sensors), completely removing manual triggers and improving ingestion reliability.

Tech Stack: Apache Airflow, Snowflake, AWS S3, Python, Docker

Access Control Dashboard - ETL Platform

• Built a data integration and automation framework, ingesting data from multiple APIs to support dynamic updates and historical tracking.
• Developed secure Flask APIs using JWT authentication to control data access.
• Implemented Celery-based asynchronous processing, improving ingestion throughput by ~25%.
• Designed optimized schemas and queries across MySQL and MSSQL, reducing query latency by ~20%.
• Implemented incremental and bulk loading strategies to manage historical and current datasets efficiently.

Tech Stack: Python, Flask, Docker, Pandas, Celery, MySQL, MSSQL.