Abhinav Sharma

Mars

Data Quality Framework Implementation Using Great Expectations:

• Designed and deployed scalable data quality framework using Great Expectations library for validation of over 100 datasets across multiple pipelines.
• Automated expectation suite generation with dynamic configuration files, reducing manual effort by 80% in data validation setup.
• Implemented custom validation checks such as null value detection and value range verification to ensure data integrity.
• Integrated validation results into data pipeline to prevent downstream processing of invalid data, enhancing overall data reliability by 40%.

Telstra (telecom domain)

Received Customer Experience:

• Built pipelines to process data from diverse sources, ensuring quality, quantity checks and analyzed metrics related to customer calling and data usage experiences.
• Adapted framework for seamless platform migration.
• Monitored anomalies in data, addressing pipeline failures and executing backfills for data loss.
• Spearheaded end-to-end automation of data quality assurance process, achieving cost savings exceeding $30,000.

PNC (BFSI Domain)

Big Data Hub Protegrity Tokenization:

• Expanded data protection capabilities by implementing tokenization for sensitive data.
• Developed framework with custom Hive, Spark, and Impala UDFs for Protegrity integration.
• Led the team in tokenizing 220 TB of data, ensuring secure access restrictions, and maximizing application performance.
• Facilitated movement of tokenized data between clusters using various Hive replication techniques.