Rushikesh Dhangar
Data Engineer
Bengaluru
Summary
Accomplished Data Engineer at TheMathCompany Pvt. Ltd., specializing in ETL development and data pipeline design. Expert in Azure Databricks and Delta Lake, I enhanced data integrity and stability, mentoring junior developers while optimizing performance. Proven multitasker with a strong focus on delivering high-quality data solutions for machine learning and SAP-based enterprise applications.
Overview
3
3
years of professional experience
Work History
Senior Data Engineer
TheMathCompany Pvt. Ltd.
11.2022 - Current
- Designed and developed scalable PySpark jobs on Databricks to orchestrate data movement across medallion layers, ensuring high performance and reliability.
- Architected and implemented transformation logic to modernize legacy systems and establish a centralized Single Source of Truth (SSOT) across multiple enterprise data sources.
- Implemented structured logging, monitoring, and root cause analysis mechanisms to proactively detect and resolve data pipeline failures, significantly improving ETL stability and data integrity.
- Managed Delta Lake operations, including partitioning, optimization, and schema evolution, to ensure performance, scalability, and maintainability.
- Mentored junior team members in best practices for software development, code optimization, and troubleshooting techniques.
- Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.
- Created and managed Delta Lake tables in Azure Data Lake, implementing efficient incremental load (CDC-based) logic in Azure Databricks to optimize performance and reduce processing time.
- Developed and maintained PySpark-based AWS Glue jobs for data cleansing, transformation, and feature engineering to support machine learning models for sentiment analysis.
- Designed and implemented a reusable Python-based framework to standardize data processing logic, reduce code redundancy, and improve overall maintainability and code quality across pipelines.
Education
Bachelor of Engineering -
R. C. Patel Institute Of Technology
Shirpur, Maharashtra, India 04.2001 -
Skills
ETL development
Data pipeline design
Azure Services
Azure Databricks
AWS Services
Data migration
Big data processing
Spark framework
Hadoop ecosystem
Multitasking Abilities
Scripting languages
SQL programming
Certifications
Databricks Certified Data Engineer Associate (02/2025 - 02/2027)
Timeline
Senior Data Engineer
TheMathCompany Pvt. Ltd.
11.2022 - Current
Bachelor of Engineering -
R. C. Patel Institute Of Technology
04.2001 -