DHRUV DAVE - Data Engineer

Summary

Experienced Data Engineer skilled in batch data processing, ETL development, and REST API building to seamlessly integrate and analyze data. Demonstrated expertise in scalable data architectures and optimized data pipelines. Collaborative approach to machine learning applications, utilizing vector databases and large language models. Committed to delivering efficient and effective solutions for data integration and analysis.

Overview

2

years of professional experience

2

Certifications

Work History

Data Engineer

Rishabh Software Pvt Ltd

01.2023 - 08.2024

Efficiency Analysis: Conducted a comparative analysis of Spark optimization techniques, improving data processing speed by 30% and enhancing overall system performance by 25%
Data Lakehouse Implementation: Designed a data lakehouse infrastructure to support complex data processing and analytics, leading to improved efficiency and scalability
Data Flow Management: Engineered robust data flow processes that seamlessly managed streaming data from SharePoint to Azure Blob storage, achieving a 40% increase in processing efficiency while minimizing bottlenecks and improving system reliability
Pay-Per-Click Revenue Analysis Pipeline: Collaborated on developing a Single Source of Truth OLAP database to consolidate PPC data from Google, Microsoft, and Amazon Ads into Azure SQL, enabling real-time visualizations and improving advertising strategy optimization by 30%
RESTful API Development: Co-developed a RESTful API using Python (FastAPI) and Azure Web App to integrate Google, Microsoft, and Amazon Ads for a manufacturing client, streamlining Pay-Per-Click revenue analysis and increasing management efficiency by 70%
Seamless Data Pipeline Integration: Designed a workflow to upload files from SharePoint to Azure Blob Storage, triggering upserts into Qdrant vector database; improved RAG application response accuracy by 40% through collaboration with LLM teams using OpenAI models
Data Quality Management: Established and implemented data quality checks for a client's financial data, collaborating with data scientists to ensure the integrity of processed data for predictive modeling
This initiative led to a 35% reduction in errors, enhancing the reliability of analytics
Scalability and Portability: Developed a systematic approach to dockerizing the application, facilitating smoother rollouts and reducing configuration errors by 30% in production; the tool is now used by over 4 teams across the organization.

Education

Bachelor of Technology - Computer Science And Engineering

Parul Institute of Engineering And Technology

Vadodara, India

04.2019 - 01.2023

Skills

Databases: MSSQL, MYSQL, Azure SQL, Snowflake

Vector Database: QdrandDB

ETL Tools: DBT(Data Build Tool), Dataverse, SSIS

Containerization: Docker

Version Control: Git

Programming: Python

Cloud Platform: Microsoft Azure

Azure Services: Azure Databricks, Azure Function, Azure Data Factory, Azure Synapse Analytics, Azure EventHub

Data Processing: Apache Spark, Databricks, Snowflake

Data Visualization: PowerBI

API Building: FastAPI (POST, GET, PUT,DELETE)

Infrastructure as Code: Terraform

Certification

Microsoft Azure Data Fundamentals

Work Availability

monday

tuesday

wednesday

thursday

friday

saturday

sunday

morning

afternoon

evening

swipe to browse