Saylee Patil

• Developed reusable batch processing pipelines for data ingestion and processing using Azure Data Factory, Azure Datalake, Azure SQL Databases to process data coming from 10+ sources
• Optimised existing ADF pipelines to reduce execution time significantly and optimised resource configurations to save cost
• Experience in writing complex SQL queries to retrieve data from SQL databases and optimization
• Worked on data archival of large SQL tables

Data Engineer/Scientist

• Developed and monitored ETL pipelines for near real-time processing using Azure Data Factory, Azure Databricks, Azure SQL database and Data Lake Storage
• Processed streaming data from Event hubs in Azure Databricks using PySpark and SQL
• Queried SQL tables to show the desired results on clients web application
• Provided processed data to Data Science models for forecasting time series, detecting anomalies and identifying ongoing events
• Implemented data quality checks to capture bad records from source
• Configured alerts and monitored resources for ADF pipelines, ensuring system reliability
• Contributed to Python code generalization efforts to enhance code efficiency
• Utilised Delta Tables in databricks for efficient & reliable data storage and management
• Optimised Databricks resources to save cost

• Worked closely with client to understand requirements
• Extracted raw data from Azure storage and processed it using Pandas to built a prototype of a data-based soft sensor for wastewater treatment
• Utilised feature selection techniques and built a predictive regression model for key parameters affecting water conductivity

• Pre-validated ADAS software using HILS and SILS environments ensuring vehicle safety
• Led and managed a team of 3-4 associates, coordinated with onsite members and trained newly joined associates