Azure Data Engineer
- Proficient in Data Lakehouse with experience in creating and implementing Delta Lake and ETL pipelines
- Responsible for developing, monitoring, and debugging issues in ADF pipelines and Databricks notebooks using PySpark and PySQL
- Practical experience in creating and maintaining both technical and functional documentation
- Managing snow incidents related to ADB and ADF in the backend, as well as maintaining Power BI dashboards to deliver business value
- Strong knowledge of developing, deploying, and visualizing AirFlow DAGs, as well as understanding their architecture
- I discussed the challenges our team faces in maintaining the applications
- I was involved in creating an end-to-end architecture to enhance the data flow for multiple projects