Data Engineer with more than three years of work experience in ETL , Data Warehousing and Reporting .
Project: Reporting and analytics in the gaming domain.
Worked on different POCs:
Version Control for Scripts :
Developed a solution to the latest generated script and replaced it with the previous version in Git using powershell scripting.
PowerBI Refresh :
Developed and implemented a data pipeline leveraging Pyspark that automate the process of checking for new data files, updating Metadata , and triggering a PowerBI refresh so that he reports reflect the latest data.
Designed and implemented functions for calling APIs and refreshing PowerBI datasets, enhancing data reporting capabilities
Configured and managed Azure Data Lake Storage authentication for secure data access within the Spark environment.