Vasundhara Pawar

RetailADFusion:

• Created ADF pipeline to copy raw data to Azure Delta Lake Storage staging area efficiently.
• Ensured seamless data transfer and integrity preservation during the migration process.
• Utilized PySpark in Databricks for tailored data transformations, meeting business needs.
• Implemented optimized code practices, including partitioning and caching, to enhance system efficiency and speed up processing.
• Engaged in active collaboration with team members to comprehensively understand data requirements and ensure the processed data aligned with their needs.
• Facilitated productive discussions and feedback loops to iteratively refine data transformation processes.

Sales Analytics:

• Applied Copy data activity to ingest raw data from source.
• Integrated Databricks activity to trigger a notebook in Databricks.
• Utilized Copy data activity to move transformed data to the destination.
• Read the ingested data using PySpark, DataFrame APIs.
• Applied transformations and aggregations using PySpark functions and Spark SQL queries.
• Wrote the processed data to a destination, such as another Azure Blob Storage container.
• Specified the Databricks Linked Service connection.
• Ran the ADF pipeline to execute the data flow seamlessly.