APURBA PAL

Data Quality Framework Development on Snowflake:

• Designed and implemented a comprehensive data engineering framework on Snowflake, utilizing metadata tables and stored procedures to conduct data health and business-specific validations on finalized datasets.
• Integrated the framework with Power BI to enable accurate and reliable dashboard creation, ensuring seamless data quality monitoring for end users.

Data migration from Snowflake to Microsoft Fabric:

• Contributed to a lift-and-shift migration of data systems from Snowflake to Microsoft Fabric, converting SnowSQL to Spark SQL, and transforming JavaScript-based stored procedures into PySpark notebooks.
• Ensured the preservation of existing logic and functionality while optimizing runtime performance during the migration process.