Ashraf Baba Ns

Built scalable, modular ETL solutions incorporating Slowly Changing Dimension Type 2 (SCD Type 2) logic to maintain historical accuracy of dimension data in data warehouse models. Ensured referential integrity and traceability for evolving business entities.

Implemented comprehensive data quality frameworks, including schema validation, null checks, duplicate detection, and business rule enforcement. Developed logging, alerting, and monitoring systems to proactively manage data reliability and pipeline performance.

Collaborated closely with data governance teams to embed governance principles into engineering workflows. Applied metadata management, access control (RBAC), data lineage, and tagging strategies to ensure compliance, transparency, and secure handling of sensitive information. Promoted a culture of data stewardship and supported enterprise-wide data trust and compliance initiatives.

• Designed and maintained scalable ETL pipelines using Databricks (PySpark) on both Azure and AWS, supporting batch and real-time data processing. Leveraged Databricks Autoloader for efficient ingestion of streaming and incremental data from cloud storage into Azure Data Lake. Integrated Azure Data Factory for orchestration of data workflows and scheduling. Enabled seamless migration of structured and unstructured data across cloud and on-premises environments. Optimized data workflows for performance and scalability, handled dynamic schema evolution, and implemented robust data quality checks to ensure accurate and reliable data delivery.

Have strong foundation in building scalable data pipelines, automating ETL processes, and delivering data-driven insights using cloud platforms like Microsoft Azure and AWS. My experience includes automating data ingestion workflows from SQL databases to Azure Data Lake (DBFS) for preprocessing and analytics.

In a Supply Chain Analytics and DevOps role, I worked on Short-Term and Long-Term Demand Planning (STDP & LTDP) for the Australian market, forecasting demand using Python and Azure Databricks. I also provided technical and functional support for a three-tier architecture Demand Forecasting Tool, where I identified new KPIs through exploratory data analysis, improving tool effectiveness—recognized with a Spot Award for exceptional contribution.

Additionally, I was involved in production deployment, handling daily CI/CD operations using Azure DevOps, deploying Azure Data Factory pipelines, fixing bugs, maintaining code quality, and ensuring adherence to version control and review standards.