Shadan Musarrat

Technologies: AWS, Microsoft Fabric, Power BI, OneLake, Semantic Models, Data Lakehouse

Project Description: Led the integration of AWS data sources into Microsoft Fabric's OneLake platform for the creation of a Lakehouse architecture. The project aimed at centralizing and optimizing data storage, management, and visualization to support business decision-making processes through scalable data solutions.

Key Responsibilities:

• Data Integration: Migrated and integrated existing data from AWS services such as S3, into Microsoft Fabric's OneLake to establish a unified data lakehouse.
• Lakehouse Architecture: Designed and implemented a Lakehouse model in OneLake.
• Semantic Models Development: Created semantic models using Microsoft Fabric.
• Data Visualization: Developed interactive and insightful dashboards using Power BI to visualize the integrated data.

• Accomplished data analytics designing, implementing, and optimizing data pipelines and ETL processes on Amazon Web Services (AWS) platform.
• Proficient in leveraging AWS services such as AWS Glue, Amazon EMR, Amazon Redshift, and Amazon S3 to build scalable and reliable data solutions for diverse business requirements.
• Skilled in data modeling, schema design, and performance tuning to ensure efficient data storage, retrieval, and analysis in AWS cloud environments.
• Experienced in developing and deploying data processing workflows using Apache Spark, PySpark, SQL, and Python, enabling real-time and batch data processing capabilities.

• Spearheaded ETL projects on Google Cloud Platform (GCP), leveraging Data Proc, BigQuery, and Data Lake services to extract, transform, and load large volumes of data for analysis and decision-making.
• Designed and implemented data pipelines using GCP services, optimizing data processing workflows for scalability, performance, and cost-effectiveness.
• Utilized Power BI for intuitive and interactive reporting, enabling stakeholders to visualize insights derived from GCP data sources and drive data-driven business decisions.
• Collaborated closely with stakeholders to gather requirements, design data models, and develop dashboards and reports tailored to specific business needs and objectives.

• End-to-End migration of on-premises infrastructure to AWS cloud.
• Develop ETL Process on AWS cloud in order to replace legacy processes.
• Orchestrated seamless transition of critical workloads to AWS, ensuring high availability, scalability, and security in cloud environments.
• Implemented AWS migration tools and methodologies to rehost, refactor, and replatform applications, maximizing performance.