Project : Automated Ingestion and Transformation using Snowflake for Investment Banking
Domain : BFSI
Environment : AWS S3,Snowflake DB
- Implementation of scalable Snowflake data warehousing solutions, enhancing efficiency in data management processes.
- Demonstrated proficiency in developing a comprehensive array of Snowflake objects, encompassing warehouses, schemas, databases, tables, views, pipes, stages ensuring comprehensive data governance.
- Managed structured data migration with meticulous attention to detail, overseeing stages, copy operations, data transformations, error handling protocols, and employing a wide range of copy options for enhanced precision.
- Implemented incremental loads to enhance performance of large scale data loads.
- Loaded unstructured data efficiently using VARIANT datatype, flattening methods, and innovative techniques to improve data accessibility.
- Create and enhance sophisticated SQL queries to bolster data analysis and elevate business intelligence capabilities.
- Developed Snowflake procedures, functions, and User Defined Table Functions (UDTF), automating operations and improving capabilities for streamlined data manipulation.
_____________________________________________________________________
Project : Financial Data Warehouse Modernization Using Snowflake, DBT.
Domain : BFSI
Environment : DBT, Snowflake DB
- Designed and developed modular dbt models for staging, dimension, and fact tables using Snowflake as the backend.
- Implemented incremental models with strategies to optimize large-scale data loads. Utilized dbt tests and snapshots to ensure data quality and track historical changes.
- Created reusable macros and utilized Jinja templating to streamline development, while maintaining clear model-level documentation in YAML.
- Handled version control using GitHub - created branches, committed changes, raised Pull Requests (PRs), and merged into main.
- Managed multiple environments using dbt Cloud. scheduled production runs with email alerting.
- Collaborated with cross-functional teams to gather requirements, improve pipeline performance, and deliver clean, trusted data.