Domain: BFSI Client: Bank of America Duration:3 years
Processed trading application data in CSV, JSON, and Parquet formats.
Our upstream team loads files into AWS S3 bucket.
Set up Snowflake Snowpipe for automated ingestion, copied ARN from Snowpipe properties, and mapped notification channel to SQS queue in S3 event notification settings.
Wrote Python script to split large files and load data into Snowflake.
Used dbt to transform data for SCD Type 1 with incremental model and merge strategy, built reusable macros, loaded small CSV files with seeds, and implemented SCD Type 2 with snapshots.
Ran validation through singular tests and custom tests.
Scheduled jobs with Apache Airflow, processed PDF files with Cortex AI functions such as AI Complete, Summarize, Sentiment, and Parse Document, applied RBAC and RLS for security and compliance.
Collaborated with business analyst team and client.
Previous Project:
Domain: Healthcare Client: United Health Group Duration:2.5 years
Migrated data from Oracle and SQL Server into Snowflake.
Used Talend ETL tool to move data directly into target platform.
Applied transformation logic and business rules in Snowflake through procedures and functions.
Used Streams and Tasks for scheduling and CDC data capture.
Used Time Travel to retrieve data and review historical records.
Used Zero Copy Cloning during unit testing.
Applied RBAC and Dynamic Data Masking for security and compliance.