Ganesh Gouda

Associate ETL Developer

MResult

1 2023 - Current

Utilized AWS Glue to implement a latency alert framework, enabling real-time monitoring of data replication pipelines. Designed automated email alerts to promptly notify stakeholders of any latency issues, ensuring timely resolution and maintaining data replication efficiency.
Developed and implemented AWS Glue ETL scripts to extract data from various sources, load it into S3, and seamlessly ingest into Snowflake data warehouse. Enhanced data integrity and synchronization by incorporating robust functionality to handle deletion and update operations within Glue codebase.
Developed IICS(Informatica Intelligent Cloud Services) mass ingestion CDC jobs to replicate over 120+ tables to snowflake data warehouse in near real time.
Identified and fixed root cause for latency in IICS mass ingestion pipelines and improved replication latency by 96%.
Developed python script to automate Snowflake CICD deployment through GitHub workflows.
Designed and developed various IICS mapping tasks and dynamic mapping tasks to extract the incremental/batch data from multiple source and load it into snowflake .
Managed and coordinated platform upgrades for IICS and HVR, ensuring seamless transitions through effective coordination and planning.
Utilized Snowflake features such as Snowpipe, Tasks, and Streams across multiple use cases, enhancing data processing efficiency and enabling real-time data ingestion and automation.
Managed monthly Snowflake housekeeping, including storage metrics analysis, query tracking across accounts, identification of admin-level users, monitoring of user activity and login methods, ensuring optimal performance and security compliance.
Managed access requests to Snowflake databases, ensuring proper permissions and security protocols were followed.
Created snowflake roles and objects, and assigned proper privileges to ensure secure and efficient data access.
Automated daily Informatica Processing Unit(IPU) usage reports through AWS Glue.