

GCP Certified Professional Data Engineer with expertise in designing and implementing end-to-end data pipelines on GCP Cloud. Contributed to multiple data pipeline projects across diverse teams while effectively engaging with UK clients in client-facing roles. Emphasized Agile and Scrum methodologies to enhance project delivery and collaboration.
Working on enterprise-scale Teradata to GCP migration, transforming complex legacy SQL workloads into optimized BigQuery-based pipelines., Migrating complex Teradata queries into optimized BigQuery SQL, Designing config-driven pipeline frameworks to automate workflows, Developing and maintaining Airflow (Composer) DAGs, Implementing Python-based logic for state-specific report generation, Standardizing schemas using Teradata-to-GCP column mapping, Creating technical specifications and architecture/design documents, Handling data encryption & decryption during transformation and loading, Developing reusable views and configuration files, Managing code using GitHub (SQL & Python version control), Ensuring data quality, validation, and consistency, Supporting end-to-end pipeline development and deployment
Migration of data from On-prem hive system to GCP using infoworks tools., Working on migration of on-prem data from Hive to Dataproc cluster using infoworks replicator., Pulling data from Dataproc cluster to GCP using infoworks ingestion tools., As per business requirement doing encryption and decryption on hive(on-prem)data before migrating to GCP., Doing data validation between on-prem data and GCP data using VM and Python., Jobs orchestration using Tidal tool.
Building Analytics over GCP using Bigquery, GCS, Composer. Also Migration from Netezza to Bigquery., Development of end to end data mart from Development to Production Deployment., Semi-structured data (JSON) loading from GCS to BQ., SQL Scripts Development from created STM., Creation of Composer dags in python., Written generic wrapper scripts to load data from GCS bucket to BQ tables., Creation of Confluence (analysis) document according to the business requirements., Creation of builds using GitHub.
Migration from On-premises Hadoop system to GCP making it platform agnostic with configuration and code base isolation and security enhancements., Worked over 2-3 subject areas for migrating the modules from Hadoop to GCP., Worked on Data Acquisition of hive tables into GCS bucket for multiple modules., Developed BQ scripts from conversion of HQL Scripts with further optimisation., Creation of Composer Dags in python., Data validation as per the test cases and creating unit testing documents.
Migration from Teradata to Bigquery and build reports over Tableau as per Business needs using Agile methodology., Ingestion of about 750 tables with different loading strategies. (Full Load, Incremental)., Write SQL scripts with reference to the Confluence Page and Load Final tables., Creation of Composer dags in python., Data validation as per the test cases and creating unit testing documents.