SaiRam

Client - HSBC

• Worked as a GCP Data Engineer on the HSBC account, mainly involved in developing and supporting data pipelines and ETL processes.
• Contributed to PEGA projects, working on data integration and processing requirements based on different business needs.
• Worked with SCD Type 1 and Type 2 approaches to manage changes in dimensional data and maintain historical records where required.
• Prepared and implemented Source-to-Target Mapping (STM) by understanding source data, defining transformation rules, and mapping the required fields to target tables.
• Built and maintained ETL workflows using Apache Airflow and BigQuery on GCP, handling pipeline scheduling, data processing, monitoring, and troubleshooting.
• Worked with Jenkins for CI/CD activities, including code deployment and promoting pipeline changes across different environments.
• Carried out data checks and reconciliation between source and target systems to identify data mismatches and ensure the expected data was loaded correctly.

• Helped set up a private JupyterLab server and made it accessible through a Cloud Run reverse proxy using Flask, and VPC connectors.
• Supported the team in creating and scheduling basic data pipelines using Cloud Composer for tasks involving BigQuery, Dataflow, and other relevant technologies.
• Worked on simple DAGs to load and transform data from external sources into BigQuery.
• Assisted in configuring Pub/Sub and Dataflow for real-time data movement from APIs to BigQuery.
• Assisted in creating basic monitoring dashboards in Cloud Monitoring to check pipeline status and errors.
• Explored Looker Studio to build basic dashboards using BigQuery datasets for visualising data trends and KPIs.
• Observed and learned about tools such as Dataform and Workload Identity Federation during project discussions and team activities.

• Designed and managed hybrid data pipelines across GCP (BigQuery, Pub/Sub, Dataflow, Cloud Storage) and Azure (ADF, Synapse, ADLS, Databricks), enabling scalable batch and streaming data solutions.
• Developed PySpark transformations for both Dataflow and Databricks, optimizing performance for high-volume, near real-time workloads.
• Migrated structured and unstructured datasets from on-prem into BigQuery and Synapse, improving query efficiency, scalability, and cost management.
• Automated infrastructure deployments using Terraform and implemented CI/CD pipelines with Azure DevOps + Git, ensuring secure and repeatable releases.
• Implemented monitoring & observability using GCP Cloud Monitoring/Logging and Azure Monitor, providing proactive alerts and pipeline health dashboards.
• Built data models, partitioning, and clustering strategies in BigQuery, reducing query cost and accelerating performance for analytics workloads.
• Developed event-driven ingestion pipelines using Pub/Sub → Dataflow → BigQuery, powering real-time analytics for business KPIs.
• Secured pipelines with Secret Manager (GCP) and Key Vault (Azure), enforcing compliance with enterprise data security standards.
• Designed reusable ETL components (logging, error handling, parameterization) to streamline pipeline development and reduce defects.
• Collaborated with business analysts and stakeholders to translate business requirements into scalable BigQuery schemas and transformations.
• Conducted code reviews, CI/CD validation, and release readiness checks, ensuring high-quality deployments and adherence to coding standards.
• Delivered training and mentorship sessions for junior engineers on GCP BigQuery, Dataflow, and hybrid pipeline best practices.

• Implemented measures to ensure all transactions were secure, trusted, and fraud-free.
• Utilized advanced fraud detection systems and robust security protocols to protect customer data.
• Reviewed and examined customer orders to identify and flag potential fraudulent activities.
• Conducted thorough investigations into suspicious transactions to mitigate risks and reduce financial losses.
• Generated and filed detailed investigation reports for senior management, highlighting trends and progress.
• Inspected products and services to ensure compliance with internal quality and regulatory standards.
• Applied control measures to uphold product/service excellence and maintain high customer satisfaction.
• Continuously monitored, tracked, and managed risk across order and service pipelines.
• Protected operational integrity and ensured long-term customer trust and satisfaction.

• Assisted in building foundational BigQuery datasets to support internal IT operations, reporting, and analytics.
• Created SQL-based queries in BigQuery to analyse system logs, backup statuses, and endpoint activity trends.
• Supported the setup of real-time ingestion pipelines using Pub/Sub and Dataflow, loading data into BigQuery tables.
• Helped optimise query performance by applying partitioning and clustering strategies under senior team guidance.
• Participated in the integration of BigQuery data with reporting tools such as Looker Studio for monitoring dashboards.
• Gained hands-on experience with GCP IAM, roles, and billing configurations for BigQuery projects.
• Assisted in deploying Java-based Spring Boot microservices using Cloud Run, integrated with BigQuery and Firestore for real-time analytics and storage.
• Supported the development and management of RESTful APIs through Apigee, including route configuration, proxy setup, and access control policies.
• Configured Cloud Scheduler to automate backend data sync tasks and trigger Cloud Functions for scheduled processing workflows.
• Worked with Firestore to perform read/write operations on metadata collections used by distributed applications and dashboards.
• Helped manage sensitive service credentials using Secret Manager, ensuring secure authentication across GCP services.
• Gained hands-on exposure to configuring Cloud Load Balancer, DNS records, and security rules for scalable microservice deployments.