Summary
Overview
Work History
Education
Skills
Timeline
Generic

Sakthivadivel Kalaivanan

GCP Data Engineer
Cuddalore

Summary

6.3 years of GCP Data Engineer experience in architecture, analysis, design, development, implementation, maintenance, support and responsible for increasing the availability of data in the enterprise by developing and maintaining a large variety of software and hardware components for storing, processing, analyzing, building data pipelines for Cloud Migration. I have streamlined the import of data from various sources into the Big Query warehouse, enhancing the quality of data insights and improving access to data sources. I have created airflow python operator and python application to onboard the data as business required. I have automated the deployment via GitHub into the production environment, reducing development time by 20%, and optimized data pipelines to reduce costs by 20% while ensuring data integrity and accuracy.

Overview

2025
2025
years of professional experience
4
4
years of post-secondary education
2
2
Languages

Work History

Staff Engineer

Altimetrik
10.2024 - Current

Senior Consultant/GCP Data Engineer

Capgemini
8 2021 - 09.2024
  • Analyzing and getting the requirements for the development
  • Creation, Enhance and support of Airflow python operator as per business required
  • Created Google Cloud function on Python program
  • Created airflow Dag's to move data from various sources into the GCS bucket, such as Snowflake, Azure Blob Storage, S3, social media platforms, and client API endpoint
  • Created GCS and Secret Id based on development needs via terraform
  • Worked on the cloud monitoring to check the status of the composer instance
  • Created Big Query tables via terraform scripts (External, Native, and Partition)
  • Loading data into Big Query tables from the GCS via airflow Dag's
  • RCA and fix by analyzing the logs from the cloud logging if the prod jobs fail after getting the necessary approval
  • New enhancement on existing pipeline and airflow python operator
  • End-to-end data and airflow pipeline workflow validation
  • Product deployment pre-work (PR creation and baseline), post-work and support

Data enthusiast / Junior Data Engineer

INNIVE BUSINESS SOLUTION
08.2018 - 08.2021
  • Created Big Query tables via BQ commands
  • Created python application to schedule, transform and onboard the data into GCS or Big Query
  • Created KPI metric query
  • Data load in the information layer table
  • Created Big Query view and materialized view
  • Created DBT models and macros templates for KPIs
  • End-to-end data validation with all test cases
  • Airflow Dag's support and monitoring
  • Data integration via JavaScript in the UI components
  • NBC Universal/Global and International Alchemy
  • Alchemy is a Film-analytics solution
  • Collecting data from different social media platforms and Client Servers (Client API, Snowflake, Azure blob storage, gSheet, and S3) and generate information and their insights data before the film release
  • Clay County and LAUSD (Los Angeles Unified School District)/Remote Learning
  • It's an online student studies data for the data analytics solution of education that aggregates data from various district systems, transforming it into clear and concise analysis that educational leaders can access on-demand via user-friendly, intuitive visualizations

Education

Bachelor of Engineering - Electrical and Electronics Engineering

Agni College of Technology
08.2012 - 11.2016

Skills

Python

GCP (GBQ, Composer Airflow, GCS, GCF, Secret manager, cloud logging & monitoring, cloud build, cloud source repo, IAM, VM, Analytic Hub)

Snowflake

Azure blob Storage

GitHub

Terraform

Bash Script

DBT

API Postman

Timeline

Staff Engineer

Altimetrik
10.2024 - Current

Data enthusiast / Junior Data Engineer

INNIVE BUSINESS SOLUTION
08.2018 - 08.2021

Bachelor of Engineering - Electrical and Electronics Engineering

Agni College of Technology
08.2012 - 11.2016

Senior Consultant/GCP Data Engineer

Capgemini
8 2021 - 09.2024

Similar Profiles

CREATE PROFILE
Sakthivadivel KalaivananGCP Data Engineer