Client – Evergent (Subscription & Revenue Management Platform)
Role: Data Engineer
Tools/Tech: Google Cloud Platform (BigQuery, DataStream, Composer/Airflow, STS), MySQL, MS SQL Server, SSIS
- Led end-to-end data migration from on-premise MySQL and MS SQL Server databases to Google BigQuery, ensuring minimal downtime and full data integrity.
- Utilized GCP STS to perform historical data loads and DataStream for real-time incremental replication.
- Translated complex SSIS packages and stored procedures into BigQuery SQL, orchestrating their execution using Apache Airflow (Composer).
- Achieved a 60% reduction in data processing time, significantly improving pipeline efficiency and enabling faster analytics and reporting across business teams.
Client – Indigo Airlines
Role: Data Engineer
Tools/Tech: Azure Data Factory, Snowflake, Azure Blob Storage, SQL
- Developed a data pipeline using Azure Data Factory to ingest data from third-party sources into Azure Blob Storage efficiently.
- Created Snowflake stored procedures to transform and load data into dimensional and fact tables, enhancing reporting accuracy to 99%.
- Streamlined and automated data integration, enabling consistent, timely data availability for BI/reporting use cases.
Client – Jubilant Foodworks Limited
Role: Data Engineer
Tools/Tech: Snowflake, Amazon Redshift, PostgreSQL, Python, Kafka
- Built a robust ELT pipeline using Snowflake to extract data from Amazon S3, perform complex transformations, and load into Snowflake for analytics and reporting.
- Designed and developed KPI tables by transforming customer feedback data, enabling data-driven performance tracking.
- Developed a Python-based reconciliation module to identify and reprocess missed data into snowflake, achieving 99% data accuracy.
- Optimized the SMS reporting module using transformed KPI data, resulting in a 50% reduction in report generation time.
Client – Fantasy Gaming Organization (MPL)
Role: Data Engineer
Tools/Tech: Databricks, GCP (BigQuery, Dataproc), Spark SQL, Python
- Migrated the entire ETL process, data warehouse, data analytics pipelines, and 1000+ Data Science notebooks from Databricks to Google Cloud Platform (BigQuery and Dataproc), achieving a 40% cost reduction.
- Designed and automated the conversion of Spark SQL to BigQuery-compatible SQL, accelerating the migration process.
- Contributed to one of the largest cloud migration projects to GCP from another data platform.