Summary

Overview

Work History

Education

Skills

Accomplishments

Certification

Timeline

ZUBIN BAL

Senior Data Engineeer

Angul

Summary

Senior Data engineering professional with deep expertise in data architecture, pipeline development, and big data technologies. Proven track record in optimizing data workflows, enhancing system efficiency, and driving business intelligence initiatives.Skilled in Snowflake, SQL, Python, Spark, and cloud platforms, with strategic approach to data management and problem-solving.

Overview

years of professional experience

Certifications

Work History

Sr. Data Engineer

Accenture

11.2023 - Current

Developed and optimized Snowflake data pipelines to handle high-volume, real-time data loads using Snowpipes for 800+ tables, ensuring minimal latency, and efficient data processing.
Implemented automated data orchestration workflows using Prefect,
Automated, robust monitoring, scheduling, and error handling for end-to-end data pipeline management using Prefect.
Designed complex pipelines in Prefect by enabling job dependencies to accurately populate the accurate results in the reconciliation table for streaming data pipelines.
Innovative solutions were provided that cut ETL pipeline run time and cost by 90%.
Leveraged the Snowpark API, Snowpipes, Streams, Dynamic Tables and PySpark to build custom transformation logic and stored procedures.
Built test scripts for automated monitoring of table counts for every batch load.
Provided automated solutions to handle data type mismatches and schema evolution, ensuring smooth integration and consistency between source data and target Snowflake tables.
Used AWS S3, AWS Glue, Snowflake services, AWS Step Functions, and Lambda functions for data migration.

Data Modelling / Engineer

Motorola Solutions

01.2021 - Current

Involved in Data Modelling Projects using ERWIN Tool for Telecom Domain
Cleaned, transformed, and manipulated raw data CDR from upstream to prepare it for analysis in downstream using PySpark and SQL for Billing purposes resulting in improved data quality and integrity.
Development of various VOIP Traffic testing Models for PERFORMANCE TESTING using SIPp Tool
AUTOMATION TESTING on NodeJs framework and New Test Suite Creations and Framework Development

Education

Master of Engineering - Manufacturing Systems

Birla Institute of Technology And Science, Pilani

Pilani, India

04.2001 -

Bachelor of Technology - Mechanical Enginerring

Government College of Engineering, Keonjhar

Odisha

06-2016

Skills

Snowflake Data Services: Snowpark,Streams,Snowpipes,Dynamic Tables

AWS Cloud Services: S3, Glue, AWS Lambda, StepFunctions

Python programming: Pyspark

Data pipeline design: Prefect Orchestration tool

Machine learning: Snowflake Cortex AI

ETL development

Git version control

Performance tuning

Data modeling

NoSQL databases

Big data processing

Data quality assurance

Real-time analytics

Spark development

Metadata management

Data migration

Database design

Data visualization

OS: Linux

Data governance

Analytical thinking