VIGNESH C
Bangalore
Summary
- Data Engineer with 5.11 years of experience in Data Engineering and ELT, with expertise in Snowflake, DBT, SQL, Python, AWS and Power BI.
- Experienced in Snowflake data warehouse development, working with databases, schemas, tables, views, warehouses, stages, SnowSQL and data loading processes.
- Hands-on experience in AWS S3 to Snowflake data ingestion using external stages, COPY INTO and Snowpipe for continuous data loading.
- Experienced in Snowflake Streams and Tasks for incremental data processing, CDC workflows and task dependency-based data pipelines.
- Good experience with DBT for transforming raw database data into clean, structured and analytics-ready datasets, including developing and maintaining transformation models.
- Strong SQL expertise with joins, subqueries, CTEs, aggregations, window functions, CASE expressions and transformation logic for data processing and validation.
- Experienced in handling structured and semi-structured data, including JSON, VARIANT and FLATTEN, along with ETL,ELT and data migration processes.
- Strong experience in source-to-target validation, data reconciliation, data quality analysis and data warehouse validation across heterogeneous data platforms.
- Proficient in Python for data analysis and validation, with experience collaborating with Data Engineers, Developers, Business Analysts and stakeholders to troubleshoot data issues and deliver reliable data solutions.
Overview
1
1
Language
6
6
years of professional experience
Work History
Technical support Engineer
Cimpress India Pvt Ltd
Bangalore
10.2024 - 08.2026
- Developed and maintained ELT data pipelines across source, staging and target data layers to support reliable data movement and transformation.
- Developed and optimized SQL queries using joins, subqueries, CTEs, aggregations and window functions for data transformation and analysis.
- Implemented and validated source-to-target data transformations, business rules, data mappings and transformation logic.
- Worked with AWS S3-based data ingestion workflows, managing data movement from cloud storage into downstream data processing layers.
- Developed and maintained DBT transformation models with reusable business logic and source-to-target mappings.
- Developed data validation and reconciliation logic using SQL to ensure data completeness, accuracy and consistency.
- Used Python and Pandas for data processing, data analysis and reconciliation across heterogeneous data sources.
- Developed and maintained datasets supporting Power BI dashboards and reports, including KPI calculations, business logic, filters and reporting requirements.
- Performed data profiling, quality checks, duplicate detection, null validation and record-level reconciliation across large datasets.
- Troubleshot data pipeline issues by analyzing source data, transformation logic and target results, and implemented fixes to resolve data discrepancies.
- Project 1 — Insurance | Boltech, USA
software Engineer
Index Info systems
Chennai
10.2023 - 10.2024
- Developed and maintained Snowflake data warehouse objects, including databases, schemas, tables, views, stages and virtual warehouses.
- Designed and implemented AWS S3 to Snowflake data ingestion pipelines using external stages, file formats and COPY INTO.
- Implemented Snowpipe for automated and continuous ingestion of files from AWS S3 into Snowflake.
- Developed Snowflake Streams and Tasks to implement incremental data processing and change data capture (CDC) pipelines.
- Built task dependencies and task-based ETL workflows to process incremental data and populate downstream tables.
- Developed transformations for structured and semi-structured data using JSON, VARIANT and FLATTEN.
- Created secure views and materialized views to serve optimized and controlled datasets to downstream reporting applications.
- Utilized Time Travel and Zero-Copy Cloning for data recovery, development, backup and environment provisioning.
- Developed complex Snowflake SQL using CTEs, joins, subqueries, aggregations, CASE expressions, window functions and analytical functions.
- Developed data transformation and reconciliation processes across SQL Server, AWS S3 and Snowflake.
- Used Python and Pandas for data processing, file handling, data transformation and reconciliation of heterogeneous data sources.
- Project 2
- Data Engineer — Investment Banking | BDA Partners, Canada — 10/2023 to 10/2024
Data Engineer
Wissen Technology
01.2023 - 09.2023
- Developed and maintained ELT workflows using SSIS for extracting data from Oracle and SQL Server and loading transformed data into target data warehouse layers.
- Developed complex SQL and PL/SQL programs using joins, subqueries, CTEs, aggregations and analytical functions.
- Designed and implemented source-to-target transformation logic based on business requirements and technical specifications.
- Developed SQL procedures and transformation logic for data extraction, cleansing, transformation and loading.
- Implemented data reconciliation and data-quality processes to ensure completeness, accuracy and consistency across source and target systems.
- Developed and maintained datasets consumed by Power BI for analytical reporting and business insights.
- Optimized SQL queries and Snowflake workloads by analyzing query execution behaviour and improving transformation logic.
- Project 3
- Software Engineer — —Healthcare | Global Care, UAE
Software Engineer
Zifo Data
11.2021 - 01.2023
- Developed reusable SQL queries and stored procedures for data processing and analytical requirements.
- Worked with large datasets and performed data profiling, data cleansing and transformation using SQL and PL/SQL.
- Supported data migration activities across Oracle and SQL Server environments.
- Developed datasets and data structures supporting Tableau reporting and analytics.
- Analyzed ETL workflow performance and optimized SQL queries and transformation logic to improve processing efficiency.
- Maintained technical documentation covering ETL workflows, data mappings, transformation rules and data-processing logic.
- Worked within an Agile/Scrum environment to deliver data engineering requirements across project releases.
- Project 4
- Data Engineer — Investment Banking | TSB Bank, UK
Software Trainee
Persistent systems
12.2020 - 10.2021
- Migrated data from Oracle to Snowflake using flat files and staged data loading processes, ensuring accurate and reliable data migration.
- Loaded structured and semi-structured data (CSV and JSON) from AWS S3 into Snowflake using COPY INTO, PUT, GET, and internal and external stages.
- Developed and managed Snowflake Streams to capture DML changes and support Change Data Capture (CDC) and audit tracking.
- Optimized SQL queries and Snowflake pipelines through join optimization, predicate filtering, and query pruning to improve performance.
- Created SQL queries, views, and stored procedures to implement business logic and support data transformation requirements.
- Project 5
- Data Engineer — Investment Banking | BDA Partners, Canada
Education
Master of Arts - Computer Application
Muthayammal College of Arts Science, Periyar University
Salem, TN2017
Skills
- Snowflake
- Database, Schema, Tables, Views, Warehouses, Internal and External Stages, SnowSQL, COPY INTO, Snowpipe, Streams, Tasks, Task Trees, Time Travel, Fail-safe, Zero-Copy Cloning, Secure Views, Materialized Views, VARIANT, JSON, FLATTEN, CDC, Data Sharing, CortexAI
- Cloud
- AWS S3, AWS IAM, Cloud Data Ingestion, Cloud Data Migration, IAM, SNS, Athena, Glue, Lambda
- Data Warehousing
- Fact & Dimension Tables, Star Schema, Snowflake Schema, Data Marts, Transformation Validation
- Programming and Scripting
- Python, PL/SQL, SQL
- Data Engineering
- ETL and ELT, Glue, Talend, Data Migration, Source-to-Target Mapping, Full and Incremental Load, Data Quality, Data Reconciliation
- BI & Tools
- Power BI, Git, Jira, SQL Developer, Postman, ServiceNow, DBT, Kafka, Control-M, Apache Airflow, Jenkins
LANGUAGES
English | Tamil
Timeline
Technical support Engineer
Cimpress India Pvt Ltd
10.2024 - 08.2026
software Engineer
Index Info systems
10.2023 - 10.2024
Data Engineer
Wissen Technology
01.2023 - 09.2023
Software Engineer
Zifo Data
11.2021 - 01.2023
Software Trainee
Persistent systems
12.2020 - 10.2021
Master of Arts - Computer Application
Muthayammal College of Arts Science, Periyar University