Muthu Veera Pandi
Senior Technical Lead
Bangalore
Summary
Data Engineering professional with 9.5+ years of experience in designing and implementing ELT/ETL frameworks using DBT, Datastage, and Snowflake. Proven expertise in DBT Cloud for developing models, macros, incremental loads, and snapshots integrated with Snowflake to build scalable, high-performance data transformation pipelines. Experienced in leading Oracle, MSSQL, and file-based migrations to Snowflake, ensuring efficient data movement and transformation while leveraging AI assistants such as Copilot to accelerate development and reduce defects.
Strong understanding of Snowflake architecture, including virtual warehouses, clustering keys, query performance tuning, and scaling policies. Skilled in handling complex semi-structured datasets (JSON, CSV) from AWS S3 and optimizing transformation logic using CTEs, parallel processing techniques, and performance-tuning best practices.
Hands-on experience in automating ELT/ETL workflows, developing Control-M job schedules, and integrating GitHub-based CI/CD pipelines for DBT deployments. Adept in SQL optimization, data model design, and production support with a solid track record of quick defect resolution, automation of repetitive tasks using scripting and Copilot-driven code generation, and continuous performance improvement of data pipelines.
Proven leadership in managing and mentoring teams of up to 10 members, coaching new joiners on DBT, Datastage, and Snowflake, and contributing to continuous process improvement within enterprise data environments. Collaborative and proactive in business requirement analysis, technical design, and end-to-end delivery, with a focus on using automation and AI-assisted development to improve quality, speed, and reliability.
Overview
9
9
years of professional experience
8
8
Certifications
2
2
Languages
Work History
Senior Technical Lead
HCL Tech
12.2023 - Current
- Designed, developed, and maintained DBT models, macros, tests, and snapshots to build scalable ELT pipelines on Snowflake, aligned with business requirements.
- Implemented DBT incremental models and snapshot strategies for large fact and history tables, significantly reducing load times and warehouse usage.
- Converted complex Datastage jobs and mappings into modular DBT models, ensuring functional parity, better maintainability, and CI/CD readiness.
- Built reusable DBT frameworks and macros to standardize transformation patterns, data quality checks, and naming conventions across the project.
- Optimized Snowflake queries using clustering keys, CTE refactoring, micro-partition pruning, and result caching to improve performance and control costs.
- Configured and tuned Snowflake virtual warehouses, including sizing, multi-cluster scaling policies, and auto-suspend settings for efficient resource usage.
- Designed multi-layer Snowflake data models (staging, core, and data marts) to support analytics, operational reporting, and downstream consumption.
- Performed advanced SQL performance tuning for complex transformations, including rewrites of heavy queries, and optimization of joins and aggregations.
- Led end-to-end migration of on-premises Oracle and MS SQL Server workloads to Snowflake, covering schema conversion, data mapping, validation, and cutover.
- Migrated file-based workloads (CSV, JSON, and other flat files) from on-premises and AWS S3 into Snowflake using DBT, Snowflake utilities, and scripting.
- Automated ELT/ETL workflows using Datastage, DBT, and Control-M scheduling enable reliable batch processing with minimal manual intervention.
- Developed and managed Control-M job schedules for end-to-end batch flows, dependency handling, failure alerts, and restart procedures.
- Wrote robust SQL scripts for data transformation, SCD logic, reconciliation, backfills, and housekeeping operations in Snowflake and MS SQL Server.
- Processed and transformed large semi-structured datasets (JSON, CSV) from AWS S3 into Snowflake, applying schema inference and complex business rules.
- Utilized GitHub for version control, branching strategies, and code reviews for DBT, Datastage, and migration artifacts as part of enterprise DevOps practices.
- Implemented comprehensive data quality checks using DBT tests and SQL validation queries to ensure accuracy, completeness, and consistency of data.
- Created and maintained detailed technical documentation, design specifications, and runbooks to support operations, audits, and knowledge transfer.
- Managed a 10-member team delivering Oracle, MSSQL, and file-to-Snowflake migration projects, overseeing workload distribution, delivery timelines, and code quality.
- Gathered and analyzed requirements from business users and vendors, translated them into technical designs, and presented solution approaches to stakeholders.
- Mentored freshers and new joiners on DBT, Datastage, Snowflake, project standards, and best practices through KT sessions and hands-on guidance.
Senior Data Engineer
LTIMindtree Pvt Ltd
11.2021 - 12.2023
- Extensive ETL tool experience using IBM infosphere/Websphere DataStage.
- Worked on DataStage tools like DataStage Designer, DataStage Director and DataStage Administrator.
- Strong understanding of the principles of Data Warehousing using fact tables, dimension tables and star/snowflake schoma modeling.
- Worked extensively with Data migration, ETL Processes for data warehouses.
- Developed paralel jobs using different processing stages like Transformer, Aggregator, Lookup. Join, Sort, Copy, Merge, Funnel.
- CDC, Change Apply and Filter.
- Familiar in using highly scalable parallel processing infrastructure using parallel jobs and multiple node configuration files.
- Experienced in scheduling Sequence and parallel jobs using DataStage Director, UNIX scripts and scheduling tools.
- Exporionco in troubleshooting of jobs and addressing production Issues like data issues, ENV issues, performance tuning and enhancements.
- Extensive experience in Unit Tosting. Functional Tosting, Systom Tosting. Integration Tosting. Regression Testing. User Acceptance Testing (UAT) and Performance Testing.
- Worked with various databases like Oracle, DB2, SQL Server.
Associate Consultant
Capgemini Pvt Ltd
11.2020 - 10.2021
- Developed paralel jobs using different processing stages like Transformer, Aggregator, Lookup, Join, Sort, Copy, Marge, Funnel.
- Worked on DataStage tools like DataStage Designer & DataStage Director.
- Experience in creating Sequence job.
- Proporing Reports for Pre-production & Production job.
- Worked on SQL Scripts for Table creation and Truncation.
- Involved in Defect fixing for QAT.
- Worked on Improvement Jiros created depends on Requirement.
- Involved in unit Testing.
- Conducting Some technical training session to juniors every month related to the project.
Associate Consultant
Capgemini Pvt Ltd
08.2019 - 11.2020
- Verifying that data from source staging is transformed and loaded correctly to intermediato schoma and is in accordance with various business requirement and rulos.
- Ensuring that all projected data is loaded into Intermediato schoma without any data loss and truncation.
- Exclusively used SQL queries for data validation.
- Evaluated all requirements of business systems and developed test plans accordingly.
- Allocation of task to team members.
- Environment setup and coordinating with Dev team and onshore.
- Documented and reviewed all test cases based on ETL Specification Document.
- Raising defects where discrepancy found in the transformation logic and reporting it to the development toom on V878.
- Grooming the new joiners, laterals to make thom familiar with the project work.
- Maintaining daily status reports with complete information.
Associate Technical Operations
IBM India Pvt Ltd
01.2017 - 06.2019
- Involved in creating the Low level design document.
- Involving in extracting the data from Tables.
- Involved in development of Data stage jobs and sequence jobs.
- Extensively used stages like Lookup. Join Transfomer, Aggregator& Funnel.
- Involved in creating Unit testing case document. Monitoring job log in data stage director. Involved in Defect fixing for QAT.
- Monitoring of TWS jobs on timely basis with respect to informatica.
- Monitor the data and activities inside the data warehouse Administer the data warehousing environment, including security and performance.
- Technical administration of Micro strategy Monitoring of file systems on AIX servers.
- Monitoring of SAP Jobs and SAP print jobs. Know Basic things in SAP (To check error logs, Printer Issue Post checks of mass transport changes & Unlock users).
- Provide first level technical support on all applications used by the business Field.
Education
BE - ELECTRICAL AND ELECTRONICS ENGINEERING
K.L.N. College of Engineering
Madurai06.2016
DIPLOMA - ELECTRICAL & ELECTRONICS ENGINEERING
Tamilnadu Polytechnic College
Madurai10.2012
SSLC - undefined
Gurukulam High School
Madurai06.2009
Skills
DBT
Certification
Certified SAFe 5 Practitioner - Scaled Agile, 2023-01-01
Tools Used
- DBT
- Snowflake
- Fivetran
- Python
- Gen-AI Copilot
- Control-M
- Autosys
- InfoSphere Datastage
- SQL Server
- Oracle
- WinSCP
- Azure DevOps
- Informatica
Timeline
SnowPro Advanced: Data Engineer
12-2025
SnowPro Advanced: Data Scientist
12-2025
SnowPro Advanced: Architect
01-2025
SnowPro Core COF-C02 certification, 2023-01-01
11-2024
Certified SAFe 5 Practitioner - Scaled Agile, 2023-01-01
05-2024
CLF-C02 - AWS Certified Cloud Practitioner, 2023-01-01
05-2024
Senior Technical Lead
HCL Tech
12.2023 - Current
Senior Data Engineer
LTIMindtree Pvt Ltd
11.2021 - 12.2023
Associate Consultant
Capgemini Pvt Ltd
11.2020 - 10.2021
Associate Consultant
Capgemini Pvt Ltd
08.2019 - 11.2020
Python for Data Science, 2018-10-01
08-2018
Associate Technical Operations
IBM India Pvt Ltd
01.2017 - 06.2019
SSLC - undefined
Gurukulam High School
BE - ELECTRICAL AND ELECTRONICS ENGINEERING
K.L.N. College of Engineering
DIPLOMA - ELECTRICAL & ELECTRONICS ENGINEERING
Tamilnadu Polytechnic College