Hi, I’m
Yatharth Mishra
Senior Associate
Pune,MH
Summary
Senior Data Engineer with 10+ years of experience architecting and delivering scalable, low-latency, and compliance-driven data platforms across payments, wealth management, and large-scale enterprise systems. Proven expertise in ETL/ELT modernization, Snowflake, dbt, and CI/CD-driven data deployments, enabling reliable analytics and faster time-to-insight.
Strong advocate of AI-assisted engineering, leveraging tools such as Microsoft Copilot, Windsurf, and advanced prompt engineering to significantly improve developer productivity, code quality, and operational efficiency. Highly skilled in designing resilient data pipelines, optimizing large-scale data processing, and building robust data architectures that support business-critical decision-making.
Hands-on expertise in Python and SQL, with deep experience in cloud-native data platforms, including Snowflake and Amazon S3, and a strong track record of cross-functional collaboration to deliver high-impact, data-driven solutions at scale.
Overview
10
years of professional experience
Work History
BNY
Senior Data Engineer - SEPA Payments and VOP BNY
06.2023 - Current
Job overview
- Delivered SEPA standard and instant payment pipelines (
- Built Python-based API integrations for IBAN/account-name validation with Bank Verlag (Germany).
- Architected high-throughput Snowflake pipelines (800K records/day) with idempotent processing and reconciliation reporting.
- Implemented batch-oriented processing with retries, Moogsoft alerting, and S3 outputs.
- Embedded GitLab CI/CD pipelines for automated testing, versioning, and environment promotion.
- Accelerated delivery 10x via AI-driven workflows (Copilot, Windsurf, custom agents).
- Financial Services
- Tech-Stack: SQL, Datawarehousing, Pentaho , ETL , Oracle , Vertica , Gitlab (CI/CD) , JIRA , Python , Amazon S3, Splunk,Postman
BNY
Data Engineer
06.2024 - 06.2025
Job overview
- Developed DBT ELT pipelines, transforming raw data into canonical models and analytics-ready marts.
- Built incremental CDC pipelines for large datasets, optimizing performance and compute cost.
- Created reusable DBT macros and modular SQL with ref(), ensuring maintainable, testable models.
- Implemented BDD-style DBT tests for data accuracy and lineage.
- Integrated pipelines with GitLab CI/CD for reliable model releases.
- Analytics and Transformation (DBT)
- Tech-Stack: SQL, Datawarehousing , DBT , ETL , JIRA , Python , Snowflake, VS Code (ID)
BNY
Enterprise Integration Project – Lead Backend Developer
08.2021 - 06.2024
Job overview
- Migrated mortgage platform data across multiple source systems with account- and balance-level segregation.
- Developed Pentaho ETL pipelines (Oracle > Vertica / MySQL) including Metadata Injection, Transformations, Lookups, Joins, Normalizer, De-normalizer, Filters, Switch Case, Table Output, and Insert/Update.
- Implemented data quality rules, validation checks, error capture in failed records table, and reconciliation reports.
- Error Tables (i.e. failed record table) along with the reason of failure of load.
- Maintained CI/CD pipelines for version control, deployments to Test and Dev environments, and pipeline optimization.
- Performance tuning of Database queries.
- Documented processes in Confluence, tracked tasks in Jira, tuned database queries, and designed tables as per project requirements.
- Mentored junior engineers and groomed resources technically.
- Enterprise Integration Project – Lead Backend Developer
- Tech-Stack: SQL, Datawarehousing , Pentaho , ETL , Oracle , Vertica , Gitlab (CI/CD) , JIRA , Python , Splunk
CGI
Sr. Software Engineer
12.2020 - 08.2021
Job overview
- Migrated WM investment and portfolio data from legacy systems using PDI 8.2.
- Designed ETL pipelines with data integrity checks, duplicate detection, and validation rules.
- Developed Oracle PL/SQL procedures/triggers, capturing failed records with detailed logging.
- Delivered end-to-end reconciliation reports and maintained CI/CD pipelines.
- Wealth Management
- Tech-Stack: SQL, Datawarehousing , Pentaho , ETL , Oracle
Netlink Software Group America Inc
Sr. Software Engineer
08.2017 - 12.2020
Job overview
- Developed BI solutions for Sirus XM Coaching-Log with multi-user access and reporting.
- Designed PDI ETL pipelines, reusable transformations, and SQL Loader bulk loads.
- Captured hierarchical feedback for actionable reporting; prepared full ETL documentation.
- Built interactive dashboards and reports using Power BI to enable data-driven decision-making.
- BI and Analytics
- Tech-Stack: SQL, Datawarehousing , Pentaho , ETL , Oracle , Vertica , Gitlab (CI/CD) , Pentaho , Informatica , Power BI
Synques Consultancy Pvt. Ltd
Software Engineer
06.2014 - 06.2016
Job overview
- Developed school management module (attendance, timetable, exams, transfer certificates).
- Implemented full project lifecycle on-site at Billabong School, including training.
- Built web applications.
- School Management and Web Development
- Tech-Stack: SQL , PHP , HTML , CSS , Javascript , Json , XML
Education
Rajiv Gandhi proudyogiki Vishwavidyalaya BhopalBhopal
from Computer Science
01.2014
St. Paul Sr. Secondary Co-ed School (CBSE)Bhopal
from Science
01.2010
Skills
Data Engineering: Python, SQL, ETL/ELT, API Integrations, Batch Incremental Processing, Data Quality, Reconciliation , Data warehouse
Data Platforms: Snowflake, Oracle, MySQL, Vertica, Data Lakes
Worked with various formats : Different Database sources , JDON , XML , CSV , Parquet, API
Kafka
JIRA
Postman
ETL Modeling: DBT, Pentaho (PDI), Informatica PowerCenter, ETL Pipelines , ETL Optimization techniques
DevOps CI/CD: GitLab CI/CD, Version Control, Automated Deployments, Environment Promotion
Cloud Storage: AWS S3
Payments Compliance: SEPA Credit Instant Payments, IBAN Validation, Verification of Payee
AI-Augmented Development: Microsoft Copilot, Windsurf, Prompt Engineering, Custom AI Agents
Web Application: HTML , CSS , Javascript ,JSON
Timeline
Data Engineer
BNY
06.2024 - 06.2025
Senior Data Engineer - SEPA Payments and VOP BNY
BNY
06.2023 - Current
Enterprise Integration Project – Lead Backend Developer
BNY
08.2021 - 06.2024
Sr. Software Engineer
CGI
12.2020 - 08.2021
Sr. Software Engineer
Netlink Software Group America Inc
08.2017 - 12.2020
Software Engineer
Synques Consultancy Pvt. Ltd
06.2014 - 06.2016
Rajiv Gandhi proudyogiki Vishwavidyalaya Bhopal
from Computer Science
St. Paul Sr. Secondary Co-ed School (CBSE)
from Science