Vanaja R

Project Summary:
Working as a Senior Data Engineer on the WholeSale Credit Risk Platform (CRP) for a leading global bank, developing scalable, fault-tolerant data pipelines using PySpark and Python to process, cleanse, and transform financial risk data for regulatory reporting and advanced analytics.

Key Responsibilities:

• Designed and implemented large-scale PySpark-based ETL pipelines on distributed data platforms to process high-volume credit risk data from upstream systems.
• Developed modular, reusable Python code for data transformation, enrichment, validation, and aggregation across multiple risk categories and geographies.
• Tuned PySpark jobs for optimal performance and memory management using techniques like caching, broadcast joins, and partitioning strategies.
• Integrated with Oracle DB for source ingestion and final reconciliations, using JDBC connectors within PySpark workflows.
• Implemented robust data quality checks , exception handling, and logging mechanisms to ensure data integrity and traceability across all pipeline stages.
• Scheduled and monitored data jobs using Autosys , ensuring timely execution and alerting in case of job failures or SLA breaches.
• Used Bitbucket for version control and code collaboration, following best practices for branching, pull requests, and peer reviews.
• Worked closely with risk analysts, QA, and business users to gather requirements, validate output, and support UAT in production environments.

Project Summary:

Worked as a Data Engineer for a global banking client, building robust data pipelines using Hive on Hadoop to support compliance reporting, customer account analytics, and operational dashboards.

Key Responsibilities:

• Designed and developed end-to-end ETL workflows using HiveQL on the Hadoop ecosystem for processing large-scale structured data from multiple banking domains.
• Built complex Hive tables (managed and external) , performed partitioning and bucketing , and optimized queries for faster reporting and downstream consumption.
• Developed Unix shell scripts for data ingestion, job automation, error logging, and file-level validations as part of the ingestion layer.
• Created and scheduled Control-M workflows for batch processing, ensuring job dependencies and SLAs were consistently met.
• Participated in data quality checks , performed source-to-target mapping , and supported regression testing across multiple releases.
• Used Bitbucket for version control and Sonarlint to enforce code quality standards and reduce technical debt in SQL and shell code.
• Interacted with business users and analysts to understand data requirements and translated them into scalable Hive-based data models.

Project Summary:
Worked on a large-scale retail data processing system for a leading US-based client, focusing on the design, development, and maintenance of high-volume transactional and batch data pipelines in a mainframe environment.

Key Responsibilities:

• Designed and implemented high-performance batch data pipelines using JCL, COBOL, and DB2 , handling millions of daily retail transactions.
• Created and optimized SQL queries and DB2 procedures to support real-time and batch data processing for reporting and analytics needs.
• Built and managed VSAM datasets and CICS transaction screens for inventory, sales, and pricing modules, ensuring data integrity and consistency.
• Automated repetitive data workflows and reporting tasks using REXX scripts , reducing manual intervention and improving productivity.
• Orchestrated scheduled jobs and complex batch chains using Autosys , ensuring reliable and timely data processing with proper dependencies and error handling. Automated repetitive data workflows and reporting tasks using REXX scripts , reducing manual intervention and improving productivity.
• Orchestrated scheduled jobs and complex batch chains using Autosys , ensuring reliable and timely data processing with proper dependencies and error handling.