Debanshu Bhattacharya
Senior Data Engineer | Google Cloud Platform | Product Development
Bangalore
Work Preference
Summary
With over 16 years of software engineering expertise, Senior Data Engineer with expertise in Data Engineering and Data Warehousing. Delivered scalable data solutions through the design and maintenance of robust data pipelines on GCP, achieving a 60% cost reduction in data storage. Enhanced data processing speed by 30% through optimization of BigQuery pipelines, driving operational excellence.
Overview
16
16
years of professional experience
1
1
Certification
3
3
Languages
Work History
Senior Data Engineer
ANZ Banking Group
Bengaluru
02.2020 - Current
- Currently working as a Senior Data Engineer on the Wholesale Loss Data Project, CRDHS Migration Project (under the Risk Data Hub), using Google Cloud Platform, which holds data from different source systems to facilitate the calculation of Loss Given Default for defaulted customers.
- Understand the requirements from the business team and the documents from the data modelers, and build the Source, Integration, Enrich, and building block models to achieve the calculation using data from multiple source systems, such as CAP, CACHE, VISIONPLUS, LOANIQ, AMBIT, etc.
- Own and manage the end-to-end code migration, deployment, and implementation in both SIT and Prod.
- Understanding the source system and using Data Vault models (UniTemporal and BiTemporal data), I am responsible for coding and developing multiple models in Google Cloud Platform using BigQuery and the DBT tool, as well as in every layer of the warehouse, and we are setting up the standards for data migration at all levels.
- Designed and maintained scalable data pipelines on GCP, processing TB-scale datasets using Apache Airflow and BigQuery.
- Performed regular data validation and cleansing, optimizing database quality standards.
- Implemented data quality checks, schema enforcement, and alerting, significantly improving data reliability and stakeholder trust.
- Achieved the 60% cost reduction for data storage.
- Improved data processing speed by 30% through optimization of BigQuery pipelines.
- Managed a data pipeline encompassing 50TB of data monthly, ensuring seamless ETL processes.
- Achieved 40% reduction in query execution time by redesigning SQL queries for performance optimization.
- Automated the deployment tasks to reduce manual efforts and created accelerators to create models/schema for the migration activities.
- Led team meetings to discuss progress, challenges, and strategies for database management.
- Participated in peer code reviews, contributing to the maintenance of high coding standards within the team.
- Provided mentorship and guidance to junior data engineers, fostering professional development and skill advancement.
Lead Engineer
ANZ Banking Group
Bengaluru
07.2017 - 01.2020
- Led CRIS and RAZOR application support (Level 3).
- Contributed to the incident and change management process.
- Contributed to delivery excellence.
- Automated manual tasks.
- Contributed to enhancement and defect fixing using Unix and Oracle.
- Created procedures for reviewing, evaluating, and inspecting engineering projects for increased efficiency.
- Collaborated with international teams to exchange best practices and innovations, enhancing global project outcomes.
Senior Application Support Engineer
Tata Consultancy Services
Bengaluru
05.2016 - 07.2017
Senior Application Support Engineer
Cognizant Technologies Solutions
Pune
01.2013 - 04.2016
Technical Associate
Tech Mahindra
Pune
12.2009 - 01.2013
Education
B.Tech - Computer Science & Engineering
West Bengal University of Technology
Kolkata01.2008
Skills
BigQuery
CloudBuild
Composer
Apache AIrflow
DBT
ELT pipeline design
Data modeling ( Star/snowflake schemas)
Data quality, validation & observability
SQL
Python
Accomplishments
- ANZ Year End recognition Award - FY - 2023: Honored with the company's Year End Recognition FY-2023 under the category 'Deliver what matters'
- Tech area Champion Award: Got awarded twice in 2024 for successfully deploying critical deliverables into production.
- Tech area Star Award: Got awarded twice in 2025 for successfully automating manual deployment tasks, Data cost reduction and Computational cost and time optimisation.
Certification
Google Cloud Certified Professional Data Engineer
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Timeline
Senior Data Engineer
ANZ Banking Group
02.2020 - Current
Lead Engineer
ANZ Banking Group
07.2017 - 01.2020
Senior Application Support Engineer
Tata Consultancy Services
05.2016 - 07.2017
Senior Application Support Engineer
Cognizant Technologies Solutions
01.2013 - 04.2016
Technical Associate
Tech Mahindra
12.2009 - 01.2013
B.Tech - Computer Science & Engineering
West Bengal University of Technology