Suyash Kaderkar
Pune
Summary
Results-oriented Data Engineer with 3 years of experience in the complete data lifecycle, excelling in extraction, cleaning and validation. Demonstrates expertise in developing robust ETL pipelines using AWS services and Azure Databricks and performing Big Data processing with PySpark. Proficient in SQL, Redshift, S3, ADLS, with a strong focus on data integrity, and crossfunctional collaboration to drive data-driven decision-making.
Overview
4
4
years of professional experience
Work History
Data Engineer
Kotak Mahindra
Thane
05.2025 - Current
- Built an end-to-end ETL data pipeline to migrate historical and incremental data from Sybase to Amazon Redshift, using AWS S3 as a staging layer, ensuring data integrity, reconciliation accuracy, and audit compliance.
- Designed and optimized Amazon Redshift tables using appropriate DISTKEY and SORTKEY to improve query performance and support large-scale banking datasets.
- Implemented YAML-driven pipeline configurations, enabling dynamic job execution, and seamless integration with Apache Airflow DAGs.
- Orchestrated data workflows using Apache Airflow, leveraging EMR Add Steps Operator and DB To Redshift Operator for reliable batch processing and data loading.
- Managed historical data loads through manually triggered Airflow DAGs, and scheduled incremental loads aligned with legacy Sybase workflows.
- Migrated SQL queries and stored procedures from Sybase to Redshift, transforming incompatible syntax and validating business logic across dependent tables.
- Performed post-load data quality checks, including record count reconciliation, null validation, and duplicate detection, to meet banking regulatory standards.
- Integrated pipeline code with Azure DevOps CI/CD, ensuring version control, controlled deployments, and automated DAG updates.
Data Engineer
Poonawalla Fincorp
Pune
08.2023 - 04.2025
- Designed and implemented a scalable data ingestion pipeline using WinSCP, AWS S3, Databricks, and ADLS Gen2 with a Bronze–Silver–Gold architecture.
- Built PySpark-based ETL pipelines to ingest CSV files, perform data validation, cleansing, and business transformations.
- Implemented incremental data processing logic by tracking processed files via control tables, avoiding duplicate ingestion.
- Applied data quality checks including null validation, datatype enforcement, deduplication, and reject record handling.
- Orchestrated end-to-end workflows using Databricks Jobs, with retries, alerts, and logging.
- Exported curated gold-layer datasets back to AWS S3 for downstream analytics and reporting.
- Monitored pipeline health using job logs, metrics, and KPIs, such as record counts, failure rates, and SLA adherence.
LEGAL ASSOCIATE
Simplify
Pune
03.2022 - 02.2023
- Creating sale contracts for UK-based properties by scrutinizing and validating title documents of clients.
- Placing orders for searches for the mortgagee and buyer according to the property area standard needs.
- Reading and Creating Reports of Searches for Mortgagee and Buyer.
- Assisting conveyancers according to their urgent needs.
- Assisting Conveyancers to find out right documents on UK Government Websites.
Education
Master of Science - Microbiology
SPPU
Pune08-2021
Bachelor of Science - Microbiology
BAMU
Omerga04-2019
Skills
- Languages - SQL, Python, Pyspark
- AWS - Amazon S3, Lambda, AWS Glue, Athena, Amazon Redshift
- Azure - Databricks, ADF, ADLS
- MS Office - Ms word ,MS Excel,Office360
Languages
English
Proficient (C2)
C2
Hindi
Proficient (C2)
C2
Timeline
Data Engineer
Kotak Mahindra
05.2025 - Current
Data Engineer
Poonawalla Fincorp
08.2023 - 04.2025
LEGAL ASSOCIATE
Simplify
03.2022 - 02.2023
Master of Science - Microbiology
SPPU
Bachelor of Science - Microbiology
BAMU