Summary
Overview
Work History
Education
Skills
Certification
Languages
Personal Information
Affiliations
Timeline
AdministrativeAssistant

Sayan Kumar Nandy

Kolkata

Summary

With over 9.5+ years of experience in the data engineering and SAP ecosystem, I have developed a strong expertise in building, managing, and optimizing end-to-end data pipelines across various platforms and technologies. I started my career as an SAP BW and BODS developer, where I honed my skills in data modeling, extraction, transformation, and loading (ETL) processes.

Over the last 3 years, I have transitioned to a Data Engineer role, primarily working with Azure Databricks, Azure Data Factory (ADF), and SQL Data Warehousing.

Overview

10
10
years of professional experience
1
1
Certification

Work History

Senior Data Engineer

Lexmark International
04.2022 - Current
  • Spearheaded migration process of multi-layer ETL jobs from BigDecisions to Azure Data Factory and Azure Databricks, transforming UI-based solutions into scalable, reusable codebases using PySpark, improving report consistency and achieving 50% increase in data processing time.
  • Integrated and processed data from multiple sources like AWS S3, AWS RedShift, D365, SqlServer, Oracle, handling various file formats like CSV, Parquet, Json, ORC and txt, using Azure Data Factory.
  • Recreated functions and stored procedures of SqlServer in Databricks, enabling advanced reporting capabilities and decreasing report generation time by 50%, utilizing delta table optimization technique like Z-Ordering, Optimize and Vacuum and Databricks Workflow.
  • Integrated multiple notebooks to streamline data processing and governance using Databricks Workflow, following Lakehouse Medallion Architecture, to process batch datasets of size 1TB, approximately containing 2 billion records daily.
  • Planned and executed real-time streaming solution to support dynamic data processing, using Azure EventHub, Spark Structured Streaming and Databricks Workflow, efficiently managing 2 million records per hour.
  • Successfully Completed Application Migration from Datastage to SAP BODS within tight timeline during SAP S4 HANA version upgradation.

Senior Associate Consultant

Infosys Ltd.
08.2018 - 04.2022
  • Designed, Developed and Supported complex ETL jobs using Data Integrator Scripts, Workflows, Dataflow, Data stores, Transforms and Functions
  • Implemented BODS job which will Work along with FIORI App as per Client Requirements
  • Experience in Error handling and debugging coding issues and troubleshoot production problems

Application Development Analyst

Accenture
11.2014 - 08.2018
  • Worked on SAP BODS and BW projects for data migration and reporting.
  • Developed ETL jobs, InfoProviders, and queries for business users.
  • Led data integration efforts using LSMW and supported production environments.

Education

B-Tech - Computer Science & Engineering

Meghnad Saha Institute Of Technology
06-2014

Skills

  • Python programming

  • Data pipeline design

  • ETL development

  • SQL

  • SAP BODS

  • Pyspark

  • Azure Data Factory

  • Medallion Architecture

  • ADLS Gen2

  • Delta Lake

  • Databricks

Certification

  • Certified SAFe® Practitioner

  • AZ-900(Microsoft Azure Fundamentals)

  • Databricks Certified Data engineer associate

  • CCA175 Cloudera Spark and Hadoop Developer

Languages

English
Advanced (C1)
Hindi
Advanced (C1)
Bengali
Advanced (C1)

Personal Information

Affiliations

  • Won Manager Appreciation Award For key Contribution in Structured Streaming Project- Lexmark.
  • Won Quarterly Appreciation Award for successful Application Migration (Datastage to BODS)- Lexmark.
  • Received AMS Peer Award for AMS Reporting Support as an appreciation for key team player in the project- Accenture.
  • Received Insta award for contribution towards Business during Pandemic - Infosys
  • Peer Award – Infosys

Timeline

Senior Data Engineer

Lexmark International
04.2022 - Current

Senior Associate Consultant

Infosys Ltd.
08.2018 - 04.2022

Application Development Analyst

Accenture
11.2014 - 08.2018

B-Tech - Computer Science & Engineering

Meghnad Saha Institute Of Technology

Similar Profiles

CREATE PROFILE
Sayan Kumar Nandy