Tamaghna Dutta

Data Engineer | Databricks
Mar 2023 – Current

• Led migration of data from Teradata to Snowflake, leveraging Databricks' powerful capabilities.
• Utilized Databricks Notebooks for developing and executing data engineering workflows and automating pipeline tasks.
• Designed and implemented data migration pipelines using Delta Live Tables and Unity Catalog, ensuring the efficient streaming of data.
• Built robust streaming pipelines to process and manage real-time data using Delta Live Stream.
• Integrated REST APIs to publish delta records ingested in the gold layer to business endpoints, enhancing data accessibility and utilization.
• Worked collaboratively with cross-functional teams to ensure scalable and efficient data architecture.

Data Engineer | Snowflake
Oct 2021– Mar 2023

• Worked on Migration Activity from Redshift to
  Snowflake using via Matillion.
• Integrate different sources into Snowflake datalake
  and create the process to automate data ingestion
  via SNOWPIPE using AWS Lambda.
• Led a team of six people where we successfully
  delivered projects of building jobs to cater a
  customer base of 10 million. These jobs include
  high-priority assignments that have a significant
  direct impact on the client's revenue, in addition to
  reports related to supply and operations.
• Generated JSON output from SQL tables using
  Object Construct.
• Reduced clients operation cost and time of batch
  run by optimizing code and automating daily
  processes
• Implemented solutions to create workflows which
  handles the data after source migrated from SAP to
  Kraken via AWS Services, Snowflake and Matillion

Data Engineer | Redshift
Dec 2019– Oct 2021

• Designed a dataset and identified datapoints which
  enabled ingestion of data from SAP ISU and CRM,so
  it can be used on customized workflows which wasn't earlier possible in SAP or vanilla product.
• Delivered projects based on AWS services
  (S3,Redshift) and Matillion for debt collection data
  based on tables.
• Build automated jobs to create and populate tables
  on data lake based on customer requirement
• Developing and optimizing ETL jobs (PL/SQL)

Data Engineer | Databricks
Aug 2017– Dec 2019

• Designed and developed Enterprise data Lake(EDL)
  architecture with micro service
  framework based on Databricks ,Spark and AWS.
• Delivered projects based on AWS
  services(S3,Redshift,PostgreSQL,Lambda) to build
  pipelines and integrate into data lake.
  Designing and building highly scalable and efficient
  end-to-end data pipelines to meet business
  requirements in clinical supply chain
  area and publish it to planning tool.