Dipan Debnath
Summary
Experienced Data Engineer having industry experience in ETL ,Big Data and Spark framework. Demonstrates strong analytical skills and a results-driven approach to solving complex data challenges. Brainstormed multiple business cases to solve using analytical models and GenAI applications.
Overview
4
4
years of professional experience
Work History
Data Engineer
Indorama Ventures Global Shared Services
Kolkata
09.2022 - Current
- Developed scalable ETL/ELT data pipelines deployed in AWS EC2 with runtime status logged in Cloudwatch .
- Leveraging AWS Lambda, Glue to process data across various sources,invoke functions with Eventbridge and notify using SNS.
- Build regression and classification models using Machine Learning and NLP(RNN,LSTM) techniques ,used LLM models to develop GenAI applications using Langchain framework.
- Developed APIs using FastAPI to retrieve data from the Postgresql database and ERP systems.
Software Engineer
Infosys Ltd.
Mysore
07.2020 - 08.2022
- Developed ELT data pipelines, pulled CRM data from Salesforce to AWS S3 using Appflow.
- Build Spark jobs using PySpark and monitored DAG using Airflow.
- Developed Classification model using SVM,Naive Bayes along with feature engineering for engine failure analysis and reduced TTF by 15 percent.
- Deployed Machine Learning models in AWS Sagemaker and build Tableau dashboard for analytics
Education
BTech - Mechanical Engineering
NIT Durgapur
Durgapur, India08.2020
High School Diploma -
Kalyani Central Model School
Kalyani, India08.2015
Skills
- Programming language- C, Java, Python, R, SQL, Javascript
- Machine Learning- Decision Tree, Random Forest, Linear Regression, Clustering, Naive-Bayes, KNN, SVM, XGBoost
- Deep Learning- ANN, CNN, Tensorflow, Keras
- NLP- Tokenization,Vectorization,POS Tagging,RNN,LSTM, Transformer
- AWS- Glue, Lambda, Appflow, Cloudwatch,EC2, EventBridge,Sagemaker,SNS, Databricks
- Big Data- Hadoop, Spark, Hive, PySpark, Kafka
- Database-Postgresql, SQL Server, MongoDB
- Data Warehouse-Redshift, Snowflake
Accomplishments
- Received Gold Star Award in FY Q4 2023 at IVGSS
- Received Spot Award in 2021 at Infosys Limited
Timeline
Data Engineer
Indorama Ventures Global Shared Services
09.2022 - Current
Software Engineer
Infosys Ltd.
07.2020 - 08.2022
BTech - Mechanical Engineering
NIT Durgapur
High School Diploma -
Kalyani Central Model School