Summary
Overview
Work History
Education
Skills
Certification
Timeline

Jatin

Summary

1 year of experience in Big data engineering, worked on data-related projects on Databricks, AWS, Azure, and Power BI etc.

Overview

1
1
year of professional experience
1
1
Certification

Work History

Data Engineer

Reflections Info Systems
Trivandrum
12.2021 - Current
  • Data Ingestion Pipelines:WebScrapping/DBFS->ETL(Databricks)->Azure DataLake.Get the data from Various Website or CSV/JSON files in dbfs and performed Quality Checks and Transformation Using Pyspark then load this data into Azure DataLake
  • Unified DataWarehouse:Azure DataLake->ETL(Databricks)->DeltaLake.Read CSV data from DataLake then performed SCD and Transformation on it finally load into delta lake
  • DataWarehouse(AWS):S3->ETL(Glue)->Redshift.Read Data from S3 using Crawler in GLUE then performed SCD and Transformation Using Pyspark and SparkSQL then stored data into REDSHIFT
  • CDC Real-time Streaming Pipeline:PostgreSQL->Azure Event Hub->Databricks(ETL)->Delta Lake.Captured data from database using Streamsets into EVENT hub then read this data into databricks notebook performed Extraction from JSON format and Transformation using Pyspark load into delta lake with soft delete.
  • Power BI Reports:Data Read->ETL(Query editor)->Data modelling and DAX->Visualization
  • DataMarts:Created DataMarts from Warehouse according to business requirements
  • Databricks Notebook Orchestration using Apache AIRFLOW

Machine Learning Intern

Grroom
Mumbai
09.2021 - 12.2021

Education

Bachelor of Computer Science And Engineering - Computer Science

Jiwaji University, Gwalior, Madhya Pradesh ,India
09.2021

77%

12th -

Govt. Mixed Hr, Sec. Jammu, India
05.2017

77%

10th -

Govt. Mixed Hr. Sec. Jammu
05.2015

86.20%

Skills

  • SQL
  • Spark
  • Python
  • Data Warehousing & Marts
  • Change data capture
  • Kafka,Event Hub
  • Airflow,Docker,Linux
  • Streaming
  • WebScrapping
  • AWS(EMR,Glue,S3,RedShift)
  • Power BI(Data Modelling,DAX,Reports)
  • Databricks
  • Azure(DataLake,ADF)

Certification

Azure Data Fundamental DP-900

Timeline

Data Engineer - Reflections Info Systems
12.2021 - Current
Machine Learning Intern - Grroom
09.2021 - 12.2021
Jiwaji University - Bachelor of Computer Science And Engineering, Computer Science
Govt. Mixed Hr, Sec. Jammu - 12th,
Govt. Mixed Hr. Sec. Jammu - 10th,

Similar Profiles

CREATE PROFILE
Jatin