Summary
Overview
Work History
Education
Skills
Software
Interests
Timeline
Generic
Suraj Tyagi

Suraj Tyagi

Data Engineer
Delhi

Summary

Reliable Data Engineer keen to help companies collect, collate and exploit digital assets. Practiced at cleansing and organizing data into new, more functional formats to drive increased efficiency and enhanced returns on investment.

Overview

3
3
years of professional experience
4
4
years of post-secondary education

Work History

Data Engineer

IBM
Noida
12.2021 - Current
  • Working with DirectTV to handle and manipulate Consumer Video data to generate helpful insights for their business.
  • Created and maintaining Source Ingestion Framework using data mesh architecture in Databricks using python which picks data from multiple sources either it be RDBMS or delta lake storage or even kafka and stages data into AWS s3 bucket and later dumps data into snowflake warehouse.
  • Developed few Reports using Metadata Tables to Improve Production Support by getting Average counts Processed in last 10 daily runs for more than 1200 databricks jobs. For this i was highly appreciated by higher Management with quarterly award as it helped them understand flow easily.
  • Almost 1500+ Databricks jobs runs on this framework on daily basis as part of incremental load and processes around hundred's of gigabytes.
  • Integrated Great Expectations in my pyspark code into this Source Ingestion Framework for data quality audits check and designed various custom expectations as well as per business.

Data Engineer

Infosys
Pune
11.2020 - 12.2021
  • Generated detailed studies on potential third-party data handling solutions, verifying compliance with internal needs and stakeholder requirements.
  • Designed certain transformations using spark to transform and manipulate data for Microsoft retail data for projecting upcoming year's finances.
  • Developed, implemented, supported and maintained data analytics protocols, standards and documentation for a better understanding of stakeholders.
  • Analyzed complex data and identified anomalies, trends and risks to provide useful insights to improve internal controls.
  • Worked on Certain Azure Cloud Services like Azure Data Factory ,Azure Databricks , ADLS and Azure Synapse Analytics to handle big data .

Education

Bachelor of Technology - Computer Science Engineering

Abdul Kalam Technical University
Ghaziabad
08.2016 - 09.2020

Skills

    Data analysis

undefined

Software

Python

Azure Cloud Services

SQL

Spark

Power BI

Snowflake

Interests

Gaming

Sports

Timeline

Data Engineer

IBM
12.2021 - Current

Data Engineer

Infosys
11.2020 - 12.2021

Bachelor of Technology - Computer Science Engineering

Abdul Kalam Technical University
08.2016 - 09.2020

Similar Profiles

CREATE PROFILE
Suraj TyagiData Engineer