Nithin HS - Azure data Engineer - Wells Fargo

Summary

Total 6 + years ‘experience including 3.3 + years' experience in Pyspark, Spark , Big data , Azure Data Engineer hands-on experience in Cloud computing technologies ( Azure Data Factory, Azure Databricks, Spark SQL, Delta Lake, Azure Data Lake, Azure SQL DB ).
Responsive expert experienced in monitoring database performance, troubleshooting issues and optimizing database environment. Possesses strong analytical skills, excellent problem-solving abilities, and deep understanding of database technologies and systems. Equally confident working independently and collaboratively as needed and utilizing excellent communication skills.
Experience in ETL process, worked extensively on Data Extraction, Transformation and loading from different data sources using ADF.
Experience in Managing and storing confidential credential in Azure Key vault .
Orchestrated the end-to-end data integration pipelines using Azure Data Factory & Databricks

Overview

3

years of professional experience

Work History

Associate Data Engineer

Wells Fargo

01.2021 - Current

Optimized data processing by implementing efficient ETL pipelines and data transformation techniques.
Enhanced data quality for analysis by cleansing, validating, and standardizing raw datasets.
Primarily involved in Data Migration using SQL, SQL Azure , Azure Data Lake and Azure Data Factory, Pyspark, Spark, Big data, ADB.
Experience in ETL process,worked extensively on Data extraction,transformation and loading from deferent data sources using ADF
Experience on Migrating SQL data base to Azure data Lake , Azure delta lake , Azure SQL database ,Data bricks and SQL data warehouse and automate pipelines
migrating on premise databases to Azure Data Lake store using Azure Data factory
Developed and maintained data pipelines using azure services resulting in increased data processing speed.
Build analytic tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
Working with Source team to extract the data and it will be loaded in the ADLS
Creating the linked service for source and target connectivity Based on the requirement
Creating the pipelines and data sets which are deployed in ADF non-restricted
Once it's created pipelines and data sets will be triggered based on LOAD (HISTORY/DELTA) operations
Based on source (big or small) data loaded files will be processed in Azure data bricks by applying operations in Pyspark, Spark SQL . This will be deployed through Azure data factory pipelines.
Developed pipelines that can extract data from various sources and merge in to single source data sets in Data Lake using Databricks.
Designed, developed and deployed ETL solutions utilizing Microsoft synapse and data factory to Extract, Transform and Load data from various sources to the azure SQL data bases which act as a target table.
Implemented azure logic apps and functions to automate business processes and improve efficiency.

Education

Bachelor of Commerce - Accounting And Finance

St Josephs

Chikmagalūr, India

04.2001 -

Skills

Azure Services: Azure Data Factory, Azure Databricks, Azure Data Lake, Azure SQL database, Azure Key Vault, Logic App, Azure blob storage, Delta Lake, Spark SQL in Databricks

Spark Frame works: Spark Core, Spark SQL, PySpark, and HDFS

Cloud Environment: Microsoft Azure

Programming Languages: Python, SQL

Database: SQL Server, MySQL

Domain Expertise: Banking, Insurance

Additional Information

I hereby declare that the information furnished above is true to the best of my knowledge.

Place: Bangalore

Nithin HS