Nilab Nath
Guwahati
Summary
Proven Associate Software Engineer at EYGDS, adept in Python, PySpark, and Pandas, with a knack for developing innovative data validation and preprocessing solutions. Excelled in Azure Databricks, enhancing data accuracy by automation pipelines, demonstrating strong analytical and problem-solving skills.
Overview
2
2
years of professional experience
1
1
Certification
Work History
Associate Software Engineer
EYGDS
Kolkata
08.2022 - Current
- Written PySpark/Pandas codes for validation, tested codes in Databricks workflows, file management in Databricks Blob, and written file conversion code in PySpark.
- Development of PySpark and Pandas code for data validation, preprocessing, and extraction for Databricks; used the Pandera module for schema checks of a dataframe; developed PySpark code for data extraction from the dataframe; created a pipeline to schedule the task so that every day we get preprocessed data.
- Developed a preprocessing module for all tables in PySpark, and developed code in Pandas to calculate the percentage of null values and other discrepancies.
- Developed code in PySpark in order to fulfill other project needs, such as counting the check of a particular value, etc.
- Used the Plotly module to display the graph of data validation.
- Developed PySpark code to impute missing values in data according to business needs.
Education
B.Tech. - Electronics and Electrical Engineering
KIIT-Bhubaneswar
Skills
Technologies : ML/DL – Data pre-processing, Data wrangling, Exploratory data analysis, Feature Engineering
Cloud / Platform – Azure
Tools/Open source: Azure Databricks
Languages: Python, Pyspark, Pandas, Basic SQL
- GIT
- GEN AI
Certification
- AZ-900
Languages
Assamese
First Language
English
Upper Intermediate (B2)
B2
Hindi
Upper Intermediate (B2)
B2
Timeline
Associate Software Engineer
EYGDS
08.2022 - Current
B.Tech. - Electronics and Electrical Engineering
KIIT-Bhubaneswar
Similar Profiles
Vandana YadavVandana Yadav
Okta Senior Consultant at EYGDSOkta Senior Consultant at EYGDS
Maansi ThakurMaansi Thakur
Senior Software Engineer at EYGDSSenior Software Engineer at EYGDS
Ranjini Mary LyngdohRanjini Mary Lyngdoh
Assistant Manager - Talent - STC team at EYGDS BangaloreAssistant Manager - Talent - STC team at EYGDS Bangalore