Khushbu Rani
Data Engineer
Bengaluru
Summary
Result driven data engineer with approx 6 years of hands on experience in designing developing and optimizing data pipelines for ETL data processing, analysis and reporting. Detailed, oriented and adept at data modelling, ETL process with informatica power centre, database management with data warehousing solution.
Overview
7
7
years of professional experience
Work History
Project Lead
Axtria
05.2024 - Current
- Integrated existing systems with blob storage.
- Optimized queries on distributed systems like Hive and Impala or Presto to improve performance of analytics tasks.
- Executed pyspark code using databricks notebook and was involved in refreshing end to end data pipeline using ADF.
- Performed end-to-end testing of all components in the Big Data architecture.
Data Engineer II B
Bank Of America
05.2020 - 04.2024
- Worked on designing and implementation of end to end pipeline using Azure Data Factory.
- Developed pyspark script to accomodate business logic into big data platform .
- Worked in creating Informatica mappings, workflows using flat files,data ingestion for ETL process followed by QA testing .
- Hands on experience in data processing and data manipulation skills like data warehousing concepts, SCD types, etc. Worked on spark 2.0 for processing data with respect to the project requirement.
- Performed impact analysis on available data as per requirement.
- Worked on environment set up with creating target table for the environment.
- Scheduled unix script along with Spark code with Autosys.
Business Tech Analyst
ZS Associates
10.2017 - 11.2019
- Developed databricks notebook using pyspark which efficiently processes millions of records .
- Have Good understanding of storage layer and processing layer of hadoop framework.
- Good experience of data processing engines like Map reduce and spark.
- Developed managed and external hive tables with appropriate partitions and bucket.
- Deep understanding on various method to tune and optimize hive performance.
- Developed HQL scripts for analysis of data in hive tables.
- Good working knowledge of spark core(RDD) and spark SQL(dataframes)
- Good understanding of oozie workflow generation and scheduling coordinators.
- In depth knowledge of QA methodologies like agile and waterfall.
Education
Bachelor of Technology - Electrical, Electronics And Communications Engineering
Birla Institute of Technology, Mesra
04.2001 -
Skills
SQL
Accomplishments
- Global recognition gold award, 10/2022
- Global recognition Silver award, 09/2021
- Global recognition Bronze award, 03/2022
Achievements
Patented Rule based data transformation using edge computing.
Patent No.-P12549US01
Filed date: 4 Feb2022
Publication date: 10 Aug 2023.
Abstract : A system for data processing using machine learning processing and distributed architecture is described.Specifically proprietary data transformation rules to be applied for data processing may be stored at edge computing devices, the bulk of data processing performed at the central computing node that houses the databases. A subset of dataset in the database sent from central computing to edge computing node. The edge computing node may generate a second dataset based on applying data transformation rules to the subset of the data set. the central computing node determine using ML algorithm and based on the subset of the dataset and the second dataset, the data transformation rules,which may then be applied to the rest of the dataset.
Awards
Certification
- Oct 2021| Azure fundamental
- Aug 2022| Azure Data Engineer Associate
- June 2021| PCEP
Timeline
Project Lead
Axtria
05.2024 - Current
Data Engineer II B
Bank Of America
05.2020 - 04.2024
Business Tech Analyst
ZS Associates
10.2017 - 11.2019
Bachelor of Technology - Electrical, Electronics And Communications Engineering
Birla Institute of Technology, Mesra
04.2001 -