Kundan Giri
Senior Data Engineer
Bengaluru,Karnataka
Summary
With 10+ years of experience, I have been part of several projects which gave me an insight of various new and emerging technologies currently In practice in the industry. Self-learning new technologies and implementing in the industry level project has become the part of our profession. For making POC for different projects to the full lifecycle development of software, I have done it all. P.S. Talking about my skills, I have experience in Hadoop Ecosystem with HDFS, Hadoop MapReduce, Hadoop YARN, Apache HBase,Hive, Oozie,Airflow,GCP,Apache Crunch,Spark,Scala,Python and Java.
Overview
10
10
years of professional experience
6
6
years of post-secondary education
Work History
Senior Data Engineer
Walmart
Bengaluru
10.2020 - Current
- Work with Data science team to process ETL and provide required data which can be consumed by various forecast models to generate future forecast like inventory required for individual stores.
- Migrate the entire Data lake and ETL setup from on-prem cluster to GCP
- Automated the orchestration of pipeline by using Airflow
- Build the platform code for Spark ETL with generic design to cater need of multiple teams and generalizing the data pipeline development process across the organization.
- Reengineered existing ETL workflows to improve performance by identifying bottlenecks and optimizing code accordingly.
- Participated in strategic planning sessions with stakeholders to assess business needs related to data engineering initiatives.
- Create ETL pipeline to consume forecast output tables and data from different other sources like TeraData, RDBMS, Hive etc to transform and generate analytical output tables which can be leveraged by API to show reports and graphs to UI
- Also process adjustment done at UI (override forecast values) in near real time and override the same in output tables (Hive & GCS)
- Build Data lake by creating various ETL pipeline to extract data from various source, standardize, normalize and make the data production ready
Big Data Developer
True Fit
Mumbai
06.2019 - 10.2020
- Building Spark project to consume ecommerce data from Hive, transform using Spark and SparkSQL and dump result into Postgres which can then be consumed by D3 framework to generate report
- Evaluated and recommended big data tools, frameworks, and platforms to support business requirements, ensuring the highest return on investment.
Hadoop Developer
Cerner Healthcare
Bengaluru
01.2017 - 04.2019
- The main purpose of developing the project is to generate useful data that can be consumed for Tableau reporting
- The report which need to be generate are from different subject areas like Surgery, Emergency Department, Infection Control, Oncology etc which deals with huge amount of data
- We are developing an ETL project from scratch where we are taking structured/semi-structured data either from Oracle Db or HDFS where we are implementing MapReduce using framework called Apache crunch and flatting the source data with the help of Hadoop technologies and storing the data in Vertica
- Working on developing crunch transformation which have both structure and semi structure data (Oracle or HDFS) as a source and storing it to either intermediate HDFS or Vertica
- Involved in Design Reviews and Code Reviews
- Coded and executed test clients for various features and implemented Junit to test the unit level functionalities
- Played Role of virtual lead and Architect
- Developed the entire project from scratch and deployed to Prod
Big Data Developer
Cerner Healthcare
Bengaluru
01.2017 - 04.2019
- The main purpose of developing the project is to verify all the data are correctly processed and inserted into HBASE which will be used by the downstream project to process the ETL job
- We are making use of the Apache Storm to push the data into HBase
- We are creating two table namely Standardized and Normalized for each subject area (Encounter, finance, Person)
- The data we are processing is near Real-time which means each business day
Application Developer
Envestnet Yodlee
Bengaluru
07.2015 - 12.2016
- Company Overview: Client: WellsForgo, Chase, City, JP Morgan, UBS and many more financial institutes
- Scrapping or web crawling and putting all financial data together in a single UI
- This APP is going to scrap all important financial data by creation of AGENTS (automated BOT) which will crawl data from respective financial websites and dump data in xml format in backend where the data get stored in DB
- Client: WellsForgo, Chase, City, JP Morgan, UBS and many more financial institutes
- Worked on creation on new agent from scratch to scrap data from website and dump in XML
- Worked on Enhancements, Post Production, QA defect fixes
- Involved in Design Reviews and Code Reviews
- Good knowledge of Agile methodology
Education
Bachelor of Technology - COMPUTER Science
Om Dayal Group of Institution
06.2011 - 06.2015
High School - undefined
VMHS
06.2009 - 05.2011
Skills
Java 6/7/8
SQL
HDFS
Oozie
Kafka
Scala
Spark
Hive
TeraData
SOLR
Python
Airflow
GCP
Postgres
Timeline
Senior Data Engineer
Walmart
10.2020 - Current
Big Data Developer
True Fit
06.2019 - 10.2020
Hadoop Developer
Cerner Healthcare
01.2017 - 04.2019
Big Data Developer
Cerner Healthcare
01.2017 - 04.2019
Application Developer
Envestnet Yodlee
07.2015 - 12.2016
Bachelor of Technology - COMPUTER Science
Om Dayal Group of Institution
06.2011 - 06.2015
High School - undefined
VMHS
06.2009 - 05.2011
Similar Profiles
Lori VayoLori Vayo
Customer Service Representative at WalmartCustomer Service Representative at Walmart
SHAKERRA JACKSONSHAKERRA JACKSON
Lift Operator at WalmartLift Operator at Walmart
SHEHNAZPREET KAURSHEHNAZPREET KAUR
Customer Service Representative at WalmartCustomer Service Representative at Walmart