Nand Kumar Jha
Data Engineer
Banglore
Summary
Accomplished Data Engineer with a passion for delivering valuable data using Big data technologies. Bringing forth a proven track record of analyzing complex data sets and serving as a strong advisor. Committed to working as a collaborative and positive team member, striving to utilize my knowledge and expertise for optimal engineering results.
Overview
4
4
years of professional experience
4
4
years of post-secondary education
2
2
Certifications
Work History
Senior Data Engineer
Robert Bosch Engineering And Business Solutions
Banglore
10.2020 - Current
- Csv files of car diagnostic data was provided by customer, performed ETL operation on csv data from s3 to Rds,athena and s3
- Contribution:
- Performed ETL operation on csv data
- Performed quality check on data(null, empty)
- Performed pyspark transformation to form structured data from semi structured data
- Provided insights from data and created view and tables in athena
- Tools/Technologies Python, Pyspark, AWS services(Athena,S3,Glue, Lambda,Cloudwatch)
Software Engineer, Data Engineer
Robert Bosch Engineering and Business Solutions Private Limited
Banglore
07.2018 - 09.2020
- Development of multiple configuration files as per different stations in different plants using complex spark sql queries and spark structure streaming.
- Implemented features like jceks, shutdownhook for application.
- Development of deployment script for VASAPI using shell script
- Testing of multiple pipelines of VASAPI on development cluster on load, correctness, and integration.
- Deployment of multiple VASAPI pipeline in development environment and production
- Writing Scala code for different schemas specific to machines and plants using spark involving transformation and writing into parquet formats.
- Deployment of J2T with various processes in development environment and production of different clusters
- Developed compression technique for json files to be compressed in snappy, gzip, lzo, json compression type (Spark utilities).
- Integrated merging process of spark utilities to manufacturing execution system batch process for small files in multiple clusters.
Education
Bachelor Of Technology - Computer Science And Engineering
National Institute of Technology Sikkim
Ravangla, Sikkim 08.2014 - 05.2018
Skills
Java, Scala, Python
Python, PHP, Shell, Json, XML
MySQL
Windows, Linux
Hadoop,HDFS, IBM BigInsights, Hive,No SQL: Hbase, MapReduce, Pig, BigSQL
Zookeeper,Spark2X(Core and Sql), Pyspark,Spark Structure StreamingSqoop, Solace, ImpalaGit, Jenkins
Microsoft Azure, Amazon Web Services (AWS)
Scikit-Learn, Numpy, Scipy, Pandas, NLTK, BeautifulSoup
Matplotlib, Jupyter NotebookStatistics/ML : Statistics Analysis, Linear/Logistics Regression, ClusteringCertification
Azure Data Engineer Associate
Timeline
Azure Data Engineer Associate
12-2020
Senior Data Engineer
Robert Bosch Engineering And Business Solutions
10.2020 - Current
RBEI-Nipun: ML Certified
08-2020
Software Engineer, Data Engineer
Robert Bosch Engineering and Business Solutions Private Limited
07.2018 - 09.2020
Bachelor Of Technology - Computer Science And Engineering
National Institute of Technology Sikkim
08.2014 - 05.2018