Nitin Chaudhary
Noida,India
Summary
● Developed Python scripts to transform raw data into intelligent data as specified by business user
● Worked closely with data modelers to model new incoming data sets
● Processed data into HDFS by developing solutions, analyzed the data using Pyspark, Hive/Impala/Kudu and produce results to downstream systems
● Developed Shell, Python scripts to automate the manual monitoring process for ETL jobs
● Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive.
● Reading ETL jobs data from Oracle using Pyspark and store into Hive/Impala
● Skilled in interacting with clients and coordinating with multiple stakeholders for data exchange
● Written Python script for Kafka Producer Consumer
● Possess functional knowledge of designing and developing applications in Spark using Scala to compare the performance of Spark with Hive
● Knowledge on Hadoop Ecosystem such as HDFS, Resource Manager, Node Manager, Name Node, Data Node and MapReduce program paradigm
● Insightful knowledge of loading and transforming large sets of Structured, Semi-Structured and Unstructured data and analyzed them by running Hive queries and Python Scripts
● Possess strong analytical and problem-solving skills; an effective leader with excellent skills in motivating individual employee performance
Overview
9
9
years of professional experience
3
3
years of post-secondary education
Work History
Teach Lead Data Engineer
Birlasoft
Noida, Uttar Pradesh
05.2020 - Current
Title: Falcon (Birlasoft - JNJ)
Role: Development, Coding, Testing
Domain: Big Data (Pharma)
Technologies: Spark, Python, Cloudera, Hive/Impala/Kudu, Oozie, Oracle
Synopsis:
Capturing logs using Pyspark/Shell Script from different applications like GSDL, GSDR, Safety etc. and storing in Hive/Impala. Also getting server availability and ETL job status to show data in Tableau Dashboard.
Logging.
Data Engineer
Condeco Software
Gurgaon, Haryana
01.2021 - 04.2021
Title: Galaxy (Condeco)
Role: Development, Coding, Testing
Domain: Big Data (Sales)
Technologies: Spark, Scala, Azure Synapse Studio, Azure Data Pipeline, SQL
Synopsis:
Build data pipelines for data ingestion from source (SQL) to destination (blob) and automate it. Applied transformations using spark and store into data warehouse for reporting. Also wrote Stored Procedure for custom
Logging.
Big Data Consultant
Capgemini
Gurgaon, Haryana
12.2019 - 01.2021
Title: AML Reconciliation (Capgemini – Citi Bank)
Role: Development, Coding, Testing
Domain: Big Data (Financial)
Technologies: Spark, Scala, Hive, CDH5, Oracle
Synopsis:
Data ingested to HDFS in parquet format and we get the data from HDFS and process it to different layers using Spark Dataframe/SQL API by applying some business rules. In last layer, creating reconciliation report for valid transactions.
Associate Programmer
Clavax Technologies
Gurgaon, Haryana
02.2017 - 10.2019
Title: Novus Loyalty Analytics (Clavax Technologies)
Role: Development, Coding, Testing
Domain: Big Data Analytics
Technologies: Java, Hadoop, MongoDB, PowerBI
Synopsis:
Novus is a loyalty program for RuPay debit card. In this project providing vouchers, points and other loyalty benefits to our customers on using RuPay card. Our clients are Banks, NPCI, Tempe Golf Range, BookMyShow and others. Data comes in different formats (TEXT, CSV, TSV, JSON, and XML) and stored in HDFS. Processed data using MapReduce and Hive. Power BI is used to prepare the report from data.
Software Developer
MBD Alchemie
Delhi, Delhi
01.2017 - 02.2019
Education
No Degree - Master of Computer Application (MCA)
IGNOU
Delhi 04.2001 -
Bachelor of Science - Computer Application
Apex Institute
Meerut 07.2007 - 07.2010
Skills
Spark, MapReduce, HDFS, Hive/Impala, Kudu, Oozie, Sqoop, Yarn, Python, Scala, Java , SQL, Cassandra, Java, Spring MVC, Hibernate, Flask, Eclipse, VSCode, NetBeans, IntelliJ, Azure, AWS
Timeline
Data Engineer
Condeco Software
01.2021 - 04.2021
Teach Lead Data Engineer
Birlasoft
05.2020 - Current
Big Data Consultant
Capgemini
12.2019 - 01.2021
Associate Programmer
Clavax Technologies
02.2017 - 10.2019
Software Developer
MBD Alchemie
01.2017 - 02.2019
Bachelor of Science - Computer Application
Apex Institute
07.2007 - 07.2010
No Degree - Master of Computer Application (MCA)
IGNOU
04.2001 -