Summary
Experienced Senior Software Engineer with 4+ years' experience, working at Euclid Innovations. Proficient in Azure Data Bricks, Datawarehouse, and SQL, with a track record of delivering high-quality projects and meeting project deadlines. Experienced in Application Development in Big Data Hadoop Environments like HDFS, HIVE, SQOOP, PySpark in Databricks and Azure Data factory, storage like ADLS, blob.
I am passionate about collecting, processing, and storing data. Designing, testing, and maintaining data management systems and processes, and analyze databases. The ultimate goal is to make data accessible so that organizations can use it to evaluate and optimize the performance.
Overview
4
4
years of professional experience
1
1
Certification
Work History
Senior Software Engineer
EUCLID INNOVATIONS PVT LTD
10.2023 - Current
- Input is the different teams will dump the data in a common location called Sharepoint
- Input format of input files are usually CSV, excel and Json
- Worked on using spark data Frame API to complete data manipulation within spark Session
- Inside the Jupyter notebooks we can write transformation logic by using PySpark and schedule that notebooks for execution
- From the data lake we create pipelines by applying logics and transformations required for the business
- From the data lake we use transformed data to load it into different locations in the data lake which will be in Avro & Parquet formats
- These transformed files will be used by data science team to create reports or dashboards in Tableau visualization tool.
Software Engineer
COGNIZANT TECHNOLOGY SOLUTIONS
12.2021 - 03.2023
- Extract Transform and Load data from Azure Blob storage and processing the data in Azure Databricks
- Ingest Customers Transaction data into the ADLS location by using Spark
- Worked on using spark data Frame API to complete data manipulation within spark Session
- The business logic will be done in Azure Databricks
- Developing Spark code in python and Spark-SQL environment for faster testing and processing of data and loading the data into Spark RDD and doing In-memory computation to generate the output response with less memory usage.
Software Engineer
ACERT IT SOLUTIONS PVT LTD
01.2020 - 11.2021
- Worked on a live 30 node Hadoop cluster running on CDH 5.1
- Involved in development and testing of various module of the product
- Involved in the first level analyzing of defect raised by users
- Providing the queries, procedures, and functions to the users according to requirements
- Lead problem-solving and resolution efforts by assisting with issue identification, resolution, documentation, and escalation.
Education
B. Tech -
Skills
Azure data factory
blob storage
data lake
Azure Databricks
Azure Active Directory
ADLS
Python
Spark-SQL
Airflow
Crontab
Pyspark
Hadoop
Hive
Bigdata
Spark
SQL
Jira
Jenkins
Spark Core
Certification
Microsoft Certified: Azure Data Engineer Associate, https://learn.microsoft.com/api/credentials/share/en-us/NikileshwaraTK3810/14E3872DF019FB4F?sharingId
Timeline
Senior Software Engineer - EUCLID INNOVATIONS PVT LTD
10.2023 - Current
Software Engineer - COGNIZANT TECHNOLOGY SOLUTIONS
12.2021 - 03.2023
Software Engineer - ACERT IT SOLUTIONS PVT LTD
01.2020 - 11.2021
Visveswaraiah Technological University (VTU) - B. Tech,