Aman Mishra
Kolkata
Summary
Hardworking individual with a knack of learning new technologies having 3 years of Experience in Big Data & Analytics as a developer using skills such as Azure Databricks,Azure DataFactory,Pyspark,Hadoop,Hive & MySql
Overview
4
4
years of professional experience
Work History
Data Analyst
MOL Information Technology
Kolkata
2023.05 - Current
- Retrieve semi-structured data from various divisions within MOL, such as Dry Bulk, Shipping, Liner, Pure Carrier, LNG, Phoenix Tanker, etc., in formats like CSV, JSON, Excel, Parquet, etc., employing diverse methods like REST API calls and Email Attachments through Azure Logic App. Store this data in Azure Data Lake Storage.
- Subsequently, leverage PySpark on Azure Databricks to read the data stored in Azure Data Lake Storage and perform necessary transformations as required.
- Establish a connection to the MOL Pearl Database (SQL Server) from Azure Databricks to process the data, loading it into equivalent transaction and master tables in the Staging Layer within the MOL Pearl Database.
- Develop stored procedures to facilitate the movement of data from the Staging Layer to the Standardized Layer.
- Construct views to efficiently access data from the Standardized Layer
- Develop Azure Data Factory (ADF) ETL Pipelines to orchestrate the aforementioned activities from the source to the standardized layer..
- Finally, enable the BI team to connect their data sources in Tableau/Power BI to generate reports essential for analytics and to facilitate client-specific business decisions for the MOL Group of Companies.
Senior Analyst
Exusia
Kolkata
2022.11 - 2023.05
- Optum - CPP Azure Cloud Migration Migrating the onsite/on premise health care data environments on databases Netezza, Oracle, SQL Server as well as the data on the Abinitio ETL tool to the Azure cloud using tools like Databricks, Scala, Pyspark and Azure Datafactory Building the equivalent spark code using Scala, Pyspark in databricks for the data/graph provided by the abinitio ETL team or the Netezza, Oracle, SQL Server team
- Running the comparator notebook on databricks in order to perform the full load testing with the on premise input files
- Building the ADF pipelines for the code built and processed in databricks and analyzing the dependencies for the input files with the cloud data source stored in ADLS Data lake Storage.
Project Engineer
Wipro Limited
Kolkata
2020.09 - 2022.11
- Digi Malaysia - Telenor Global Working as a member of the Development team for DigiMalaysia project for processing raw, structured and unstructured data using certain ETL and DB tools like DB Visualizer and My SQL workbench for 3 tier data warehousing project
- Based on the behaviour of data flow patterns, data from source was transformed and integrated into the data warehouse
- Apache Hive and Vertica were used to hold the migrated data
- Development and Unit Testing of Hive and Vertica queries of different sprints for different SCD types as per the SMX mapping provided by the Data Modelling team for Phase 1-A and Phase 1-B
- Migrating the On Premises data on the hadoop cluster setup to the Azure Cloud Setup and processing and analyzing the data using Azure Databricks
- Provided appropriate justifications for the JIRAs raised by the client testing team during SIT/UAT and making the necessary code modifications and optimizations
- Development and Unit Testing of various dimension and fact tables for tier 3 layer
- Worked on Production dataloading activity.
Intern
Wipro Limited
Kolkata
2020.06 - 2020.09
- Wipro Pre Joining Program (PJP Training) on Big Data Technologies such as Azure Databricks, Pyspark, Scala Spark, Azure Datafactory, Hadoop, Hive, Sqoop and MySql
Education
Bachelor Of Technology - Electronics & Communication Engineering
University Of Engineering & Management,Kolkata
04.2020
Computer Science -
Mansur Habibullah Memorial School
04.2016
General -
Mansur Habibullah Memorial School
04.2014
Skills
- Azure Databricks
- Azure DataFactory
- Pyspark
- Scala Spark
- Apache Hadoop
- Apache Hive
- MySql
- Apache Scoop
- Vertica
- SQL Server Management Studio
Accomplishments
- Edureka Online Course, 01/2021, 04/2021, Attended Edureka Online Training on Azure Databricks, Azure DataFactory, Pyspark, Apache Hive, Apache Sqoop & Apache Hadoop
- Trendy Tech Online course, 07/2021, 01/2022, Attended Azure Databricks, Azure DataFactory, Pyspark, Apache Hive, Apache Sqoop & Apache Hadoop & MySql tutorial onTrendy Tech Insights
- Udemy Certification, 04/2022, 07/2022, Udemy certification on Real World Project on formula1 using Azure Databricks.Azure Datafactory,Azure Datalake,Data Warehousing and Lakehouse architecture (Jan 2022 -April 2022)
Personal Information
Title: Senior Analyst - Azure Data Engineer
Languages
- English, Native or Bilingual Proficiency
- Hindi, Native or Bilingual Proficiency
- Bengali, Full Professional Proficiency
Hobbies and Interests
- Gym
- Cricket
- Professional Wrestling
Projects
Bharat Sanchar Nigam Limited(BSNL) Circle Telecom Training Centre(CTTC), 06/2018, 07/2018, Kolkata, West Bengal, Summer Vocational Training on Advanced Level Telecommunication system and Networking UEM,Kolkata, 07/2019, 04/2020, Kolkata, West Bengal, Early Earthquake Warning System
Timeline
Data Analyst
MOL Information Technology
2023.05 - Current
Senior Analyst
Exusia
2022.11 - 2023.05
Project Engineer
Wipro Limited
2020.09 - 2022.11
Intern
Wipro Limited
2020.06 - 2020.09
Bachelor Of Technology - Electronics & Communication Engineering
University Of Engineering & Management,Kolkata
Computer Science -
Mansur Habibullah Memorial School
General -
Mansur Habibullah Memorial School
Similar Profiles
Siddhartha DattaSiddhartha Datta
Principal Associate- Database Manager at MOL Information Technology Private LimitedPrincipal Associate- Database Manager at MOL Information Technology Private Limited
Shakibu Mussa NsekelaShakibu Mussa Nsekela
Senior Accounts Officer at Ministry of Information, Communication and Information TechnologySenior Accounts Officer at Ministry of Information, Communication and Information Technology
Hao Yuan (Mark) ChenHao Yuan (Mark) Chen
Research Engineer at Institute for Information Industry (III) / Taiwan Association of Quantum Computation and Information Technology (TAQCIT)Research Engineer at Institute for Information Industry (III) / Taiwan Association of Quantum Computation and Information Technology (TAQCIT)