I am a highly skilled Data Engineer with over 3+ years of experience in Python, SQL, PySpark and AWS. I specialize in data extraction, processing, analysis, and visualization, using tools like DataBricks, GitLab, AWS Redshift, Glue and Athena. My expertise in Python libraries like Pandas and NumPy coupled with Tableau and QuickSights for data visualization, enables me to provide valuable insights. I excel in version control with Git, fostering efficient collaboration. Proficient in SQL, ETL and feature engineering, I thrive in dynamic, fast-paced environments and am committed to continuous learning and innovation.
CMOTS Project: CMOTS (Comprehensive Market Overview and Trading System) is a sophisticated financial application designed to provide real-time market data, analytics, and trading capabilities.
The application serves various stakeholders in the financial industry, including traders, analysts, and portfolio managers, by delivering up-to-date market information and enabling efficient trade execution.
It integrates multiple data sources, processes vast amounts of data, and supports advanced analytics to enhance decision-making.
Project Goal: The goal of the CMOTS project is to develop a robust and scalable platform that integrates real-time market data, facilitates comprehensive market analysis, and supports both automated and manual trading operations to enhance decision-making for financial industry stakeholders.
Responsibilities: As a Data Engineer in the CMOTS project handles ETL processes, optimizes databases, implements data processing frameworks, ensures data quality and compliance, supports analytics, optimizes pipelines for scalability, ensures security, automates deployments, monitors system health, and collaborates with cross-functional teams.
Key Technologies: Python, PySpark, SQL, AWS, Databricks, GitLab, Jira, Agile
Electric Vehicle Project: The Electric Vehicle Dashboard project focuses on tracking the total number of EV charging tools or stations, which are groups of charging stations aggregated based on common properties
It also involves managing Electric Vehicle Supply.
Equipment (EVSE) to facilitate energy exchange between EVs and the grid, including slow chargers, fast chargers, and battery switch stations
Project Goal: The project aims to make EVSE dynamic and responsive to real-time needs
Responsibilities: Data retrieval, transformation, AWS Athena database management, GitLab data pipeline scheduling, Tableau dashboard creation, ongoing maintenance, real-world testing
Key Technologies: Python, SQL, AWS, Jira, GitLab, Agile
Kohl's - Stores Cloudification Project: Enhancing customer experience and increasing sales at Kohl's stores through data analytics
Focus on understanding customer behavior and product performance optimization
Responsibilities: Data transformations, AWS for storage, PySpark and SQL for processing, Databricks for scalability, Azure DevOps/Git for collaboration, Agile methodology, PySpark for efficient data handling
Project Goal: Leverage data analytics to improve customer experience and boost sales
Key Technologies: Python, SQL, PySpark, AWS, Databricks, Azure DevOps, Git, Agile
Outcome: Efficient data engineering process delivering value to customers and enhancing the shopping experience.