
Experienced data engineer with 3 years of expertise in designing and implementing robust data solutions using cutting-edge technologies. Skilled in Python, Tableau, SQL, Terraform, and AWS services including Glue, S3, Athena, Airflow, and Databricks. Excels in optimizing performance, driving efficiency, and communicating complex insights effectively. Passionate about continuous learning and innovation.
ETL Process Design: Designed and implemented robust ETL processes for diverse source systems, ensuring seamless data integration.
AWS Glue Expertise: Leveraged AWS Glue for data transformation and integration, optimising data handling and enhancing automation.
Infrastructure Management with Terraform: Proficiently used Terraform to create and manage AWS infrastructure, enabling scalability and resource provisioning efficiency.
Efficient Notification Systems: Implemented AWS SNS and Event bridge for timely error and success notifications, ensuring effective workflow management.
API Data Extraction: Extracted data from various source systems through REST and SOAP APIs, ensuring data accessibility for analysis.
Proactive Error Resolution: Demonstrated a proactive approach to debugging and error resolution, leading to improved report accuracy.
Tableau Dashboard Design: Skilfully designed interactive Tableau dashboards, seamlessly integrating data from multiple sources, including custom SQL queries and data sources like Athena.
Apache Airflow for Workflow Management: Proficiently utilised Apache Airflow to create schedules and configure and manage DAGs, enhancing workflow automation and orchestration.
Python
Postgresql
AWS S3
AWS Glue
Pyspark
Databricks
Tableau
Amazon Redshift
Terraform
Octopus