Results-driven Data Engineer with experience at MuSigma Business Solutions, skilled in SQL and Python. Successfully optimized data processing and built automated pipelines, enhancing efficiency. Proven ability to analyze and visualize data, leading to significant cost savings and improved decision-making. Strong communicator with a focus on delivering impactful data solutions.
Data product creation | Geosteering | Fortune 15 Oil & Gas Company
· Integrated data from multiple sources and Created CI CD/CM approved data products for different entities
· Optimized the read transformation and write speeds by using different spark techniques
· Built automated daily refresh pipelines to track data changes and streamline data consumption
· Delivered: Delivered multiple CI CD/CM-approved data products by consolidating geological data from diverse sources into a centralized data lake; optimizing Spark performance; and ensuring reliable and efficient data processing.
· Tech Stack: Pyspark , Python , Databricks , Ansible – CI/CD
Unified Trading Solution | Fortune 15 Oil & Gas Company
· Utilized Azure Data Factory and Azure Databricks for Extract Transform Load (ETL)
· Designed and implemented a high-performance accurate SQL database with automatically refreshes from external sources into Databricks for transformation
· Built automated daily data refresh pipelines and deployed Databricks workflows using integrated Ansible for streamlined orchestration management
Optimized resource allocation dynamic React-based UI with a Python middleware fetching the data from a SQL database; and implemented security features like Azure AD login and role-based access control
SQL
Python
Pyspark
React JS
GIT
Devops - CI/CD
Data cleaning
Data Visualization
Data Analysis
SQL
Python
Pyspark
React JS
GIT
Devops - CI/CD
Data cleaning
Data Visualization
Data Analysis