Sohail Magdum
Senior Data Engineer
Pune,Maharashtra
Summary
Data Engineer with 4+ years of experience in designing and developing efficient ETL processes and robust data pipelines. Proficient in optimizing data workflows to enhance performance and reliability. Strong expertise in PySpark and Databricks for scalable data processing. Skilled in leveraging cloud services from Azure and AWS to deliver seamless data solutions. Adept at collaborating with cross-functional teams to drive data-driven decision-making and improve operational efficiencies. Committed to continuous learning and staying updated with the latest industry trends.
Overview
5
5
years of professional experience
2
2
years of post-secondary education
1
1
Certification
3
3
Languages
Work History
Senior Data Engineer
DataEaze System
Pune
11.2022 - Current
- Led and contributed to multiple data engineering projects across diverse domains, including [finance, healthcare, e-commerce, etc.].
- Led end-to-end data pipeline development, including the extraction, transformation, and loading of data from multiple sources.
- Worked with cutting-edge technologies like [ PySpark, Databricks ,AWS, Azure , Apache Kafka, SQL.] to build scalable and high-performance data solutions.
- Actively collaborated with data analysts, and other stakeholders to define and implement data requirements, ensuring alignment with business objectives.
- Led data engineering engagements, guiding a team of engineers and ensuring the delivery of high-quality solutions within deadlines.
- Oversaw the optimization of data storage, ensuring efficient querying, indexing, and data retrieval to support reporting and analytics.
- Optimized data pipelines and storage solutions, resulting in improved performance and reduced processing time, enabling faster access to critical business data.
- Designed and implemented efficient data models to support business intelligence and reporting requirements, ensuring data consistency, accuracy, and scalability.
ETL Developer
Tata Consultancy Services
08.2020 - 11.2022
- Worked on an ETL project where I was responsible for handling ad hoc queries to extract, transform, and load data into the data warehouse.
- Collaborated with cross-functional teams to analyze and understand business requirements, ensuring data was structured and delivered in a manner that met business needs.
- Worked with databases such as [mention the technologies like SQL, Oracle, etc.] to execute complex queries for data extraction and manipulation.
- Developed and optimized ETL processes to improve data quality, accuracy, and efficiency.
- Performed data validation and troubleshooting of ETL processes to ensure smooth data flow.
Education
Master of Science (MSC) - Computer Science(CS)
Modern College
Pune, Maharashtra 01.2020 - 01.2022
Skills
Python
Scala
PySpark
Databricks
Apache Airflow
Kafka
MQTT
AWS Glue
undefinedCertification
Databricks Certified Data Engineer
Timeline
Senior Data Engineer
DataEaze System
11.2022 - Current
ETL Developer
Tata Consultancy Services
08.2020 - 11.2022
Master of Science (MSC) - Computer Science(CS)
Modern College
01.2020 - 01.2022
Similar Profiles
Siddhant KadamSiddhant Kadam
Data Scientist at Dataeaze systems PuneData Scientist at Dataeaze systems Pune
Umer SayedUmer Sayed
Senior BI Developer at Dataeaze SystemsSenior BI Developer at Dataeaze Systems
Rebecca Bates CPCRebecca Bates CPC
Certified Professional Coder at Augusta University Health System / Wellstar Health SystemCertified Professional Coder at Augusta University Health System / Wellstar Health System
Shyam KhamkarShyam Khamkar
Unit Manager - Data Science and Analytics at Bajaj FinservUnit Manager - Data Science and Analytics at Bajaj Finserv
Amol SonawaneAmol Sonawane
Accounts Payable Analyst at AGCO Trading (India) Private Limited Pvt LtdAccounts Payable Analyst at AGCO Trading (India) Private Limited Pvt Ltd