Sufi Pathan
Hyderabad
Summary
Data engineer with over 3+ years of experience in ETL development, data processing, and cloud technologies. Proficient in SQL, Python, GCP Servies and AWS services including S3, Redshift, and EMR. Skilled in building scalable data pipelines, monitoring workflows, and implementing CI/CD processes. Strong expertise in data quality and transformation, with a focus on end-to-end data processing.
Overview
6
6
years of professional experience
Work History
Data Quality Engineer
GENPACT
05.2025 - Current
- Analyze source-to-target data flows and transformation logic to validate data accuracy, completeness, and integrity.
- Perform end-to-end data validation and testing using SQL across source, conformance, curated, consumption, and Cloud SQL layers.
- Work with data pipelines and ETL processes to identify data issues and validate business transformation rules.
- Translate business requirements into SQL validations and data quality checks; identify and document defects, tracking them through Jira.
- Documented and tracked 15+ defects across multiple user stories, enhancing data quality and product reliability.
- Develop and enhance Python-based automation scripts to reduce repetitive manual testing activities and improve testing efficiency.
- Utilised SQL, Python, automation frameworks, AWS, Databricks, and ETL processes in cloud-based data environments.
Data Engineer
INFOCEPTS
05.2024 - 09.2024
- Developed and supported ETL pipelines using Python, PySpark, Spark, and SQL for data processing and integration.
- Collaborated with cross-functional teams to develop ETL pipelines, ensuring adherence to requirements tracked through Jira.
- Worked with AWS S3 and Amazon Redshift for cloud-based data storage and processing.
- Worked with Databricks and Amazon EMR for monitoring and execution of scheduled data processing jobs.
- Used Control-M for job scheduling and production job monitoring.
- Performed data filtration, cleansing, transformation, and manipulation to prepare data for target data warehouse systems.
- Created data mappings for dimensions and data segregation to align with business requirements.
- Used Amazon Athena for data analysis and query-based validation.
- Contributed to BI migration project by facilitating data integration and migration activities to improve data accessibility.
Support / Data Engineering
INFOCEPTS
03.2024 - 04.2024
- Supported data engineering teams by providing resources for SQL, AWS, Databricks, and job monitoring, ensuring smooth production activities.
- Monitored data processing jobs and identified troubleshooting steps for data pipeline issues, contributing to timely resolutions.
- Collaborated with project teams to clarify data-related issues and facilitated resolution efforts through effective support.
- Assisted in the development of business intelligence documentation for project clarity. and reporting-related documentation and project deliverables.
Data Engineer
INFOCEPTS
03.2021 - 02.2024
- Developed scalable Python-based data processing code for efficient data transformation and integration.
- Built and maintained ETL pipelines to ingest, transform, and load data from multiple sources into Amazon Redshift.
- Developed data processing solutions using Python, PySpark, SQL, and ETL technologies.
- Developed GitLab CI/CD pipelines to automate data ingestion, movement, and transformation workflows.
- Created CI/CD workflows using GitHub Actions to support continuous integration and deployment activities.
- Implemented data quality checks and validations across AWS and Databricks environments, ensuring data accuracy and integrity.
- Monitored production jobs using Databricks, AWS Athena, EMR, S3, Control-M, and Crontab.
- Developed SQL queries for data analysis and validation based on business requirements and Jira tickets.
- Used Linux environments for executing data-related tasks, troubleshooting, and source-to-target synchronization activities.
Education
B. Tech - Computer Science and Engineering
GH Raisoni College of Engineering
Nagpur, MH01-2022
Diploma - Computer Engineering
Govenment Polytechnic
Nagpur, MH01-2016
Skills
- SQL
- Python
- ETL
- Data Pipelines
- Data Transformation
- Data Cleansing
- Data Validation
- Data Integration
- Data Warehousing
- AWS
- Amazon S3
- Amazon Redshift
- Amazon EMR
- Databricks
- GitLab CI/CD
- ETL development
- Jira
- Production Support
- GCP
Career Break
Medical Reasons & Professional Upskilling:
Took a planned career break for personal medical reasons. Utilized the period for professional upskilling and strengthening knowledge in data engineering, Python, SQL, cloud technologies, and data processing concepts.
Timeline
Data Quality Engineer
GENPACT
05.2025 - Current
Data Engineer
INFOCEPTS
05.2024 - 09.2024
Support / Data Engineering
INFOCEPTS
03.2024 - 04.2024
Data Engineer
INFOCEPTS
03.2021 - 02.2024
B. Tech - Computer Science and Engineering
GH Raisoni College of Engineering
Diploma - Computer Engineering
Govenment Polytechnic