Pranjal Agawane
Pune
Summary
Driven by a passion for data engineering, I excelled at Tredence Analytics Solutions, where I spearheaded the implementation of the Medallion Architecture, enhancing data processing efficiency. My expertise in SQL, Python, and PySpark, coupled with my leadership in automating ETL processes, showcases my technical acumen and proactive problem-solving skills.
Overview
5
5
years of professional experience
Work History
Associate Manager
Tredence Analytics Solutions Pvt Ltd
Pune
05.2024 - Current
- Implemented the multi-layer Medallion Architecture (Bronze, Silver, Gold) for efficient data processing, transformation, and storage.
- Gathered and analyzed business requirements to create customized, business-driven Gold Layer Views
- Led the ingestion of raw data in CSV format from Google Cloud Storage (GCS) into the Bronze layer, followed by transformation and cleansing in the Silver layer, ensuring business-ready views in the Gold layer.
- Automated the ETL (Extract, Transform, Load) processes using SQL scripts orchestrated by Apache Airflow, ensuring smooth, reliable workflows and job monitoring.
Senior Engineer
Ascendion Engineering Pvt Ltd
Bengaluru
10.2022 - 05.2024
- Migrated data from on-prem Hive to Google Cloud BigQuery.
- Refactored existing on-prem Hive SQL queries to be compatible with Google BigQuery, optimizing performance and ensuring compatibility with GCP.
- Updated and optimized existing PySpark code to run efficiently on GCP Dataproc.
- Created and managed Apache Airflow DAGs to automate the testing and execution of SQL scripts.
- Transformed traditional Shell scripts into Airflow DAGs, automating complex processes.
- Developed a data validation tool to compare migrated data in BigQuery with on-prem data
Software Engineer
ABC
Pune
10.2019 - 10.2022
- Developed and optimized PySpark code to perform large-scale data transformations and data cleansing.
- Created SQL scripts as per client's requirements.
- Created Airflow Dag to automate ETL process in GCP.
- Created Python scripts to interact with various Google Cloud APIs such as BigQuery, Dataproc, Cloud Storage.
Education
B.E. -
Sinhgad College of Engineering
Pune06-2018
Skills
- SQL
- Python
- PySpark
- Hive
- BigQuery
- Airflow(GCP Composer)
- Dataflow
- PubSub
- Dataproc
- Databricks
- Azure Data Factory
- Delta Live Table
Accomplishments
- Champion Award for Q3-2024 and Q4-2024 in Ascendion Endingeering
Timeline
Associate Manager
Tredence Analytics Solutions Pvt Ltd
05.2024 - Current
Senior Engineer
Ascendion Engineering Pvt Ltd
10.2022 - 05.2024
Software Engineer
ABC
10.2019 - 10.2022
B.E. -
Sinhgad College of Engineering
Similar Profiles
Rani AherRani Aher
Sr Manager Architect at VIRTUSA PVT. LTDSr Manager Architect at VIRTUSA PVT. LTD
ADARSH MADARSH M
Consultant at Tredence Analytics SolutionsConsultant at Tredence Analytics Solutions
Deepa CTDeepa CT
PMO at Tredence Analytics Solution Pvt LimitedPMO at Tredence Analytics Solution Pvt Limited