Gunjan Gawande
Summary
Results-driven data engineer with expertise in designing and deploying scalable data solutions. Proficient in large-scale data engineering using Databricks and AWS services, with a strong foundation in data modeling and warehouse architecture. Skilled in analyzing business needs and translating them into actionable data strategies. Experience in optimizing performance and collaborating with stakeholders to align technical requirements with organizational goals.
Overview
4
4
Languages
1
1
Certification
8
8
years of professional experience
Work History
Senior Data Engineer at State Street
Tata Consultancy Services
Pune
04.2024 - Current
- Designed and implemented a reusable metadata-driven Medallion Architecture framework using Databricks, PySpark, Spark SQL, and Delta Lake to build scalable data ingestion, transformation, and orchestration pipelines, reducing development effort and standardizing data processing across multiple projects.
- Led the end-to-end delivery of enterprise data engineering solutions, driving solution architecture, technical design, code quality, performance optimization, and adherence to engineering best practices.
- Designed and optimized PySpark and Spark SQL workloads by implementing efficient joins, partitioning strategies, caching, and query optimization techniques to improve pipeline performance and resource utilization.
- Implemented Unity Catalog to establish centralized data governance, role-based access control (RBAC), metadata management, and secure data access across Databricks environments.
- Implemented robust data quality, validation, and monitoring frameworks to ensure data accuracy, consistency, reliability, and compliance across enterprise ETL pipelines.
- Partnered with business stakeholders to gather and analyze requirements, translating business needs into scalable, cost-effective, and production-ready data engineering solutions.
- Mentored data engineers through technical design discussions, code reviews, debugging support, and knowledge-sharing sessions, promoting engineering best practices and maintainable solutions.
- Collaborated with cross-functional teams, product owners, and stakeholders to prioritize technical deliverables, resolve architectural challenges, mitigate delivery risks, and ensure successful project execution.
- Performed root cause analysis for complex production issues, implementing performance optimizations and preventive measures to improve pipeline stability, reliability, and operational efficiency.
- Continuously identified automation opportunities and implemented process improvements to enhance developer productivity, reduce manual effort, and improve overall engineering efficiency.
Data Engineer at State Street
Tata Consultancy Services
Pune
04.2021 - 03.2024
- Designed and implemented scalable data transformation pipelines using Databricks Declarative Pipelines, PySpark, and Spark SQL, integrating data from multiple source systems through complex joins, transformations, and validation logic to deliver production-ready datasets.
- Developed and maintained a standalone Python desktop application using Pandas, NumPy, and Tkinter to automate offline data ingestion, transformation, validation, and report generation, reducing manual effort and improving data processing efficiency.
- Built reusable ETL frameworks and modular components to standardize data ingestion and transformation workflows, improving maintainability, code reusability, and development efficiency across the team.
- Optimized large-scale data processing using Spark SQL and advanced SQL techniques, writing high-performance queries, implementing complex joins, and improving query execution on high-volume datasets.
- Integrated and reconciled data from diverse source systems, ensuring data quality, consistency, and availability for downstream analytics and business reporting.
- Collaborated with business stakeholders and cross-functional teams to translate functional requirements into scalable, production-ready data engineering solutions and ETL pipelines.
- Automated, scheduled, and monitored ETL workflows using Autosys, ensuring reliable execution, timely data availability, and proactive issue resolution in production environments.
Python/Java Automation Engineer at Banco Pichincha
Tata Consultancy Services
Nagpur
12.2018 - 03.2021
- Developed automated reporting applications to process large datasets and deliver actionable insights for business stakeholders.
- Developed Python- and Java-based automation solutions to clean, transform, and prepare raw data for reporting, significantly reducing manual effort and improving data accuracy.
- Designed and executed transactional data validation workflows using Java to ensure data reliability across systems.
- Automated application availability checks and developed end-to-end test scripts to enhance system stability and minimize operational overhead.
- Integrated automation workflows with existing enterprise systems to streamline processes and boost overall productivity.
- Developed reusable Python modules for data parsing, transformation, and validation to standardize processing across different teams.
- Created monitoring scripts to detect failures, performance bottlenecks, and data inconsistencies, improving system observability.
- Collaborated closely with QA, development, and operations teams to ensure smooth execution of automated processes in production.
- Contributed to requirement analysis and helped convert business rules into automation logic.
Education
B.E. - IT
Prof Ram Meghe Institute of Technology and Research
Badnera, AmravatiSkills
TECHNICAL SKILLS
Programming & Scripting: Python, SQL, Unix Shell Scripting
Big Data Technologies: Apache Spark, PySpark, Apache Hive, Hadoop (HDFS)
Databricks & Lakehouse: Databricks, Spark SQL, Delta Lake, Unity Catalog, Declarative Pipelines (Delta Live Tables), Databricks Workflows
Cloud Platforms: Microsoft Azure, Amazon Web Services (AWS), Google Cloud Platform (GCP)
Data Engineering: ETL/ELT Pipeline Development, End-to-End Data Pipeline Development, Data Ingestion, Data Transformation, Data Modeling, Data Quality, Data Validation, Data Governance, Data Warehousing
Architecture & Design: Medallion Architecture, Metadata-driven Framework Design, Enterprise Data Architecture
Workflow Orchestration: Apache Airflow, Autosys
DevOps & Version Control: Git, GitHub, GitLab, Jenkins, Docker
Certification
- Microsoft Certified: Azure Databricks Data Engineer Associate (DP-750)
- Databricks Certified Data Engineer Associate
- Microsoft Certified: Azure Data Fundamentals (DP-900)
- Microsoft Certified: Azure Fundamentals (AZ-900)
- Google Generative AI Leader Certification
- HackerRank SQL Certificate
HackerRank Python Certificate - Python for Data Science and Machine Learning (Udemy)
Languages
English
Proficient (C2)
C2
spanish
Elementary (A2)
A2
Hindi
Proficient (C2)
C2
Marathi
Native
Native
Languages
- English
- Hindi
- Marathi
Timeline
Senior Data Engineer at State Street
Tata Consultancy Services
04.2024 - Current
Data Engineer at State Street
Tata Consultancy Services
04.2021 - 03.2024
Python/Java Automation Engineer at Banco Pichincha
Tata Consultancy Services
12.2018 - 03.2021
B.E. - IT
Prof Ram Meghe Institute of Technology and Research