Years of relevant professional experience
Chetan Parishwad
Data Engineer Lead
Santa Clara,CA
Summary
Results-driven Data Engineer Lead with 9.5+ years of hands-on work experience and 6+ years of education in Computer Applications with Masters degree. Expertise lies in leading teams in building efficient ELT data pipelines in Hadoop Ecosystems
Overview
10
10
10
10
years of professional experience
Work History
Data Engineering Lead
Applied Materials (Tech Mahindra)
Santa Clara, CA
05.2019 - Current
- Revamped over 200 MS APS-based T-SQL stored procedures into PySpark, slashing critical base table processing times from over 24 hours to under 2 hours, ensuring up-to-date data for downstream
- Empowered downstream data science teams by crafting Python parsers to extract vital data from semi-structured KLARF files, a proprietary wafer defect reporting format utilized by KLA-Tencor, facilitating defect classification and analysis processes in Near Real-time
- Collaborating with stakeholders to grasp business requirements, and subsequently devising and executing solutions
- Efficiently oversee, delegate, and supervise team of 5 data engineers. Planning and organizing sprint cycles, Monitoring progress and ensuring timely completion of tasks and sprint goals
Senior Data Engineer
Tata Consultancy Services (TCS)
Hyderabad, India
08.2017 - 04.2019
- Developed an end-to-end framework on Spark for ELT, efficiently managing various data loads such as append, merge, CDC, and full. Streamlined common tasks across multiple Spark applications, resulting in a significant reduction in development time for other developers
- Recognized as subject matter expert (SME) in Spark and SQL, providing valuable assistance to peers in optimizing Spark job performance. Achieved remarkable performance improvements of up to 35x through effective tuning techniques
- Revamped and transferred scripts from Hive to Spark, followed by upgrade from Spark 1.6 to 2.1. These initiatives led to impressive 4x enhancement in runtime performance.
Azure Data Engineer
Accenture
Bangalore, India
08.2016 - 03.2017
- Developed and deployed data pipelines in Azure Data Factory to seamlessly move, transform, and integrate diverse data sources into Azure's datalake/blob storage, ensuring efficient and optimized data processing by partitioning and indexing
- Leveraged Azure services such as Azure Databricks and Azure HDInsight to drive data transformation, processing, and analytics initiatives using cutting-edge technologies like Spark, Hive, and SQL. Created reports in Power BI
Data Engineer
Accenture
Bangalore, India
02.2014 - 08.2016
- Developed ETL Pipelines utilizing Apache Crunch to extract data from HBase, perform KPI metric calculations, and seamlessly store results in HDFS. Additionally, proficient in bulk loading data into HBase for efficient data management
- Migrated Batch ETL Crunch pipelines to high-performance in-memory Spark Pipelines, resulting in remarkable 5x improvement in job runtimes
Education
Master of Computer Applications (MCA) (STEM) - Full-Time
R. V. College of Engineering (RVCE)
Visvesvaraya Technological University (VTU) 10.2011 - 2014.06
Bachelor of Computer Applications (BCA) - Full-Time
Karnataka University
Dharwad 08.2008 - 2011.07
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Quote
There is a powerful driving force inside every human being that, once unleashed, can make any vision, dream, or desire a reality.
Tony Robbins
Timeline
Data Engineering Lead
Applied Materials (Tech Mahindra)
05.2019 - Current
Senior Data Engineer
Tata Consultancy Services (TCS)
08.2017 - 04.2019
Azure Data Engineer
Accenture
08.2016 - 03.2017
Data Engineer
Accenture
02.2014 - 08.2016
Master of Computer Applications (MCA) (STEM) - Full-Time
R. V. College of Engineering (RVCE)
10.2011 - 2014.06
Bachelor of Computer Applications (BCA) - Full-Time
Karnataka University
08.2008 - 2011.07
Additional Information
Big Data : Spark, Hadoop, Hive, Impala, Kafka, MapReduce, Sqoop
Database : Spark SQL, Oracle, DB2, MS SQL Server, MySQL
Programming language : Python, Shell Scripting, C#, Scala, Java
Dev. Tools : PyCharm, GitHub, BitBucket, Jenkins, Notebooks, Putty, WinSCP
Cloud : Azure Data Factory, Data Lake, analytics, Databricks, DocumentDb
SDLC : AgileCentral, Jira, HP ALM, HP QC
Schedulers : Oozie, AirFlow, AutoSys, Redwood, NiFi
Visualization : PowerBI, Excel
Domain : Semiconductor, BFSI, Retail, Healthcare
Similar Profiles
Bhaskar Ramesh GiriBhaskar Ramesh Giri
IT Asset Manager at Tech Mahindra (Client Applied Materials)IT Asset Manager at Tech Mahindra (Client Applied Materials)
Ramkumar DondaRamkumar Donda
Software Engineer at Tech Mahindra - Applied Materials(Contingent Employee)Software Engineer at Tech Mahindra - Applied Materials(Contingent Employee)
Prarun T GPrarun T G
Senior Engineer - Design at Applied Material India Private Limited - (Under Tech Mahindra)Senior Engineer - Design at Applied Material India Private Limited - (Under Tech Mahindra)