SHUBHAM MOHAN MORE - Data Engineer - TVS Motor Company

Summary

Data Engineer with experience in ELT Processes, Data warehousing, Data modelling, Azure cloud services and BI tools. Proficiency in data orchestration, creating optimized data pipelines and data visualization. Strong background in Spark SQL, Pyspark, Data modelling, SQL Server, Cosmos db, Databricks warehouse and delta lake, cloud platforms like Azure Data Factory, Azure Data Lake Storage, Azure Devops and BI tools like Power BI.

Overview

6

years of professional experience

Work History

Data Engineer 2

TVS Motor Company

04.2024 - Current

Engineered, managed, and coordinated full-scale data pipelines employing Azure Data Factory, Azure Databricks workflows, and PySpark to ensure reliable data ingestion, transformation, and usage in downstream business-focused digital solutions, and predictive models.
Built and maintained the Medallion Architecture (Bronze, Silver, Gold) by developing scalable ETL pipelines to ingest data from SAP, SharePoint, Azure SQL Database, and public APIs into Delta Lake.
Developed SCD Type 2 frameworks and incremental data loading solutions using Azure SQL stored procedures to preserve historical data, and improve data processing efficiency.
Containerized and deployed Dockerized ETL applications using the Flask framework for cross-functional development across multiple teams.
Configured and administered Unity Catalog, external locations, clusters, and SQL Warehouses, while integrating Databricks with Azure SQL Database, MySQL, and PostgreSQL.
Developed secure data transfer workflows for sharing data with third-party vendors outside the organization's VNet, ensuring compliance with security and data governance standards.
Architected and developed business-ready data marts for R&D, NPD, Finance, and Sales, enabling seamless data consumption by enterprise applications, AI chatbots, ML models, and Power BI dashboards.
Led a cross-functional team of 2 Data Engineers and 3 Data Analysts in delivering 30+ enterprise data models, establishing Power BI workspaces for global teams, and publishing 75+ dashboards to support business-critical decision-making.
Managed source code using Azure DevOps Repos, developed secure data-sharing workflows for third-party vendors, and implemented CI/CD best practices through code reviews, pull requests, and version control.
Monitored Databricks platform usage and cloud costs using system billing tables, driving optimization initiatives that reduced cloud expenditure by 20% while maintaining platform performance and reliability.

Graduate Engineering Trainee

Inteva Products India Automotive Pvt Ltd

06.2017 - 06.2019

Achieved operational effective efficiency (OEE) target with well-planned and solidly implemented engineering solutions kaizen projects by evaluating materials costs, projected savings, and other sourcing requirements.
Working closely with cross-functional teams on operational performance projects for FY 2017-18 and FY 2018-19.

Education

Master of Science - Research

Indian Institute of Technology Guwahati

Guwahati, Assam

06-2022

Bachelor of Engineer - Mechanical

Maharashtra Institute of Technology Savitribai Phule Pune University

Pune, Maharashtra

06-2017

Skills

Programming Language: Spark SQL, PySpark, Python

Cloud and Data Platforms: Databricks (Unity Catalog), Azure Data Factory, ADLS Gen2, Azure Devops, Docker (Beginner)

Databases: Azure SQL Server, My SQL, Cosmos Db, SAP HANA

AI Tools: Databricks Genie, MS Copilot Studio

Data Visualization: Power BI, MS Power Automate, Measure Killer

ERP Systems: SAP, Windchill

Methodology: SAFe 6 Agile Certified

Accomplishments

Enterprise Project Tracking & Analytics Platform

Designed and developed a centralized analytics platform using Azure Databricks (PySpark, Spark SQL, Databricks Workflows) by integrating data from 25+ SAP source systems and consolidating 40+ project KPIs into business-ready data marts. Enabled real-time project tracking, contributed to a 10% reduction in time-to-market, and automated stakeholder notifications through an SMTP-based alerting framework.

PLM Data Access & Integration Framework

Built secure, high-performance REST APIs using Azure Function Apps to enable data access across multiple PLM systems. Optimized query performance on datasets exceeding 40 million records through indexing and hashing techniques, achieving response times under 2 seconds. Later optimized this platform by deploying few modules in Azure Cosmos database.

Workforce Analytics Solution

Designed and implemented a normalized data model by creating master data tables and key-based relationships, improving data integrity and reducing storage overhead. Developed ETL pipelines and business-ready data marts using Azure Databricks (PySpark and Spark SQL) to support workforce analytics. Built interactive Power BI dashboards for KPI reporting and automated dashboard distribution and email notifications using Microsoft Power Automate, delivering a scalable and maintainable analytics solution for business decision-making. The application had a traffic of more than 2000 users per day.

Financial Data Analytics Platform

Developed a generic ETL framework using Azure Databricks Workflows to ingest finance data from SAP T-codes, capturing CAPEX, REVEX, freight, and project cost metrics and storing them in ADLS in Parquet format. Implemented SCD Type 2 logic to build Delta tables, creating a business-ready finance data mart that served as the input for a business viability application to automate KPI calculations. Built interactive Power BI dashboards with Row-Level Security (RLS) to deliver secure, role-based financial reporting and analytics.

ETL Metadata framework for BI

Developed a reusable Python-based script to automate Power BI dataset refreshes using the Power BI REST API. Integrated the solution into multiple ETL pipelines, reducing dashboard refresh latency by 30 minutes per dashboard while improving automation, consistency, and operational efficiency.

Moto AI Chatbot

Architected the data model for an AI chatbot project that summarized the HR documents and answer the questions of user. Worked on Microsoft Co-Pilot Studiofor Chatbot UI development. Developed the chatbot workflow in Co-pilot studio and feedback capturing automated workflows in MS power automate. A python script was developed to capture the output and the user details in Azure SQL Server.

Affiliations

Implementation of Soft Computing Techniques in Predicting and Optimizing the Operating Parameters of Compression Ignition Diesel Engines: State-of-the-Art Review, Challenges, and Future Outlook, ASME Journal of Computing and Information Science in Engineering, Mar 24, 2022
Analysing the Operating Parameters of a Compression Ignition Diesel Engine Using Artificial Neural Network, ASME POWER 2022 Conference, Sep 26, 2022
Evaluating the Operating Parameters of a Compression Ignition Engine Fueled with Waste Cooking Oil—Diesel Blends Using Artificial Neural Network and Ensemble Methods, ASME Journal of Engineering for Gas Turbines and Power, May 25, 2023

PUBLICATIONS

Implementation of Soft Computing Techniques in Predicting and Optimizing the Operating Parameters of Compression Ignition Diesel Engines: State-of-the-Art Review, Challenges, and Future Outlook, ASME Journal of Computing and Information Science in Engineering, Mar 24, 2022
Analysing the Operating Parameters of a Compression Ignition Diesel Engine Using Artificial Neural Network, ASME POWER 2022 Conference, Sep 26, 2022
Evaluating the Operating Parameters of a Compression Ignition Engine Fueled with Waste Cooking Oil—Diesel Blends Using Artificial Neural Network and Ensemble Methods, ASME Journal of Engineering for Gas Turbines and Power, May 25, 2023

Timeline