Sanjiv Tiwari
Gurgaon
Summary
Accomplished Data Engineer with over 6 years of experience designing and implementing scalable data solutions using Azure, PySpark, SQL, and Python. Expertise in building unified data platforms, optimizing pipelines, and developing dashboards for enterprise reporting. Background includes cloud migration and machine learning automation, complemented by certifications in Azure Data Engineering and SQL development.
Overview
1
1
Certification
8
8
years of professional experience
Work History
Data Engineer
Maersk
Bangalore
02.2022 - Current
- Led ETL cost and performance optimization initiatives, reducing Data Lake storage volume by 76% (3.7TB → 900GB), ADF runtime by 45% (22 hrs → 12 hrs), and AAS costs by 55% ($15K → $7K/month), delivering ~$126K annual savings.
- Designed and deployed end-to-end ETL pipelines using Azure Data Factory (ADF) and Azure Databricks to ingest, transform, and load data from SQL databases, APIs, and files into Azure Data Lake and Synapse Data Warehouse.
- Built scalable dimensional models (Fact & Dimension tables) in Azure Synapse SQL using robust ETL workflows, enabling efficient enterprise analytics and reporting.
- Built a Python-based Data Quality (DQ) framework in Databricks to ensure accuracy and reliability of raw Data Lake data prior to transformation and loading.
- Created Power BI dashboards and semantic data models with Azure Analysis Services, Dremio, and SQL Endpoints to meet business reporting requirements and enhance data visibility.
- Unified and standardized reporting data for Ocean (~35 years of operations) and LnS (~10 years) into a single enterprise reporting platform to streamline analytics and improve decision-making.
Data Engineer
MAQ Software
11.2018 - 02.2022
- Used Machine Learning (Sentiment Analysis, Text Analytics, Recommendation System) to optimize support request process
- Migrated Prem architecture to the cloud (Microsoft Azure)
- Participated in full life cycle of development, from requirements specification and design to implementation, testing, and maintenance.
- Created tool to categorize, prioritize, and find similar tickets for support team, saving time and aiding new users.
Education
B. TECH - INFORMATION TECHNOLOGY
NIT DURGAPUR
Durgapur, West Bengal01-2018
Intermediate - Science & Math's
Govt. Jubilee Inter College
Gorakhpur01-2013
Skills
- ETL/ELT
- SQL
- Python
- PySpark
- Azure Databricks/ADB
- Azure Data Lake
- Data Modelling using AAS/Dremio/Sql-Endpoint
- Data factory
- Data lake storage
- Azure Synapse
- MS Fabric
- Snow Flake
- Azure Key Vault
Certification
- DP-200 Implementing an Azure Data Solution, Microsoft
- Exam 767: Implementing a Data Warehouse, Microsoft
- Exam 762: Developing SQL Databases, Microsoft
- Exam 778: Analyzing and Visualizing Data with Power BI, Microsoft
- Exam 761: Querying Data with Transact-SQL, Microsoft
Positions Of Responsibility
- Organized various certification courses as Campus ambassador
- Been Elected as Vice President of college gymkhana (Student's Union)
- Member at Recursion Programming Group (2017-05-2018)
Languages
English
Proficient (C2)
C2
Hindi
Proficient (C2)
C2
Timeline
Data Engineer
Maersk
02.2022 - Current
Data Engineer
MAQ Software
11.2018 - 02.2022
B. TECH - INFORMATION TECHNOLOGY
NIT DURGAPUR
Intermediate - Science & Math's
Govt. Jubilee Inter College