Siva Tirri

1. Designed and implemented data ingestion pipelines using Azure Data Factory (ADF) to hydrate data from multiple ERP sources such as Oracle, SQL Server, SFTP, and flat files into Azure Data Lake (ADLS).

2 .Built and maintained Bronze, Silver, and Gold layer architectures in Azure Databricks using PySpark and SQL, ensuring scalability, modularity, and performance optimization for downstream analytics.

3. Applied complex data transformation logic, implemented business rules, and performed joins, aggregations, and validations using Databricks notebooks and Delta Lake architecture.

4. Ensured data quality, lineage, and compliance by integrating validations, logging, and monitoring across ADF and Databricks pipelines; followed best practices for governance and security.

5. Collaborated with business and BI teams to expose curated Gold layer data for Power BI dashboards, reporting solutions, and Azure Synapse Analytics for enterprise wide consumption.

1. Responsible for building big data pipelines to process monthly payments for retailers and distributors selling HP's Instant Ink subscriptionbased service globally, using customer database insights to generate payment reports.

2. Migrated existing NiFi pipelines to Databricks ETL workflows as part of optimization efforts, streamlining processes and developing reusable data pipeline components to support the team’s framework evolution.

3. Contributed to overall data architecture design, applying data modeling, storage, and retrieval expertise for efficient and scalable solutions.

4. Created data products for business analysts and data scientists, developing automated jobs to support their analysis and collaborating with stakeholders to define requirements and deliver insights.

5. Guided and mentored junior data engineers, fostering knowledge sharing and supporting professional development within the team.

6. Built data pipelines and implemented efficient ETL/ELT proce

• As a Data Engineer, I'm responsible for building big data pipelines to process monthly payments for retailers and distributors who sell HP's Instant Ink subscription-based service worldwide.
• Utilizing the customer database, I will generate a report of all retailers and distributors and their respective payments, which will be made by HP on a monthly basis.
• As part of optimization efforts, I migrated an existing NiFi pipeline to Databricks ETL pipelines, thereby streamlined the process.
• Develop reusable data pipeline components and contribute to the team's data pipeline framework evolution.
• Contributed to data architecture design, applying data modelling, storage, and retrieval expertise.
• I create data products for business analysts and data scientists, and also develop automated jobs for their analyses.
• Partnering with stakeholders, data teams, and engineers to define requirements, troubleshoot, optimize, and deliver data-driven insights.
• Mentorship & Knowledge Transfer: Guide and mentor junior data engineers, fostering knowledge sharing and professional growth.

• The aim of the project is to predict the remaining useful life of spark plugs used in cars, with the objectives of reducing maintenance costs and avoiding unexpected downtimes.
• I am responsible for ingesting data from various sources into ADLS from an HTTP endpoint provided by the client.
• Processing structured & semi-structured data using pyspark, and data manipulation with spark SQL.
• Design and implementing robust prediction pipeline and retraining pipeline with Azure Data factory.
• Creating, running, scheduling, and managing jobs for prediction pipeline and retraining pipeline.
• Storing the prediction results, error logs in the SQL Database and sending them via Email to respective user groups.
• Manage, monitor data pipelines, and ensuring the security and privacy of data.