Indu Marthu

• Overall 3 years in Information Technology with strong background working experience in Data analysis, Azure Data Factory, Azure Databricks, Azure Data Lake, Azure SQL Database, Logic Apps, Python basics, PySpark, Power BI,Oracle, MS SQL Server.
• Experience on Azure Data Factory, Azure Data Lake, Blob Storage, Azure SQL
  Database, Azure Key Vault, Azure Data Bricks and Azure SQL Data warehouse and controlling
• Experience in Developing Spark applications using PySpark - SQL in Databricks for data extraction,
  transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Driver Node, Worker Node, Stages, Executors and Tasks.
• Created Pipelines in Azure Data Factory using Linked Services / Datasets / Pipeline / to Extract, Transform and Load Data from different sources like On-Premises Databases, Blob storage, Azure SQL Database.
• An excellent team player with highly favorable interpersonal, multitasking with hard working, fast learning and prioritizing capabilities.
• Interacting with client on scrum calls.

Project :              

Organization       : Intelligence  

Project name      : Sales reporting  

Client                    : PepsiCo  

Duration               :  Feb 2022- Present   

Environment       : Microsoft Azure (Data Factory, Azure Databricks, Blob Storage, Azure Data Lake), SQL server , Azure SQL Database 

Project description :
This project is to reduce the involved costs and risks in preparing reports from the data  which is coming from Legacy system. Not only this, even its consuming more time and it is increasing the gap  between the business and its customers. As an Azure Data Engineer, I design, develop, and optimize scalable data pipelines and cloud-based data solutions on Microsoft Azure to enable advanced analytics, reporting, and business intelligence. My work involves integrating large-scale datasets, implementing Azure Data Factory, Databricks, and SQL Databases, and ensuring secure, efficient data processing for batch analytics. I collaborate with cross-functional teams to enhance data governance, ETL workflows, and AI/ML model deployment, driving data-driven and decision-making across the organization.

Responsibilities : 

• Led business requirement gathering sessions with stakeholders and translated them into technical design documents for Azure data pipelines.
• Collaborated with Data Validation and mapping teams to define data quality rules, reducing data inconsistencies.
• Built Azure Data Factory pipelines using Copy Activity to ingest large volumes of daily data from diverse sources improving load times.
• Implemented PySpark transformations in Databricks (Delta Lake) to process raw data into business-ready formats, enabling real-time analytics for client teams.
• Automated data validation checks using Python/Pandas, identifying and resolving data quality issues monthly.
• Developed PySpark scripts to optimize data movement from Azure Data Lake to Azure SQL DB, reducing latency.
• Designed and scheduled Databricks notebooks for complex transformations such as aggregations and joins, ensuring a high job success rate.
• Leveraged Azure services including Key Vault, Logic Apps, and Functions to secure credentials, orchestrate workflows, and reduce manual effort.
• Partnered with QA teams for user acceptance testing, resolving defects pre-production.