Devendar Rao Gujja

Dynamic Senior System Engineer with over 2 years at HSBC, excelling in Azure Data Factory and Data Bricks. Proven track record in optimizing data pipelines and enhancing performance, while demonstrating strong analytical skills. Adept in Python and SQL, with a commitment to delivering impactful solutions in cloud environments.

• Certified AZ-104: Microsoft Azure Administrator
• Certified by Databricks Lakehouse Fundamentals
• Cloud architect program completed by external and received by training certificate.

Project #1, HSBC BANK, Azure Data bricks, Azure SQL, Azure Data Lake, pyspark, Azure data factory, Spark, python, Data engineer, 7, This is a business intelligence project sponsored by the cash management department which resides in Illinois, Chicago. The Cash management department currently provides 'value added' services to about 50,000 of the customers who have commercial checking accounts. This is currently a 'hit rate' of about ten percent. The INSIGHT BI solution is designed to help provide the cash management officers with useful views and reports that allow them to serve their existing customers more profitably and to lure some off the 450,000 other customers to start using some or all of the value added 'analysis' services., Designing and implementing data pipelines to ingest, process, and analyze large volumes of data related to INSIGHT application transactional data., Utilizing Azure Data bricks for distributed data processing, optimization, and collaboration within a cloud environment., Architected and implemented a cloud-based data warehouse solution using Azure Data Factory & Azure Data bricks., Implementing and maintaining data architecture using Medallion architecture principles for scalable and modular data solutions., Designed and implemented a robust Medallion Architecture using cutting-edge technologies such as Azure ADF, Azure Data bricks, Delta Lake, and Unity Catalog as the storage solution., Leveraged Pyspark and a metadata-driven process to ensure efficient data processing and management within the architecture., Implementing Delta Lake for ACID transactions and version control, ensuring data integrity and reliability in data processing workflows., Optimizing Delta Lake tables for efficient querying and analytics., Collaborating with data analysts to understand requirements for analyzing transactional data in the context of INSIGHT systems., Created different layers of data on the Azure Data Lake and cleaned the data., Performed transformations on the data using Azure Data bricks., Applied key performance indicators on the data and then the data was aggregated., Moved the data to synapse to facilitate the PowerBI team to populate the reports., Tuning and optimizing data pipelines for performance and scalability, ensuring timely processing of data for analytical purposes., Documenting data engineering processes, workflows, and pipeline architectures for knowledge sharing and future reference., Keeping abreast of the latest developments in data engineering, cloud computing, and analytical tools to continuously improve and innovate data solutions., Troubleshooting and resolving issues related to data pipelines, ensuring minimal downtime and maximum reliability. Project #2, HSBC BANK, Azure SQL, Azure Data Lake, Azure data factory, This project is aimed at strategizing a cost-effective solution to maintain sustainability in supply chains., Implemented data pipelines on reading raw operational data and move it to the staging layer using medallion architecture., Used Delta tables on ADLS storage., Built data pipelines that generate data to monitor supply chain management and optimizing inventory levels., Led cross-functional teams to identify and implement data quality improvements, resulting in a 20% increase in overall operational effectiveness., Implemented data pipelines to identify the compliance with company policies and relevant regulations., Utilized ADF and workflows to build data pipelines., Sourced data from SQL, CSV files and transformed the data based on business rules defined by the stakeholders., Extensively worked on spark optimization., Implemented alerting systems on failure of data pipelines.

Azure Data bricks, Azure Data Factory, Azure Data Lake, Jenkins, Jira, GitHub, CloudWatch, ServiceNow, Power BI, Azure Cloud, Azure Key Vault, Python, Pyspark, SQL, Windows, UNIX/LINUX, SQL, Azure SQL Data base, Power BI, Jenkins, JIRA, Azure, GCP