Results-driven software engineer with expertise in Azure Data Factory, Python, and PL/SQL. Proven track record in designing and implementing complex ETL processes that enhance data integration and performance.
Overview
11
11
years of professional experience
1
1
Certification
Work History
Senior Software Engineer
Virtusa
09.2021 - Current
Played a pivotal role in migration of on-premises data systems to Azure, driving seamless transitions with minimal downtime and enhanced performance in the cloud environment.
Documented migration processes, cloud architecture, data integration, and other essential aspects, conducting knowledge transfer sessions to educate the client's team about the new environment.
Designed and implemented ETL processes to extract, transform, and load data from various source systems into Azure Data Storage services, leveraging tools such as Azure Data Factory.
Built and maintained pipelines in Azure Data Factory, utilizing Linked Services, Datasets, and Pipelines to ingest data from diverse sources like Oracle, relational databases, streaming data, Blob Storage, Azure Data Lake and REST API's. Configured and scheduled appropriate triggers to automate workflows.
Implemented unit testing frameworks in Azure Data Factory pipelines to validate data transformations, ensuring accurate and reliable ETL processes.
Developed and optimized Spark applications using Spark SQL and PySpark in Databricks to transform, aggregate, and analyze datasets, driving insights into customer usage patterns.
Developed reusable, automated data frameworks using Python and PySpark to streamline large scale data processing and reduce operational overhead.
Authored and optimized SQL queries to deliver efficient and scalable solutions that align with technical specifications and business objectives.
Ensured 0% latency and 100% on-time data processing by maintaining proactive checks.
Deployed a modern cloud data warehouse capable of handling large datasets and providing quick performance for analytical queries, while maintaining cost-effectiveness.
Implemented Incremental loading in Databricks notebook & Knowledge of Unity Catalog for data governance.
Provided continuous enhancement for existing codes and developed new codes based on client requirement.
Collaborated cross-functionally with business stakeholders, analysts, and product owners to define data requirements and deliver high-impact, high-quality data solutions.
Partnered with reporting teams provide curated datasets tailored to analytics requirements, enabling insightful decision making.
Utilized Azure DevOps for version control, continuous integration (CI), and continuous deployment (CD) pipelines.
Familiar with Azure Fabric for data integration and orchestration.
Technology Specialist
Standard Chartered Bank
04.2018 - 09.2021
Responsible for systems analysis, development enhancement, maintenance, and support for project releases.
Gathered requirements, wrote technical specifications, workflow diagrams, mapping documents etc.
Independently handled end to end testing.
Contributed significantly towards designing and executing test cases and summarizing test results for SIT and designed test cases for UAT.
Convert the Business need into Technical complex queries.
Use different strategies of database development to meet the client functional requirement.
Responsible for using dynamic SQL in PL/SQL which is an advanced programming technique that makes the applications more flexible and versatile.
Woking on SQL and PL/SQL programming, developing complex code units.
Creating complex views, Materialized views on multiple tables as per customer requirement.
Used PL/SQL Tables, Cursors to process huge volumes of data and used bulk collect for mass update as performance improvement process.
Having good experience on Client Interaction and Presentation Skills.
Assisted in developing and implementing project upgrades.
Support all day-to-day activities related to project testing cycles (PAT, SIT & UAT) and debugging the issues.
Involved in project upgrade by supporting End to End.
Resolved issues raised by business consultants by providing them with correct analysis of issues within the SLA.
Took care of enhancements and change requests and fixed them within SLA.
Software Engineer
HCL Technologies
11.2014 - 07.2017
Gained knowledge on Basel methodology of computing the credit risk in both Standard and Internal Ratings based approach.
Worked with Relational Tables (Oracle) as Source and target systems. Involved in handling queries like validating, testing. Worked with DDL, DML scripts in oracle database.
Involved in Creating/Modifying Database objects, Enhancements of procedures to meet business requirements.
Escalated issues to onshore team and onshore coordinator in timely manner.
Understanding the client requirements by studying functional documents.
Preparation of technical specification document based on the analysis and signed off TSD.
Development of source code, Unit Test Plan, Unit Test Case and performing Unit Testing.
Pre-User ACCEPTANCE Testing - End to End run will be done by the development team and result will be verified and confirmed by business team.
SIT, UAT testing will be conducted at SCB environment, and the development team will provide support to fix any issues raised by SIT and UAT users.
Performance tune queries, packages, procedures and functions.
Troubleshooting issues raised by Client as part of Post Implementation support.
Involved in upstream and downstream requirement given to the project and supported with technical solutions.
Investigation on the data issues of downstream.
Design queries in Oracle SQL Developer for analyzing data and finding/fixing issues.
Schema setup, Data loading and Data Masking activities.
Education
Bachelor of Technology (B. Tech) - Electronic and Computer Engineering
PVP Siddhartha Institute of Technology (PVPSIT)
Kakinada, India
Skills
PL/SQL
SQL
Python
Pyspark
Azure Data Factory
ADLS
Azure Key Vault
Azure Logic Apps
Azure Databricks
Data factory
Data Engineering
Storage
One Lake
MySQL
Oracle
SQL Server
Databricks
Azure HDInsight
Git
Bitbucket
Jira
Confluence
Cloud architecture
ETL processes
Spark applications
SQL optimization
Data integration
Certification
Databricks Certified Data Engineer Associate DP-700
Microsoft Certified Azure Fundamentals - DP-900
Microsoft Certified Azure Data Engineer Associate - DP-203
Certified Scrum Master - 2020
Data bricks Lake house Fundamentals
Awards
SCB Top Achiever and Live Wire Award in 2020
SCB Team Excellence Award 2021
Timeline
Senior Software Engineer
Virtusa
09.2021 - Current
Technology Specialist
Standard Chartered Bank
04.2018 - 09.2021
Software Engineer
HCL Technologies
11.2014 - 07.2017
Bachelor of Technology (B. Tech) - Electronic and Computer Engineering