Sudheer Kumar Oleti

Data Engineer

Empower Retirement Solutions PVT. LTD

05.2016 - Current

Project: Financial Complaints Dashboard
The Financial Complaints Dashboard is a comprehensive platform designed to track, analyze and visualize complaints related to financial products and services
Participated in agile development processes, contributing to sprint planning, stand-ups, and reviews to ensure timely delivery of data projects.
Optimized SQL queries and database schemas for performance improvements in data retrieval operations.
Collaborated with cross-functional teams to gather requirements and translate business needs into technical specifications for data solutions.
This project aims to provide insights into customer issues, monitor industry practices and enables to improve their services
This project helps to identify areas for improvement and enhance their customer service
This project deals with end to end data ingestion and transformations based on business requirements
Responsibilities: Involved in analysing business requirements and prepared detailed specifications that follow project guidelines required for project development
Implemented data ingestion pipelines to fetch, transform, and load data from various sources into ADLS and ensuring data accuracy and consistency
Performed import and export of data from different source files like parquet
Worked on Spark SQL that meets the required business transformation like performing aggregations, masking data, joins, etc., using Dataframes
Data Lake for reporting and analytics purposes using different component of Azure Services
Developing and maintaining end-to-end operations of ETL data pipelines and worked with large dataset in Azure
Researched and implemented various components like pipelines, storage accounts, containers, SAS, Key Vault etc
Implemented the databricks workflows for various tasks through the Git, Azure Data Lake Storage Also actively monitoring the recently deployed critical batches scheduled to make sure smooth delivering process on time as the downstream reports used in the front end much dependent on the deliverable data
Drove the team to understand and execute Change and Incident management
Coordinated with the Project Manager in Production implementation
Worked on job scheduling for daily, weekly execution of the batch
Collaborating with data scientists and business stakeholders to understand data requirements and identify opportunities for data-driven decision making
Monitoring performance and optimizing big data systems to ensure they meet performance and availability requirements
Collaborated with business teams, incident resolution, automation and technical changes, supported UAT and release management, delivered to meet the operations requirement
As a Data Engineer I was involved in the complete life cycle of the EDW Data Warehouse implementation which included System Architecture
Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion etc
Functional/Technical Design, Development - Coding and Testing (Unit Testing, System Integration Testing and User Acceptance Testing) and Implementation.