Akhilesh Singh

Experienced lead with 12+ years in design, dedicated to creating innovative processes. Expert in developing workflows and frameworks to streamline operations. Skilled in migrating on-premises data to AWS, GCP for optimal performance. Strong focus on Object-Oriented Programming for high-quality solutions. Committed to continuous learning and professional growth.

1. SQL to Redshift Migration using AWS DMS & Glue Orchestration, Iris Software, Inc.
This project involved end-to-end migration of an on-premises SQL-based data warehouse to Amazon Redshift. The primary objective was to modernize the data platform by leveraging AWS cloud-native services to improve scalability, performance, and reliability.

Responsibilities & Key Highlights:

• Solely designed and developed the entire ETL orchestration framework in Python/PySpark, following robust Object-Oriented Programming (OOPs) principles for flexibility, reusability, and modular design.
• Designed and implemented a multi-layered ETL architecture
• consisting of Raw, Consumed, and Engineered zones, using Amazon S3 as the landing zone and staging area.
• Utilized AWS DMS (Database Migration Service) to perform Change Data Capture (CDC) replication from the source SQL database to S3 in Parquet format, ensuring minimal downtime during cutover.
• Built scalable AWS Glue Jobs to process and transform data between the layers:
• Bronze→ Silver: Applied basic cleansing, validation, and partitioning.
• Silver → Gold: Applied business logic, schema harmonization, and column-level transformations.
• Loaded curated data into Amazon Redshift for downstream analytics and dashboarding using Power BI and QuickSight.
• Orchestrated the entire workflow using AWS Step Functions, providing visibility and failover control across:
• DMS task triggers
• Glue job sequencing
• S3 data checks and success/failure logging
• Implemented reusable scripts and parameterized configurations to support new pipelines with minimal code changes.
• Enabled role-based access to S3 and Redshift using IAM policies, Lake Formation tags, and column-level permissions.
• Maintained documentation and conducted knowledge-sharing sessions with team members.

Tech Stack:
AWS DMS, Glue, Redshift, Step Functions, S3, IAM, Lake Formation, PySpark, Python, SQL Server

2. DaaS Server: Migration on-prem to AWS, Iris Software, Inc. 3. DQ Framework, Iris Software, Inc.

It's having DaaS Server developed in Java & DaaS client developed in C#. User request the data from server using client and server return the Kafka Topic to the user to consume the data. Client is implemented with both sync & Async APIs.

It's having 100+ APIs. DaaS server need to deploy on AWS and convert the client APIs to AWS API Gateway. Without interfering the user experience, I have gradually integrated the APIs with API Gateway and to achieve this I have created a proxy as Lambda Service. This proxy is responsible to check that the request is coming for AWS migrated API or still on-prem API and then it's sent the response to the user.

Responsibilities:

● Build AWS API Gateway for 50+ APIs.

● Update C# client to send an additional parameter as api_name.

● Create proxy as AWS Lambda which redirect the request to on-prem or AWS DaaS server.

● Testing of all the APIs.

3. DQ Framework, Iris Software, Inc.

This framework is taking care of validating the data quality as per the defined rule by the user. There is 50+ rules which user can select and run on the data like, is_distinct, is_number, check_length, check_file_size, is_blank, ignore_blank etc.

Input data can we S3 files, DB Tables and SQL Script. Best part is, user can create own rule in the form of python script.

Responsibilities:

● Design the code for the flexibility and scalability.

● Converting rules in to Python/PySpark.

● Testing with different use-cases like single or group of files.

● Using best suited data-structure.

● Code review of the Glue Job script.

● Regular discussion with business owner and the user for the new features and improvements.

4. Opti-Price, QL2 Software

Opti Price uses proprietary auto-matching technology to help customer examine their price competitiveness, identify pricing opportunities and outliers for their product catalog, and manage their product matches. Boost revenue growth by making data driven decisions aligned with client's pricing strategies and the market landscape in real time.

Responsibilities:

● Re-design the code as per the OOPs standards.

● Implementing the new features like pricing on region basis.

● Optimize the SQL queries by correcting the sequences of joins and order by.

● Created indexes if any of the column impacting the performance.

● Implemented the multi-threading for the multiple customers of the same vertical/domain.

● With the help of Pandas, all the different type of aggregation like min-max, median, no. of matches, etc. is read to consume.

5. QL2 Python Package, Ql2 Software

No credentials in code of static in code. This package is responsible to return different connection objects like postgres, snowflake, SFTP/FTP, AWS.

Responsibilities:

● Creating different modules and configuration related to different type of connections.

● Maintenance and support.

● Taking care of the up-gradation of the features with different utilities.

• Accolade for Q2-2025
• Python Trainer with 4.5/5 Rating Q3-2024
• Development Environment Setup for Glue Jobs, 04/23
• 2nd Position Award: Cloud Hack, 12/22
• Most Valued Person Award, 01/22
• Bravo Award, 08/15
• Bravo Award, 02/15

AWS Knowledge: Object Storage