Shyam Sundar R

• Building data solutions like data warehouse, data lakes, data pipelines.
• Responsible for setting Tech standards, roadmap and best practices.
• Responsible for building cloud native, multi cloud, hybrid cloud solutions for multiple clients ranging start-ups to enterprise.
• Building scalable infrastructures, multi tiered architectures and serverless.
• Responsible for building data pipelines for customers using cloud native solutions.
• Contributing to implement Cloud Adoption Framework and Security Benchmarks.
• Working on ETL framework using spark, python.

CCDA microservice

• Scaling the service which can handle more than a million requests per day without disrupting dependant services.
• Designed a metrics builder which looks for on-prem DB metrics and publishes it as metric which will be used by the service to handle dynamic scaling

Technologies Used - AWS, kotlin, Prometheus, Grafana, AlertManger, Graylog, Elasticsearch

Data Lake

• Built a data lake solution using Athena, Glue, Lambda, API Gateway, S3, Spark, Elastic Cache (Redis).
• Lead the data engineer team.
• Responsible for the architecture and used the data built to send to other BI tools like domo using SDKs and connectors.
• Built the live analytics platform using elasticsearch.
• ETL processing of data more than a million per day was done in Glue.
• Developed a custom REST API, for querying the data lake which was exposed to multiple customers

Technologies Used - AWS, Redis, Domo, ElasticSearch, Pyspark

CCDA microservice

• Designed and developed CCDA microservice in AWS.
• Developed a monitoring and metrics system using Grafana, Prometheus and Alertmanager which gather system metrics as well as application metrics.
• Designed a logging environment using Graylog, Elastisearch and Filebeat to gather application logs from docker containers.

Technologies Used - AWS, Kotlin, ECS, Prometheus, Grafana, AlertManger, Graylog, Elasticsearch

Bulk Export Solutions

• Designed and developed event driven service to export CCDA's in bulk which was deployed in AWS.
• Developed a monitoring and metrics system using Cloudwatch Metrics.
• Designed and developed a custom throttling for the service which allowed the service to throttle based on various factors using SQS, S3.
• Designed the whole architecture to be serverless, tearing down the infrastructure when it's done for bulk export.

Technologies Used - AWS, NodeJS, Perl, DynamoDB, SQS, Lambda, S3, Cloudwatch, S3 select

Data Exports Solutions

• Built a distributed export system using worker, RabbitMQ, Oracle, Perl which would scale more than 5 million messages a day.
• Created a simple UI for requesting exports using HTML,JavaScript.
• Used the same architecture to other systems (Migrations,Imports), so that it helped in scaling inside monolith.
• Implemented throttling, priority, dynamic scaling, cancel and resume to the application which is hosted on-prem.

Technologies Used - Perl, Oracle, RabbitMQ, Grafana, Linux

Data Liquidity

• Developed Migrations application a tool which plays a vital role in moving data within Athena.
• Designed and implemented a new architecture for data migration projects which reduced the overall turnaround time of the projects by50%.
• Developed standard perl modules to automate the process of importing data to the product and eliminated writing custom scripts for the same.
• Designed and implemented the Migrations workbook tool as a UI. Previously for a migration the users had to mention the mapping and the mapping was patched as code and in turn was used by the migration.

Technologies Used - Perl, Oracle, RabbitMQ, Grafana, Linux

POS Returns

• Developed a returns application where items can be returned from the POS, developed in C.

SEO

• Developed a PHP application which helped clients to see how the SEO has helped their product search.

Technologies Used - PHP, MySQL