Sid Sharma

Project Description: Project aimed to build desktop/Web applications in a variety of languages, including Python, to incorporate the customer's API directly into their own, and created a Python Software Development Kit (SDK) for their financial information API, leading to increased revenue and customer satisfaction.

• Created API’s to consume data from REST endpoints and cleansing the data and store it in database.
• Used Spark for streaming APIs to perform transformations and actions on the fly for building the common learner data model which gets the data from Kafka in near real time and persists into Cassandra.
• Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena.
• Designed detailed Migration Plans for Workloads from any platform to AWS & Utilized workload migration tools to AWS (i.e.
• Cloudscape, Zerto, Platespin, etc.).
• Implemented a 'server less' architecture using API Gateway, Lambda, and Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets.
• Created a Lambda Deployment function and configured it to receive events from S3 bucket.
• Developed AWS centralized logging solutions for security teams to consolidate AWS logs and analyze them to detect incidents, using Elasticsearch Service, CloudTrail, VPC flow logs and EC2 server logs.
• Worked on Stretch Database feature of SQL Server for the cold data residing on on-premises SQL Server Databases can be migrated transparently and securely to AWS.
• We also worked on DynamoDB to maintain the record of all the logs.
• These were all constructed via IaaC like CloudFormation and Terraform.
• Worked on functions in Lambda that aggregates the data from incoming events, and then stored result data in Amazon DynamoDB.
• Created various performance metrics and configured notifications using CloudWatch and SNS.
• Worked on Amazon Kinesis to process and analyze streaming data with standard SQL.
• Setup the CloudWatch monitoring dashboards with SNS notification configurations on various performance metrics retrieved from various AWS Resources.
• Worked on Service Oriented Architecture (SOA) and its related technologies like Web Services, BPEL, WSDLs, SOAP1.1, XML, XSD, XSLT etc.
• Worked on writing complex SQL queries with multiple joins to fetch data and work on validation for the processed data in ETL jobs.
• Designed and developed Use-Case Diagrams, Class Diagrams, and Object Diagrams using UML Rational Rose for OOA/OOD techniques.
• Set-up databases in AWS using RDS, storage using S3 bucket and configuring instance backups to S3 bucket.
• Created functions and assigned roles in AWS Lambda to run python scripts, and AWS Lambda using java to perform event driven processing.

Project Description: Corporation wanted to mine complex customer and prospect behavioral data as part of a digital transformation project. The company used Python to initiate different data science and machine learning initiatives to examine the structured data it had been collecting for years and correlated it with unstructured data from the web and social media to increase cross-selling and reclaim resources.

• Designed and managed API system deployment using fast HTTP server and Amazon AWS architecture & Setup database in AWS using RDS and configuring backups for S3 bucket.
• Used Docker Container, EC2, IAM, RDS, Route 53, REST API gateway and Lambda of AWS.
• Developed data transition programs from DynamoDB to AWS Redshift (ETL Process) using AWS Lambda by creating functions in Python for the certain events based on use cases.
• Build servers using AWS Importing volumes, launching EC2, creating security groups, Auto-scaling, load balancers, Route53, SES and SNS in the defined virtual private connection (VPC).
• Used AWS Redshift, S3, Spectrum and Athena services to query large amount data stored on S3 to create a Virtual Data Lake without having to go through ETL process.
• Built/deployed custom Docker images from Artifactory into Amazon Elastic Kubernetes Service (EKS) K8s cluster as part of a Gitlab CI pipeline.
• Worked on Amazon Kinesis Streams to collect and process large streams of data records.
• Used DynamoDB global tables to replicate data across multiple AWS Regions to get fast, local access to data for globally distributed applications.
• Experience in setting up Elastic Load Balancers (ELB's) and Auto Scaling groups on Production EC2 Instances to build Fault-Tolerant and High Availability applications.
• Created S3 buckets and managing policies for S3 buckets and Utilized S3 bucket and backup on AWS.
• Designed code to transform logs from AWS CloudWatch to Kibana.
• Used IAM to create new accounts, roles and groups and polices and developed critical modules like generating amazon resource numbers and integration points with S3, Dynamo DB, RDS, Lambda and SQS Queue.
• Designed CI/CD processes in the context of a Jenkins orchestration, including the use of automated build, test/ QA and deployment tools.
• Used GitHub repository to submit code changes & for source code version control and deploying in AWS Lambda.

• Developed Microservices, and creating API's using Java Spring Boot framework using Maven as a build tool and Cassandra as an enterprise level database.
• Used Test driven (TDD) approach for developing the application and Implemented the unit tests using Python PyTest Framework.
• Generated Python Django forms to record data of online users and used Py-test for writing test cases.
• Performed debugging, testing, maintaining and deploying the application.
• Developed RESTful Web Services for getting and sending data from the external interface in the JSON format.
• Created Jenkins build and deployment pipeline jobs to deploy the docker images into AWS ECR repositories and integrated with GITHUB.
• Designed the data models to be used in data intensive AWS Lambda applications which are aimed to do complex analysis creating analytical reports for end-to-end traceability.
• We created an on-premise application to trigger AWS GLUE ETL process, those triggers were mainly using lambdas and step functions.
• Responsible for configuring NoSQL databases like Apache Cassandra and Mongo DB to for increasing compatibility with Django.
• Developed Python scripts to monitor health of MongoDB and performed-hoc backups using Mongo dump and Mongo restore.
• Implemented continuous integration using Jenkins and involved in the deployment of application with Ansible automation engine.
• Imported required tables RDBMS to HDFS using Sqoop & also used Storm and Kafka to get real time streaming of data into HBase.
• Developed Docker container based micro services using Spring Boot/Cloud and deployment on Kubernetes cluster & a fully automated continuous integration system using Git, SQL Server and custom tools developed in Python and Bash.
• Developed Sqoop scripts to handle change data capture for processing incremental records between new arrived and existing data in RDBMS tables.
• Managed datasets using Panda data frames and MySQL, queried MYSQL database queries from python using Python-MySQL connector and MySQL dB package to retrieve information.
• Implemented AWS high availability using AWS Elastic Load Balancing (ELB), which performed balance across instances.

• Designed and developed Application based on the Django MVC Framework using MVC design pattern.
• Developed the REST API using Django for fetching the data from NoSQL MongoDB database.
• Used Angular JS framework for single page applications and validating the client-side user information based on their business rules.
• Developed new RESTful API services that work as a middleware between our application and third-party APIs that we will used using Golang.
• Designed and managed API system deployment using fast http server and Amazon AWS architecture.
• Implemented AWS solutions using E2C, S3, RDS, EBS, Elastic Load Balancer, Auto-scaling groups.
• Implemented AWS client API to interact with different services as Console configuration for AWS EC2.
• Wrote Python, Golang and Linux shell scripts to streamline the data analysis, algorithm development, and model optimization workflows.
• Worked on container Services (AKS) for deploying CI/CD pipelines integrated with Git, Jenkins and Docker for micro services-based applications.
• Created MySQL queue using AWS SNS, SQS, and Golang (Go), to concurrently process SQL queries in the background and Built a Golang web server to host multiple tools so another has http access to them.
• Wrote Python modules to view and connect the Apache Cassandra instance.
• Developed PySpark code using Python and Spark-SQL for faster testing and data processing.

• Developed views and templates with Python and Django view controller and templating language to create a user-friendly website interface.
• Used AWS Cloud Watch to monitor and store logging information.
• Created scripts for data modelling and data import and export.
• Design and implemented custom Bash scripts for controlling the data flow from the PostgreSQL database.
• Wrote REST APIs, as part of developing web-based applications for customer feedback and complaints calculations, using Django's REST framework.
• Developed multi-threaded standalone app in Python to view performance.
• Performed various Parsing technique's using PySpark API'S to cleanse the data from Kafka.
• Created an automated cloud image creation process using packer, git, server spec and some bash/python scripting.