Kaleeswaran M

Working with Endoscopy R&D division of Stryker to build efficient computer vision algorithms to be deployed in Stryker's endoscopic devices.

Surgical Phase Estimation.

• Developed a Vision Transformer based model to estimate the surgical phase at a given point of time and achieved near state of art accuracy of 88%.
• Developed a GAN based data standardization model to improve generalizability by 8%. Have filed the patent - 'Style transfer of Surgery videos for Custom Visualization'.
• Implemented a modified Grad-CAM for explainability of the model.

Surgical Foundation model

• Trained a large scale self-supervised model based on VideoMAE on surgical videos.
• Validating the foundational model generalizability on Image quality assessment task.

Surgical anatomy segmentation

• Developed a surgical anatomy segmentation model with super-resolution supervision.
• Optimized the model to run at 13 FPS at 1280x720p on the Jetson Xavier.

Surgical Automatic Speech Recognition

• Finetuned Whisper ASR model on surgical vocabulary on a TTS generated dataset.
• Improved the finetuning performance with LoRA based finetuning.

Worked mostly with AthenasOwl team in solving AI/ML problems in Media and Entertainment domain. Work involved solving problems mainly in Video processing and audio processing for Media companies.

Audio Keyword Detection

• Built a CNN-LSTM based architecture with CTC loss to detect particular keywords in Football commentary track for Bundesliga.

Audio-Visual Speech separation

• Built a model based on ‘Looking to Listen: Audio-Visual speech separation’, to separate audio waveforms of speech from rest in a mixed audio waveform and assign the separated speech waveforms to the individual speakers in the video.

Scene Boundary Detection

• Conceptualized and implemented a model end-to-end from data to network to detect meaningful boundaries of scenes in a media content.

Auto-highlight clip classification

• Build a model leveraging Visual and Audio modality to classify the event happening in a football clip.
• Was able to validate model's improved performance by fusing visual and audio representations rather than having the model trained only through the visual representations.

Content Recommendation

• Built an explainable Recommendation system on the MovieLens 20M dataset based on ‘Knowledge Graph Convolution Network for Recommendation system’, which uses knowledge graphs as auxiliary information.