I am a third year PhD student in Computer and Information Science at University of Pennsylvania (GRASP Lab), advised by Prof. Despina Kontos working on computer vison and deep learning.
Google Scholar  /
I completed my undergraduate at IIT Bombay where I worked on low-level computer vision with Prof. Arjun Jain. I have also obtained a Masters in Electrical Engineering at UPENN where I worked with Prof. Despina Kontos and Dr. Aimilia Gastounioti in the Computational Biomarker Imaging Group(CBIG) .
I am also working on a long term project with Dr. Vivek Buch at Stanford Medicine to develop an artificial intelligence (AI)-based surgical video analysis platform for intraoperative guidance.
Several topics in vision are of interest to me broadly in self-supervised/unsupervised learning and keenly in compositional visual reasoning,task-based vision/language learning, biological vision.
Recent Projects/Publications(for full list, visit google scholar link)
Automated Line labelling: Dataset for Contour Detection and 3D Reconstruction
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2023
Understanding the finer details of a 3D object, its contours, is the first step toward a physical understanding of an object. Many real-world application domains require adaptable 3D object shape recognition models, usually with little training data. For this purpose, we developed the first automatically generated contour labeled dataset, bypassing manual human labeling. Using this dataset, we study the performance of current state-of-the-art instance segmentation algorithms on detecting and labeling the contours. We produce promising visual results with accurate contour prediction and labeling. We demonstrate that our finely labeled contours can help downstream tasks in computer vision, such as 3D reconstruction from a 2D image.
Designing and developing a novel deep computer vision platform for intraoperative prediction and analytics
John Lee, Jianbo Shi, Vivek Buch
Congress of Neurosurgeons Annual Meeting, 2022   (Oral Presentation)
In this study, we aim to design and develop an AI based computer vision architecture for automated pixelwise object prediction, tracking, and novel analytics during endoscopic skull base surgery.
In a sparse labeling paradigm, we design and develop a custom deep computer vision-based instance segmentation architecture to predict and track anatomical structures and surgical objects with high accuracy. We create a novel metric, the “pulsatility index”, which is able to quantify the nerve-artery interface for the first time.
ECCV 2022, Neurips 2022, CVPR 2023, ICCV 2023
I have been fortunate to work with and mentor students in my research. Please feel free to reach out through email if you are interested to work with me.
Past Students - Hari Santhanam(ROBO Masters, UPenn),Abhinav Atrishi (ROBO Masters, UPenn)