Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Rooms

arXiv:2404.09231v1 Announce Type: new
Abstract: A comprehensive understanding of surgical scenes allows for monitoring of the surgical process, reducing the occurrence of accidents and enhancing efficiency for medical professionals. Semantic modeling within operating rooms, as a scene graph generation (SGG) task, is challenging since it involves consecutive recognition of subtle surgical actions over prolonged periods. To address this challenge, we propose a Tri-modal (i.e., images, point clouds, and language) confluence with Temporal dynamics framework, termed TriTemp-OR. Diverging from previous approaches …

abstract accidents arxiv confluence cs.cv dynamics efficiency graph medical modal modeling monitoring process professionals recognition semantic semantic modeling temporal type understanding