Instrument-tissue Interaction Detection Framework for Surgical Video Understanding

arXiv:2404.00322v1 Announce Type: new
Abstract: Instrument-tissue interaction detection task, which helps understand surgical activities, is vital for constructing computer-assisted surgery systems but with many challenges. Firstly, most models represent instrument-tissue interaction in a coarse-grained way which only focuses on classification and lacks the ability to automatically detect instruments and tissues. Secondly, existing works do not fully consider relations between intra- and inter-frame of instruments and tissues. In the paper, we propose to represent instrument-tissue interaction as quintuple and present an …

abstract arxiv challenges classification computer cs.cv detection framework surgery systems type understanding video video understanding vital