MeshBrush: Painting the Anatomical Mesh with Neural Stylization for Endoscopy

arXiv:2404.02999v1 Announce Type: cross
Abstract: Style transfer is a promising approach to close the sim-to-real gap in medical endoscopy. Rendering realistic endoscopic videos by traversing pre-operative scans (such as MRI or CT) can generate realistic simulations as well as ground truth camera poses and depth maps. Although image-to-image (I2I) translation models such as CycleGAN perform well, they are unsuitable for video-to-video synthesis due to the lack of temporal consistency, resulting in artifacts between frames. We propose MeshBrush, a neural mesh …

abstract arxiv cs.cv eess.iv gap generate image image-to-image maps medical mesh mri painting rendering scans sim simulations style style transfer transfer translation truth type videos