TMSS: An End-to-End Transformer-based Multimodal Network for Segmentation and Survival Prediction. (arXiv:2209.05036v1 [eess.IV])

When oncologists estimate cancer patient survival, they rely on multimodal
data. Even though some multimodal deep learning methods have been proposed in
the literature, the majority rely on having two or more independent networks
that share knowledge at a later stage in the overall model. On the other hand,
oncologists do not do this in their analysis but rather fuse the information in
their brain from multiple sources such as medical images and patient history.
This work proposes a deep …

arxiv multimodal network prediction segmentation survival transformer