MM-SurvNet: Deep Learning-Based Survival Risk Stratification in Breast Cancer Through Multimodal Data Fusion

arXiv:2402.11788v1 Announce Type: new
Abstract: Survival risk stratification is an important step in clinical decision making for breast cancer management. We propose a novel deep learning approach for this purpose by integrating histopathological imaging, genetic and clinical data. It employs vision transformers, specifically the MaxViT model, for image feature extraction, and self-attention to capture intricate image relationships at the patient level. A dual cross-attention mechanism fuses these features with genetic data, while clinical data is incorporated at the final layer …

abstract arxiv cancer clinical cs.ai cs.cv data decision decision making deep learning fusion imaging making management multimodal multimodal data novel risk survival through transformers type vision vision transformers