Automatic Report Generation for Histopathology images using pre-trained Vision Transformers and BERT

arXiv:2312.01435v2 Announce Type: replace
Abstract: Deep learning for histopathology has been successfully used for disease classification, image segmentation and more. However, combining image and text modalities using current state-of-the-art (SOTA) methods has been a challenge due to the high resolution of histopathology images. Automatic report generation for histopathology images is one such challenge. In this work, we show that using an existing pre-trained Vision Transformer (ViT) to encode 4096x4096 sized patches of the Whole Slide Image (WSI) and a pre-trained …

abstract art arxiv bert challenge classification cs.cv current deep learning disease however image images report segmentation sota state text transformers type vision vision transformers