Adaptive Patching for High-resolution Image Segmentation with Transformers

arXiv:2404.09707v1 Announce Type: cross
Abstract: Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. …

abstract analytics arxiv attention cs.ai cs.cv cs.lg image image analytics images linear resolution segmentation space standard tokens transformer transformers type