Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models

arXiv:2403.17902v1 Announce Type: cross
Abstract: The landscape of computational building blocks of efficient image restoration architectures is dominated by a combination of convolutional processing and various attention mechanisms. However, convolutional filters are inherently local and therefore struggle at modeling long-range dependencies in images. On the other hand, attention excels at capturing global interactions between arbitrary image regions, however at a quadratic cost in image dimension. In this work, we propose Serpent, an architecture that leverages recent advances in state space …

abstract architectures arxiv attention attention mechanisms building combination computational cs.cv cs.lg dependencies eess.iv filters however image image restoration images landscape modeling processing scalable scale space state state space models struggle type via