An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViT

arXiv:2403.20230v1 Announce Type: cross
Abstract: Vision Transformers (ViTs) have achieved significant success in computer vision. However, their intensive computations and massive memory footprint challenge ViTs' deployment on embedded devices, calling for efficient ViTs. Among them, EfficientViT, the state-of-the-art one, features a Convolution-Transformer hybrid architecture, enhancing both accuracy and hardware efficiency. Unfortunately, existing accelerators cannot fully exploit the hardware benefits of EfficientViT due to its unique architecture. In this paper, we propose an FPGA-based accelerator for EfficientViT to advance the hardware …

abstract accelerator accuracy architecture art arxiv challenge computer computer vision convolution cs.ar cs.lg deployment devices efficiency efficientvit embedded embedded devices features fpga hardware however hybrid massive memory reconfigurable state success them transformer transformers type vision vision transformers