Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey

arXiv:2405.00314v1 Announce Type: new
Abstract: Vision Transformers (ViTs) have recently garnered considerable attention, emerging as a promising alternative to convolutional neural networks (CNNs) in several vision-related applications. However, their large model sizes and high computational and memory demands hinder deployment, especially on resource-constrained devices. This underscores the necessity of algorithm-hardware co-design specific to ViTs, aiming to optimize their performance by tailoring both the algorithmic structure and the underlying hardware accelerator to each other's strengths. Model quantization, by converting high-precision numbers …

arxiv cs.ai cs.ar cs.cv cs.lg cs.pf hardware quantization survey transformers type vision vision transformers