Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer

arXiv:2405.03882v1 Announce Type: new
Abstract: Motivated by the huge success of Transformers in the field of natural language processing (NLP), Vision Transformers (ViTs) have been rapidly developed and achieved remarkable performance in various computer vision tasks. However, their huge model sizes and intensive computations hinder ViTs' deployment on embedded devices, calling for effective model compression methods, such as quantization. Unfortunately, due to the existence of hardware-unfriendly and quantization-sensitive non-linear operations, particularly {Softmax}, it is non-trivial to completely quantize all operations …

abstract arxiv computer computer vision cs.ai cs.cv deployment devices embedded embedded devices free hinder however language language processing natural natural language natural language processing nlp performance processing quantization softmax success tasks training transformer transformers type vision vision transformers vit