Q-ViT: Fully Differentiable Quantization for Vision Transformer. (arXiv:2201.07703v1 [cs.CV])

In this paper, we propose a fully differentiable quantization method for
vision transformer (ViT) named as Q-ViT, in which both of the quantization
scales and bit-widths are learnable parameters. Specifically, based on our
observation that heads in ViT display different quantization robustness, we
leverage head-wise bit-width to squeeze the size of Q-ViT while preserving
performance. In addition, we propose a novel technique named switchable scale
to resolve the convergence problem in the joint training of quantization scales
and bit-widths. In …

arxiv cv transformer vision