HyperVQ: MLR-based Vector Quantization in Hyperbolic Space

arXiv:2403.13015v1 Announce Type: cross
Abstract: The success of models operating on tokenized data has led to an increased demand for effective tokenization methods, particularly when applied to vision or auditory tasks, which inherently involve non-discrete data. One of the most popular tokenization methods is Vector Quantization (VQ), a key component of several recent state-of-the-art methods across various domains. Typically, a VQ Variational Autoencoder (VQVAE) is trained to transform data to and from its tokenized representation. However, since the VQVAE is …

abstract arxiv cs.lg data demand eess.iv key popular quantization space success tasks tokenization type vector vision