HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayes

arXiv:2401.00365v2 Announce Type: replace
Abstract: Vector quantization (VQ) is a technique to deterministically learn features with discrete codebook representations. It is commonly performed with a variational autoencoding model, VQ-VAE, which can be further extended to hierarchical structures for making high-fidelity reconstructions. However, such hierarchical extensions of VQ-VAE often suffer from the codebook/layer collapse issue, where the codebook is not efficiently used to express the data, and hence degrades reconstruction accuracy. To mitigate this problem, we propose a novel unified framework …

abstract arxiv bayes cs.ai cs.cv cs.lg extensions features fidelity hierarchical however layer learn making quantization representation representation learning type vae vector