VOLTA: Improving Generative Diversity by Variational Mutual Information Maximizing Autoencoder

arXiv:2307.00852v2 Announce Type: replace
Abstract: The natural language generation domain has witnessed great success thanks to Transformer models. Although they have achieved state-of-the-art generative quality, they often neglect generative diversity. Prior attempts to tackle this issue suffer from either low model capacity or over-complicated architectures. Some recent methods employ the VAE framework to enhance diversity, but their latent variables fully depend on the input context, restricting exploration of the latent space. In this paper, we introduce VOLTA, a framework that …

abstract architectures art arxiv autoencoder capacity cs.cl diversity domain generative information issue language language generation low natural natural language natural language generation prior quality state success transformer transformer models type