[R][P] Ultra High Capacity Variational Autoencoder for Image Modelling

I am currently working on a fully-convolutional variational autoencoder for modelling images. The catch is that it is extremely small - my MNIST model is just under 700KB and the CIFAR10 model is just over 1MB.

On binarized MNIST, I am getting \~105 negative ELBO, and on CIFAR I am getting about 6.5 BPD.

Here are some reconstructions from both models, which converged in under 1 hour on a single T4 GPU.

[CIFAR10 reconstructions](https://preview.redd.it/1mqu4cluszhc1.png?width=274&format=png&auto=webp&s=baf4c12b2c2a17468eee6e164371c50b00f65136)

[Binarized MNIST reconstructions](https://preview.redd.it/4vodlg6vszhc1.png?width=242&format=png&auto=webp&s=0b6c439ecc45647929402287b1f6e975badaefa6)

[Samples from GMM …

autoencoder capacity image images machinelearning mnist modelling negative small