Sample as You Infer: Predictive Coding With Langevin Dynamics

We present a novel algorithm for parameter learning in generic deep generative models that builds upon the predictive coding (PC) framework of computational neuroscience. Our approach modifies the standard PC algorithm to bring performance on-par and exceeding that obtained from standard variational auto-encoder (VAE) training. By injecting Gaussian noise into the PC inference procedure we re-envision it as an overdamped Langevin sampling, which facilitates optimisation with respect to a tight evidence lower bound (ELBO). We improve the resultant encoder-free training …

algorithm auto coding computational computational neuroscience cs.ai cs.cv cs.lg cs.ne deep generative models dynamics encoder framework generative generative models neuroscience noise novel performance predictive sample standard training vae