A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks

Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. Training such models, however, is quite inefficient and unstable. In this work, we show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one, and has theoretical guarantees in terms of convergence. The proposed algorithm, that we call incremental predictive coding (iPC) is also more biologically …

algorithm bayesian coding cs.ai cs.lg cs.ne leads networks neuroscience predictive scheduling show statistics temporal training update work