Learning Discrete Data with Harmoniums: Part I, The Essentials

From the Archives: Generative AI in the ‘00s

I want to take you back to the last generative AI episode, in the early ’00s. During this time, Geoff Hinton, one of the founding fathers of deep learning, published an influential paper detailing the contrastive divergence algorithm [1]. This discovery allowed Smolensky’s harmonium [2] — which Hinton called the restricted Boltzmann machine — to be trained efficiently. It was soon realised that this model could be used for all sorts of …

algorithm binary-data data deep learning discovery divergence generative geoff geoff hinton hinton machine learning paper part programming