SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking

arXiv:2306.05426v3 Announce Type: replace
Abstract: In many domains, autoregressive models can attain high likelihood on the task of predicting the next observation. However, this maximum-likelihood (MLE) objective does not necessarily match a downstream use-case of autoregressively generating high-quality sequences. The MLE objective weights sequences proportionally to their frequency under the data distribution, with no guidance for the model's behaviour out of distribution (OOD): leading to compounding error during autoregressive generation. In order to address this compounding error problem, we formulate …

abstract arxiv autoregressive autoregressive models backtracking case cs.ai cs.lg data domains however imitation learning likelihood match maximum maximum-likelihood mle modelling next observation quality type