GODEL: Large-Scale Pre-Training for Goal-Directed Dialog. (arXiv:2206.11309v1 [cs.CL])

We introduce GODEL (Grounded Open Dialogue Language Model), a large
pre-trained language model for dialog. In contrast with earlier models such as
DialoGPT, GODEL leverages a new phase of grounded pre-training designed to
better support adapting GODEL to a wide range of downstream dialog tasks that
require information external to the current conversation (e.g., a database or
document) to produce good responses. Experiments against an array of benchmarks
that encompass task-oriented dialog, conversational QA, and grounded
open-domain dialog show that …

arxiv pre-training scale training