Triple-Encoders: Representations That Fire Together, Wire Together

arXiv:2402.12332v1 Announce Type: new
Abstract: Search-based dialog models typically re-encode the dialog history at every turn, incurring high cost. Curved Contrastive Learning, a representation learning method that encodes relative distances between utterances into the embedding space via a bi-encoder, has recently shown promising results for dialog modeling at far superior efficiency. While high efficiency is achieved through independently encoding utterances, this ignores the importance of contextualization. To overcome this issue, this study introduces triple-encoders, which efficiently compute distributed utterance mixtures …

abstract arxiv cost cs.cl dialog efficiency embedding encode encoder every fire history modeling representation representation learning search space together type via