Breakpoint Transformers for Modeling and Tracking Intermediate Beliefs. (arXiv:2211.07950v1 [cs.CL])

Can we teach natural language understanding models to track their beliefs
through intermediate points in text? We propose a representation learning
framework called breakpoint modeling that allows for learning of this type.
Given any text encoder and data marked with intermediate states (breakpoints)
along with corresponding textual queries viewed as true/false propositions
(i.e., the candidate beliefs of a model, consisting of information changing
through time) our approach trains models in an efficient and end-to-end fashion
to build intermediate representations that …

arxiv breakpoint modeling tracking transformers