Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction

arXiv:2212.10564v3 Announce Type: replace-cross
Abstract: Are multimodal inputs necessary for grammar induction? Recent work has shown that multimodal training inputs can improve grammar induction. However, these improvements are based on comparisons to weak text-only baselines that were trained on relatively little textual data. To determine whether multimodal inputs are needed in regimes with large amounts of textual training data, we design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG is a C-PFCG that incorporates em-beddings from text-only …

abstract arxiv cs.ai cs.cl cs.lg data grammar however improvements inputs multimodal text textual training type unsupervised work