Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses?

arXiv:2405.07099v1 Announce Type: new
Abstract: Semitic morphologically-rich languages (MRLs) are characterized by extreme word ambiguity. Because most vowels are omitted in standard texts, many of the words are homographs with multiple possible analyses, each with a different pronunciation and different morphosyntactic properties. This ambiguity goes beyond word-sense disambiguation (WSD), and may include token segmentation into multiple word units. Previous research on MRLs claimed that standardly trained pre-trained language models (PLMs) based on word-pieces may not sufficiently capture the internal structure …

abstract arxiv beyond cs.cl language language models languages multiple sense standard type word words