ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks

Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily focused on sentence-level classification tasks, and only a few considered token-level labeling tasks such as Named Entity Recognition (NER) and Part-of-Speech (POS) tagging. In this paper, we propose Token-Level Prompt Decomposition (ToPro), which facilitates the prompt-based method for token-level sequence labeling tasks. The ToPro method decomposes an input sentence into single tokens and applies one prompt template to each token. …

classification cross-lingual cs.cl labeling language language models multilingual ner paper part part-of-speech prompt recognition speech studies tagging tasks token understanding zero-shot