Designing Informative Metrics for Few-Shot Example Selection

arXiv:2403.03861v1 Announce Type: cross
Abstract: Pretrained language models (PLMs) have shown remarkable few-shot learning capabilities when provided with properly formatted examples. However, selecting the "best" examples remains an open challenge. We propose a complexity-based prompt selection approach for sequence tagging tasks. This approach avoids the training of a dedicated model for selection of examples, and instead uses certain metrics to align the syntactico-semantic complexity of test sentences and examples. We use both sentence- and word-level metrics to match the complexity …

abstract arxiv capabilities challenge complexity cs.cl cs.lg designing example examples few-shot few-shot learning however language language models metrics prompt tagging tasks training type