Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models

arXiv:2402.10353v1 Announce Type: cross
Abstract: Prompt learning is susceptible to intrinsic bias present in pre-trained language models (LMs), resulting in sub-optimal performance of prompt-based zero/few-shot learning. In this work, we propose a null-input prompting method to calibrate intrinsic bias encoded in pre-trained LMs. Different from prior efforts that address intrinsic bias primarily for social fairness and often involve excessive computational cost, our objective is to explore enhancing LMs' performance in downstream zero/few-shot learning while emphasizing the efficiency of intrinsic bias …

abstract arxiv bias cs.cl cs.lg few-shot few-shot learning intrinsic language language models lms null performance prior prompt prompting prompt learning type work