Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing. (arXiv:2205.12253v1 [cs.CL])

Despite their strong performance on many tasks, pre-trained language models
have been shown to struggle on out-of-distribution compositional
generalization. Meanwhile, recent work has shown considerable improvements on
many NLP tasks from model scaling. Can scaling up model size also improve
compositional generalization in semantic parsing? We evaluate encoder-decoder
models up to 11B parameters and decoder-only models up to 540B parameters, and
compare model scaling curves for three different methods for transfer learning:
fine-tuning all parameters, prompt tuning, and in-context learning. …

arxiv impact parsing scale semantic