Human Curriculum Effects Emerge with In-Context Learning in Neural Networks

Human learning is sensitive to rule-like structure and the curriculum of examples used for training. In tasks governed by succinct rules, learning is more robust when related examples are blocked across trials, but in the absence of such rules, interleaving is more effective. To date, no neural model has simultaneously captured these seemingly contradictory effects. Here we show that this same tradeoff spontaneously emerges with "in-context learning" (ICL) both in neural networks trained with metalearning and in large language models …

context cs.lg cs.ne curriculum effects examples human in-context learning interleaving networks neural networks q-bio.nc robust rules tasks training