An Analytical Theory of Curriculum Learning in Teacher-Student Networks. (arXiv:2106.08068v2 [cs.LG] UPDATED)

In humans and animals, curriculum learning -- presenting data in a curated
order - is critical to rapid learning and effective pedagogy. Yet in machine
learning, curricula are not widely used and empirically often yield only
moderate benefits. This stark difference in the importance of curriculum raises
a fundamental theoretical question: when and why does curriculum learning help?

In this work, we analyse a prototypical neural network model of curriculum
learning in the high-dimensional limit, employing statistical physics methods.
Curricula …

arxiv curriculum curriculum learning networks theory