Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU. (arXiv:2210.12499v1 [cs.CL])

Curriculum Learning (CL) is a technique of training models via ranking
examples in a typically increasing difficulty trend with the aim of
accelerating convergence and improving generalisability. Current approaches for
Natural Language Understanding (NLU) tasks use CL to improve in-distribution
data performance often via heuristic-oriented or task-agnostic difficulties. In
this work, instead, we employ CL for NLU by taking advantage of training
dynamics as difficulty metrics, i.e., statistics that measure the behavior of
the model at hand on specific task-data …

arxiv cross-lingual curriculum curriculum learning dynamics nlu study training