Consistency Training with Virtual Adversarial Discrete Perturbation. (arXiv:2104.07284v2 [cs.CL] UPDATED)

Consistency training regularizes a model by enforcing predictions of original
and perturbed inputs to be similar. Previous studies have proposed various
augmentation methods for the perturbation but are limited in that they are
agnostic to the training model. Thus, the perturbed samples may not aid in
regularization due to their ease of classification from the model. In this
context, we propose an augmentation method of adding a discrete noise that
would incur the highest divergence between predictions. This virtual
adversarial …

arxiv training virtual