[R]: ColBERT - understanding query augmentation technique used there

[https://arxiv.org/pdf/2004.12832.pdf](https://arxiv.org/pdf/2004.12832.pdf)

Section 3.2

In the paper they propose to use \[mask\] token as padding tokens for a query, that serves as augmentation.

"We denote the padding with masked tokens as query augmentation, a step that allows BERT to produce query-based embeddings at the positions corresponding to these masks. query augmentation is intended to serve as a soft, diferentiable mechanism for learning to expand queries with new terms or to re-weigh existing terms based on their importance for matching the query." …

augmentation bert embeddings machinelearning masks padding paper query serve tokens