C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval. (arXiv:2204.11989v1 [cs.IR])

Pretrained language models have improved effectiveness on numerous tasks,
including ad-hoc retrieval. Recent work has shown that continuing to pretrain a
language model with auxiliary objectives before fine-tuning on the retrieval
task can further improve retrieval effectiveness. Unlike monolingual retrieval,
designing an appropriate auxiliary task for cross-language mappings is
challenging. To address this challenge, we use comparable Wikipedia articles in
different languages to further pretrain off-the-shelf multilingual pretrained
models before fine-tuning on the retrieval task. We show that our approach …

ad arxiv language retrieval