Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models

arXiv:2403.16915v1 Announce Type: cross
Abstract: Fine-tuning in information retrieval systems using pre-trained language models (PLM-based IR) requires learning query representations and query-document relations, in addition to downstream task-specific learning. This study introduces coarse-tuning as an intermediate learning stage that bridges pre-training and fine-tuning. By learning query representations and query-document relations in coarse-tuning, we aim to reduce the load of fine-tuning and improve the learning effect of downstream IR tasks. We propose Query-Document Pair Prediction (QDPP) for coarse-tuning, which predicts the …

abstract arxiv cs.ai cs.cl cs.ir cs.lg document fine-tuning information intermediate language language models pre-training query relations retrieval stage study systems training type