Scaling Laws For Dense Retrieval

arXiv:2403.18684v1 Announce Type: cross
Abstract: Scaling up neural models has yielded significant advancements in a wide array of tasks, particularly in language generation. Previous studies have found that the performance of neural models frequently adheres to predictable scaling laws, correlated with factors such as training set size and model size. This insight is invaluable, especially as large-scale experiments grow increasingly resource-intensive. Yet, such scaling law has not been fully explored in dense retrieval due to the discrete nature of retrieval …

abstract array arxiv cs.cl cs.ir found insight language language generation laws performance retrieval scaling scaling up set studies tasks training type