Scaling Law for Recommendation Models: Towards General-purpose User Representations. (arXiv:2111.11294v5 [cs.IR] UPDATED)

Recent advancement of large-scale pretrained models such as BERT, GPT-3,
CLIP, and Gopher, has shown astonishing achievements across various task
domains. Unlike vision recognition and language models, studies on
general-purpose user representation at scale still remain underexplored. Here
we explore the possibility of general-purpose user representation learning by
training a universal user encoder at large scales. We demonstrate that the
scaling law is present in user representation learning areas, where the
training error scales as a power-law with the amount …

arxiv general law recommendation scaling scaling law