Wukong: Towards a Scaling Law for Large-Scale Recommendation

arXiv:2403.02545v1 Announce Type: new
Abstract: Scaling laws play an instrumental role in the sustainable improvement in model quality. Unfortunately, recommendation models to date do not exhibit such laws similar to those observed in the domain of large language models, due to the inefficiencies of their upscaling mechanisms. This limitation poses significant challenges in adapting these models to increasingly more complex real-world datasets. In this paper, we propose an effective network architecture based purely on stacked factorization machines, and a synergistic …

abstract arxiv challenges cs.ai cs.lg domain improvement language language models large language large language models law laws quality recommendation role scale scaling scaling law sustainable type upscaling