Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddings

arXiv:2404.04270v1 Announce Type: cross
Abstract: Training recommendation models pose significant challenges regarding resource utilization and performance. Prior research has proposed an approach that categorizes embeddings into popular and non-popular classes to reduce the training time for recommendation models. We observe that, even among the popular embeddings, certain embeddings undergo rapid training and exhibit minimal subsequent variation, resulting in saturation. Consequently, updates to these embeddings lack any contribution to model quality. This paper presents Slipstream, a software framework that identifies stale …

abstract arxiv challenges cs.ir cs.lg embeddings observe performance popular prior recommendation reduce research training type