all AI news
Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training. (arXiv:2108.09373v2 [cs.DC] UPDATED)
Jan. 20, 2022, 2:11 a.m. | Mark Zhao, Niket Agarwal, Aarti Basant, Bugra Gedik, Satadru Pan, Mustafa Ozdal, Rakesh Komuravelli, Jerry Pan, Tianshu Bao, Haowei Lu, Sundaram Naray
cs.LG updates on arXiv.org arxiv.org
Domain-specific accelerators (DSAs) are integrated in datacenter-scale
clusters across industry to train increasingly-complex deep learning models
over massive datasets. As innovations in DSAs continue to increase training
efficiency and throughput, the data storage and ingestion (DSI) pipeline, the
systems and hardware responsible for storing and preprocessing training data,
will dominate and constrain training capacity. Similar innovation in DSI is
urgent, demanding an in-depth understanding of DSI systems, infrastructure, and
characteristics.
To this end, this paper presents Meta's end-to-end DSI pipeline, …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Senior Data Analyst - SQL
@ Experian | Heredia, Costa Rica
Lead Business Intelligence Developer
@ L.A. Care Health Plan | Los Angeles, CA, US, 90017
(USA) Senior Manager, Data Analytics
@ Walmart | (USA) AR BENTONVILLE Home Office J Street Offices, Suite #2
Autonomous Haulage System Application Specialist
@ Komatsu | Belo Horizonte, BR
Machine Learning Engineer
@ GFT Technologies | Alcobendas, M, ES, 28108