SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators

arXiv:2405.00790v1 Announce Type: cross
Abstract: Emerging multi-model workloads with heavy models like recent large language models significantly increased the compute and memory demands on hardware. To address such increasing demands, designing a scalable hardware architecture became a key problem. Among recent solutions, the 2.5D silicon interposer multi-chip module (MCM)-based AI accelerator has been actively explored as a promising scalable solution due to their significant benefits in the low engineering cost and composability. However, previous MCM accelerators are based on homogeneous …

abstract accelerators ai workloads architecture arxiv chip compute cs.ai cs.ar cs.dc cs.lg cs.pf designing hardware key language language models large language large language models memory scalable scheduling silicon solutions type workloads