Deep Reinforcement Learning based Online Scheduling Policy for Deep Neural Network Multi-Tenant Multi-Accelerator Systems

arXiv:2404.08950v1 Announce Type: cross
Abstract: Currently, there is a growing trend of outsourcing the execution of DNNs to cloud services. For service providers, managing multi-tenancy and ensuring high-quality service delivery, particularly in meeting stringent execution time constraints, assumes paramount importance, all while endeavoring to maintain cost-effectiveness. In this context, the utilization of heterogeneous multi-accelerator systems becomes increasingly relevant. This paper presents RELMAS, a low-overhead deep reinforcement learning algorithm designed for the online scheduling of DNNs in multi-tenant environments, taking into …

abstract accelerator arxiv cloud cloud services constraints cost cs.ar cs.dc cs.lg deep neural network delivery importance network neural network outsourcing policy quality reinforcement reinforcement learning scheduling service service providers services systems trend type