Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfalls

arXiv:2405.01851v1 Announce Type: new
Abstract: There is a growing demand to deploy computation-intensive deep learning (DL) models on resource-constrained mobile devices for real-time intelligent applications. Equipped with a variety of processing units such as CPUs, GPUs, and NPUs, the mobile devices hold potential to accelerate DL inference via parallel execution across heterogeneous processors. Various efficient parallel methods have been explored to optimize computation distribution, achieve load balance, and minimize communication cost across processors. Yet their practical effectiveness in the dynamic …

abstract applications arxiv computation cpus cs.ai cs.lg deep learning deep learning inference demand deploy devices gpus inference intelligent mobile mobile devices processing processors real-time type units via