SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile

arXiv:2404.13528v1 Announce Type: new
Abstract: This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performing inference on mobile devices. Focusing on emerging transformers (specifically the ones with computationally efficient Swin-like architectures) and large models (e.g., Stable Diffusion and LLMs) based on transformers, we observe that layout transformations between the computational operators cause a significant slowdown in these applications. This paper presents SmartMem, a comprehensive framework …

abstract applications architectures arxiv chatgpt cs.ai cs.dc cs.lg devices dnn inference mobile mobile devices networks neural networks ones swin transformation transformer transformers type work