SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation

arXiv:2403.07088v1 Announce Type: new
Abstract: Large language models(LLMs) have shown its outperforming ability on various tasks and question answering. However, LLMs require high computation cost and large memory cost. At the same time, LLMs may cause privacy leakage when training or prediction procedure contains sensitive information. In this paper, we propose SPA(Side Plugin Adaption), a lightweight architecture for fast on-devices inference and privacy retaining on the constraints of strict on-devices computation and memory constraints. Compared with other on-devices seq2seq generation, …

abstract arxiv cloud collaboration computation computational cost cs.cl devices however information language language models large language large language models llms memory personalized prediction privacy question question answering seq2seq spa tasks training type