Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures

arXiv:2405.04700v1 Announce Type: new
Abstract: Large Language Models (LLMs) deployed on edge devices learn through fine-tuning and updating a certain portion of their parameters. Although such learning methods can be optimized to reduce resource utilization, the overall required resources remain a heavy burden on edge devices. Instead, Retrieval-Augmented Generation (RAG), a resource-efficient LLM learning method, can improve the quality of the LLM-generated content without updating model parameters. However, the RAG-based LLM may involve repetitive searches on the profile data in …

abstract architectures arxiv computing cs.ai cs.ir cs.lg devices edge edge devices fine-tuning implementation in-memory language language models large language large language models learn llms memory parameters rag reduce resources retrieval retrieval-augmented robust through type