Let Models Speak Ciphers: Multiagent Debate through Embeddings

arXiv:2310.06272v2 Announce Type: replace-cross
Abstract: Discussion and debate among Large Language Models (LLMs) have gained considerable attention due to their potential to enhance the reasoning ability of LLMs. Although natural language is an obvious choice for communication due to LLM's language understanding capability, the token sampling step needed when generating natural language poses a potential risk of information loss, as it uses only one token to represent the model's belief across the entire vocabulary. In this paper, we introduce a …

abstract arxiv attention capability communication cs.ai cs.cl cs.lg embeddings language language models language understanding large language large language models llm llms natural natural language reasoning sampling speak through token type understanding