Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks World

arXiv:2404.00246v1 Announce Type: new
Abstract: Language agents that interact with the world on their own have great potential for automating digital tasks. While large language model (LLM) agents have made progress in understanding and executing tasks such as textual games and webpage control, many real-world tasks also require collaboration with humans or other LLMs in equal roles, which involves intent understanding, task coordination, and communication. To test LLM's ability to collaborate, we design a blocks-world environment, where two agents, each …

abstract agents arxiv capabilities collaborative control cs.ai cs.cl cs.hc digital games language language model language models large language large language model llm matter progress tasks textual type understanding workers world