CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update

arXiv:2312.10908v2 Announce Type: replace
Abstract: Utilizing large language models (LLMs) to compose off-the-shelf visual tools represents a promising avenue of research for developing robust visual assistants capable of addressing diverse visual tasks. However, these methods often overlook the potential for continual learning, typically by freezing the utilized tools, thus limiting their adaptation to environments requiring new knowledge. To tackle this challenge, we propose CLOVA, a Closed-Loop Visual Assistant, which operates within a framework encompassing inference, reflection, and learning phases. During …

abstract arxiv assistant assistants continual cs.cv diverse however language language models large language large language models llms loop research robust tasks tool tools type update usage visual