Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models

arXiv:2404.07973v1 Announce Type: new
Abstract: While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks. In this work, we unveil Ferret-v2, a significant upgrade to Ferret, with three key designs. (1) Any resolution grounding and referring: A flexible approach that effortlessly handles higher image resolution, improving the model's ability to process and …

abstract arxiv capability cs.cv encoder ferret language language model language models large language large language model large language models limitations llm regional tasks type understanding visual work