Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation

arXiv:2404.19752v1 Announce Type: new
Abstract: Existing automatic captioning methods for visual content face challenges such as lack of detail, content hallucination, and poor instruction following. In this work, we propose VisualFactChecker (VFC), a flexible training-free pipeline that generates high-fidelity and detailed captions for both 2D images and 3D objects. VFC consists of three steps: 1) proposal, where image-to-text captioning models propose multiple initial captions; 2) verification, where a large language model (LLM) utilizes tools such as object detection and VQA …

3d objects abstract arxiv captioning captions challenges cs.cv enabling face fidelity free hallucination images objects pipeline training type visual work