Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement

arXiv:2404.04627v1 Announce Type: new
Abstract: Visual program synthesis is a promising approach to exploit the reasoning abilities of large language models for compositional computer vision tasks. Previous work has used few-shot prompting with frozen LLMs to synthesize visual programs. Training an LLM to write better visual programs is an attractive prospect, but it is unclear how to accomplish this. No dataset of visual programs for training exists, and acquisition of a visual program dataset cannot be easily crowdsourced due to …

arxiv cs.cv language language models large language large language models reinforcement self-training synthesis training type visual