DECap: Towards Generalized Explicit Caption Editing via Diffusion Mechanism

arXiv:2311.14920v2 Announce Type: replace
Abstract: Explicit Caption Editing (ECE) -- refining reference image captions through a sequence of explicit edit operations (e.g., KEEP, DETELE) -- has raised significant attention due to its explainable and human-like nature. After training with carefully designed reference and ground-truth caption pairs, state-of-the-art ECE models exhibit limited generalization ability beyond the original training data distribution, i.e., they are tailored to refine content details only in in-domain samples but fail to correct errors in out-of-domain samples. To …

abstract art arxiv attention captions cs.cv diffusion edit editing generalized ground-truth human human-like image nature operations reference state through training truth type via