Retrieval-Augmented Transformer for Image Captioning. (arXiv:2207.13162v1 [cs.CV])

Image captioning models aim at connecting Vision and Language by providing
natural language descriptions of input images. In the past few years, the task
has been tackled by learning parametric models and proposing visual feature
extraction advancements or by modeling better multi-modal connections. In this
paper, we investigate the development of an image captioning approach with a
kNN memory, with which knowledge can be retrieved from an external corpus to
aid the generation process. Our architecture combines a knowledge retriever …

arxiv captioning cv image retrieval transformer