Transformer-based Cross-Modal Recipe Embeddings with Large Batch Training. (arXiv:2205.04948v1 [cs.CV])

Web: http://arxiv.org/abs/2205.04948

In this paper, we present a cross-modal recipe retrieval framework,
Transformer-based Network for Large Batch Training (TNLBT), which is inspired
by ACME~(Adversarial Cross-Modal Embedding) and H-T~(Hierarchical Transformer).
TNLBT aims to accomplish retrieval tasks while generating images from recipe
embeddings. We apply the Hierarchical Transformer-based recipe text encoder,
the Vision Transformer~(ViT)-based recipe image encoder, and an adversarial
network architecture to enable better cross-modal embedding learning for recipe
texts and images. In addition, we use self-supervised learning to exploit the
rich information …

arxiv cross cv recipe training transformer