Modulating Bottom-Up and Top-Down Visual Processing via Language-Conditional Filters. (arXiv:2003.12739v3 [cs.CV] UPDATED)

How to best integrate linguistic and perceptual processing in multi-modal
tasks that involve language and vision is an important open problem. In this
work, we argue that the common practice of using language in a top-down manner,
to direct visual attention over high-level visual features, may not be optimal.
We hypothesize that the use of language to also condition the bottom-up
processing from pixels to high-level features can provide benefits to the
overall performance. To support our claim, we propose …

arxiv cv filters language processing top visual processing