Multi-Prompt with Depth Partitioned Cross-Modal Learning

arXiv:2305.06221v4 Announce Type: replace
Abstract: In recent years, soft prompt learning methods have been proposed to fine-tune large-scale vision-language pre-trained models for various downstream tasks. These methods typically combine learnable textual tokens with class tokens as input for models with frozen parameters. However, they often employ a single prompt to describe class contexts, failing to capture categories' diverse attributes adequately. This study introduces the Partitioned Multi-modal Prompt (PMPO), a multi-modal prompting technique that extends the soft prompt from a single …

abstract arxiv class cs.ai cs.cv however language modal parameters pre-trained models prompt prompt learning scale tasks textual tokens type vision vision-language