LLM2Loss: Leveraging Language Models for Explainable Model Diagnostics. (arXiv:2305.03212v1 [cs.CV])

Trained on a vast amount of data, Large Language models (LLMs) have achieved
unprecedented success and generalization in modeling fairly complex textual
inputs in the abstract space, making them powerful tools for zero-shot
learning. Such capability is extended to other modalities such as the visual
domain using cross-modal foundation models such as CLIP, and as a result,
semantically meaningful representation are extractable from visual inputs.

In this work, we leverage this capability and propose an approach that can
provide semantic …

abstract arxiv data diagnostics foundation language language models large language models llms making modeling space success tools