FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs

arXiv:2403.15593v1 Announce Type: cross
Abstract: Large pre-trained vision-language models such as CLIP provide compact and general-purpose representations of text and images that are demonstrably effective across multiple downstream zero-shot prediction tasks. However, owing to the nature of their training process, these models have the potential to 1) propagate or amplify societal biases in the training data and 2) learn to rely on spurious features. This paper proposes FairerCLIP, a general approach for making zero-shot predictions of CLIP more fair and …

abstract amplify arxiv clip cs.cv cs.lg functions general however images language language models multiple nature prediction predictions process tasks text training type vision vision-language models zero-shot