Cross-modality debiasing: using language to mitigate sub-population shifts in imaging

arXiv:2403.07888v1 Announce Type: new
Abstract: Sub-population shift is a specific type of domain shift that highlights changes in data distribution within specific sub-groups or populations between training and testing. Sub-population shift accounts for a significant source of algorithmic bias and calls for distributional robustness. Recent studies found inherent distributional robustness in multi-modality foundation models, such as the vision-language model CLIP, yet this robustness is vulnerable through parameter fine-tuning. In this paper, we propose leveraging the connection of robustness among different …

abstract algorithmic bias arxiv bias cs.ai cs.cv data distribution domain found highlights imaging language population robustness shift studies testing training type