OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses

arXiv:2204.02426v5 Announce Type: replace
Abstract: Dataset bias and spurious correlations can significantly impair generalization in deep neural networks. Many prior efforts have addressed this problem using either alternative loss functions or sampling strategies that focus on rare patterns. We propose a new direction: modifying the network architecture to impose inductive biases that make the network robust to dataset bias. Specifically, we propose OccamNets, which are biased to favor simpler solutions by design. OccamNets have two inductive biases. First, they are …

abstract architecture arxiv bias biases correlations cs.lg dataset focus functions inductive loss network network architecture networks neural networks patterns prior sampling strategies type