OTClean: Data Cleaning for Conditional Independence Violations using Optimal Transport

arXiv:2403.02372v1 Announce Type: new
Abstract: Ensuring Conditional Independence (CI) constraints is pivotal for the development of fair and trustworthy machine learning models. In this paper, we introduce \sys, a framework that harnesses optimal transport theory for data repair under CI constraints. Optimal transport theory provides a rigorous framework for measuring the discrepancy between probability distributions, thereby ensuring control over data utility. We formulate the data repair problem concerning CIs as a Quadratically Constrained Linear Program (QCLP) and propose an alternating …

abstract arxiv cleaning constraints cs.ai cs.db cs.lg data data cleaning development fair framework machine machine learning machine learning models measuring paper pivotal theory transport trustworthy type