Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach

arXiv:2210.12624v2 Announce Type: replace
Abstract: Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and accuracy; or, in multi-task learning where multiple tasks are optimized jointly, sharing inductive bias between them. This problems are often tackled by the multi-objective optimization framework. However, existing stochastic multi-objective gradient methods and its variants (e.g., MGDA, PCGrad, CAGrad, etc.) all adopt a biased noisy …

abstract accuracy arxiv bias cs.lg fairness functions gradient inductive machine machine learning math.oc metrics multi-objective multiple multi-task learning performance safety stat.ml stochastic tasks trade trade-off type