Multi-Objective Optimization for Sparse Deep Multi-Task Learning

arXiv:2308.12243v4 Announce Type: replace
Abstract: Different conflicting optimization criteria arise naturally in various Deep Learning scenarios. These can address different main tasks (i.e., in the setting of Multi-Task Learning), but also main and secondary tasks such as loss minimization versus sparsity. The usual approach is a simple weighting of the criteria, which formally only works in the convex setting. In this paper, we present a Multi-Objective Optimization algorithm using a modified Weighted Chebyshev scalarization for training Deep Neural Networks (DNNs) …

arxiv cs.ai cs.lg math.oc multi-objective multi-task learning optimization type