DFWLayer: Differentiable Frank-Wolfe Optimization Layer

arXiv:2308.10806v2 Announce Type: replace
Abstract: Differentiable optimization has received a significant amount of attention due to its foundational role in the domain of machine learning based on neural networks. This paper proposes a differentiable layer, named Differentiable Frank-Wolfe Layer (DFWLayer), by rolling out the Frank-Wolfe method, a well-known optimization algorithm which can solve constrained optimization problems without projections and Hessian matrix computations, thus leading to an efficient way of dealing with large-scale convex optimization problems with norm constraints. Experimental results …

abstract algorithm arxiv attention cs.ai cs.lg differentiable domain foundational frank layer machine machine learning networks neural networks optimization paper role solve type