A Mixed-Integer Programming Approach to Training Dense Neural Networks. (arXiv:2201.00723v2 [cs.LG] UPDATED)

Artificial Neural Networks (ANNs) are prevalent machine learning models that
are applied across various real-world classification tasks. However, training
ANNs is time-consuming and the resulting models take a lot of memory to deploy.
In order to train more parsimonious ANNs, we propose a novel mixed-integer
programming (MIP) formulation for training fully-connected ANNs. Our
formulations can account for both binary and rectified linear unit (ReLU)
activations, and for the use of a log-likelihood loss. We present numerical
experiments comparing our MIP-based …

arxiv lg mixed networks neural networks programming training