AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks. (arXiv:2211.03741v1 [stat.ML])

In this paper, we develop a new algorithm, Annealed Skewed SGD - AskewSGD -
for training deep neural networks (DNNs) with quantized weights. First, we
formulate the training of quantized neural networks (QNNs) as a smoothed
sequence of interval-constrained optimization problems. Then, we propose a new
first-order stochastic method, AskewSGD, to solve each constrained optimization
subproblem. Unlike algorithms with active sets and feasible directions,
AskewSGD avoids projections or optimization under the entire feasible set and
allows iterates that are infeasible. …

arxiv interval networks neural networks optimisation