Hyperparameter tuning via trajectory predictions: Stochastic prox-linear methods in matrix sensing

Motivated by the desire to understand stochastic algorithms for nonconvex optimization that are robust to their hyperparameter choices, we analyze a mini-batched prox-linear iterative algorithm for the problem of recovering an unknown rank-1 matrix from rank-1 Gaussian measurements corrupted by noise. We derive a deterministic recursion that predicts the error of this method and show, using a non-asymptotic framework, that this prediction is accurate for any batch-size and a large range of step-sizes. In particular, our analysis reveals that this …

algorithm algorithms analyze hyperparameter iterative linear math.oc math.st matrix noise optimization predictions recursion robust sensing stat.ml stat.th stochastic trajectory via