Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks with Soft-Thresholding. (arXiv:2304.06959v1 [cs.LG])

Soft-thresholding has been widely used in neural networks. Its basic network
structure is a two-layer convolution neural network with soft-thresholding. Due
to the network's nature of nonlinearity and nonconvexity, the training process
heavily depends on an appropriate initialization of network parameters,
resulting in the difficulty of obtaining a globally optimal solution. To
address this issue, a convex dual network is designed here. We theoretically
analyze the network convexity and numerically confirm that the strong duality
holds. This conclusion is further …

analysis analyze arxiv convolution convolutional neural networks convolution neural network denoising linear nature network networks neural network neural networks process solution theory thresholding training