Training Artificial Neural Networks by Coordinate Search Algorithm

arXiv:2402.12646v1 Announce Type: new
Abstract: Training Artificial Neural Networks poses a challenging and critical problem in machine learning. Despite the effectiveness of gradient-based learning methods, such as Stochastic Gradient Descent (SGD), in training neural networks, they do have several limitations. For instance, they require differentiable activation functions, and cannot optimize a model based on several independent non-differentiable loss functions simultaneously; for example, the F1-score, which is used during testing, can be used during training when a gradient-free optimization algorithm is …

abstract algorithm artificial artificial neural networks arxiv cs.ai cs.lg differentiable functions gradient instance limitations machine machine learning networks neural networks search stochastic training type