How to Encode Constraints to the Output of Neural Networks

A systematic review of available approaches

Image generated by ChatGPT based on this article’s content.

Neural networks are indeed powerful. However, as the application scope of neural networks moves from “standard” classification and regression tasks to more complex decision-making and AI for Science, one drawback is becoming increasingly apparent: the output of neural networks is usually unconstrained, or more precisely, constrained only by simple 0–1 bounds (Sigmoid activation function), non-negative constraints (ReLU activation function), or constraints that sum to one …

deep-dives deep learning machine learning neural networks optimization