The Gradient Descent Algorithm and the Intuition Behind It

A technical description of the Gradient Descent method, complemented with a graphical representation of the algorithm at work

1. INTRODUCING SOME KEY DEFINITIONS

Within optimization methods, and in the first order algorithm type, one has certainly heard of the one known as Gradient Descent. It is of the first-order optimization type as it requires the first-order derivative, namely the gradient. By optimizing, …

algorithm algorithms arthur author gradient gradient-descent hill intuition machine learning math optimization photo representation speed technical thoughts-and-theory type work