Gradient descent, visualized
Gradient descent is the algorithm that trains almost every machine-learning model, from linear regression to giant neural networks. It works by repeatedly nudging the model's parameters in the direction that reduces error the fastest — like walking downhill in fog by always stepping toward the steepest descent.
The one knob that decides whether it works is the learning rate. Too small and training crawls; too large and it overshoots the minimum and diverges. This tool lets you feel that trade-off directly instead of reading about it.
Set a learning rate, then Step downhill
Loss surface
Step 0
loss = 6.25
x = -2.5, gradient = -5 · One minimum at x = 0. Convex — descent always finds it.
Free · runs entirely in your browser · nothing to install
How to use it
- Press Step to take one descent step and watch the point move toward the minimum.
- Increase the learning rate and step again — see how it overshoots and bounces.
- Change the starting point to check it still finds the bottom of the bowl.
What you'll take away
- Why the learning rate is the single most important training hyperparameter.
- What 'convergence' and 'divergence' actually look like.
- The intuition that carries straight into training neural networks.
Want to actually build this?
This demo is one moment inside a full Math to Machine lesson — predict, build, and explain the concept, with an AI tutor that gives hints, not answers. The first five lessons are free.
FAQ
- What is gradient descent in simple terms?
- It's a step-by-step method for finding the lowest point of an error curve. At each step you move in the direction that reduces the error most, controlled by a step size called the learning rate, until you reach the minimum.
- What happens if the learning rate is too high?
- The steps are so large that each one jumps past the minimum, so the error bounces around or grows instead of shrinking. This is called divergence — you can reproduce it in the tool by cranking the learning rate up.