Reading workspace demo
Why a learning rate can feel fast and still learn slowly
Build the mental model
Gradient descent repeatedly asks one local question: which direction makes the loss smaller from where the model is now? The learning rate decides how much confidence to place in that answer.
A tiny step usually moves in the right direction, but may need many updates. A large step covers more ground, but the slope can change before the optimizer lands. The useful setting is therefore not “as large as possible”; it is large enough to make progress without destroying the signal.
Read the loss curve
The curve below is deliberately simple. Its minimum sits at zero and the slope gets steeper as the parameter moves away from that point. This lets you isolate the behavior of the optimizer before adding the complexity of a real network.
Run the experiment
Start near 0.18, then increase the learning rate. Look at the path between points, not only the final loss. Oscillation is visible before it becomes catastrophic.
Tune gradient descent
Change the learning rate and iteration count. Watch the optimizer settle, oscillate, or overshoot the minimum.
The optimizer is moving efficiently.
Take it further
Real loss landscapes have many dimensions, noisy gradients, and changing curvature. The same intuition still transfers: observe the update path, choose a stable scale, and use schedules or adaptive optimizers when one global step size cannot serve every stage of training.