โšก Activation Functions & Loss Functions

Explore how activations shape neuron outputs and how loss functions measure prediction error โ€” then see how they chain together in a forward-backward pass.

๐Ÿ“ˆ Activation Functions

Toggle functions on/off. Drag the input x slider to see how each activation transforms the input and what gradient it produces. Watch for vanishing gradients (flat zones) and dead neurons (zero output regions).

1.00
0.10
f(x) โ€” Activation Output
f'(x) โ€” Derivative / Gradient
Select activations above and drag the x slider.

๐Ÿ“‰ Loss Functions

Drag predicted value to see how each loss function penalizes the error. Notice how cross-entropy heavily penalizes confident wrong predictions, while MSE grows quadratically.

0.70
1.0
Loss Value vs Predicted ลท
Gradient โˆ‚L/โˆ‚ลท vs Predicted ลท
Toggle loss functions and drag the predicted value slider.

๐Ÿ”— Activation โ†’ Loss Pipeline

See a complete forward-backward pass: raw logit z โ†’ activation a = f(z) โ†’ loss L(a, y) โ†’ gradient โˆ‚L/โˆ‚z = โˆ‚L/โˆ‚a ยท f'(z). Observe how activation choice affects gradient flow.

0.50
Drag the logit z slider to see the full forward โ†’ backward pipeline.