Deep Dive: Filters & Repetition Penalties ▼
♻️ Repetition Penalty
To stop models from looping (e.g., "the cat the cat the cat"), this penalty artificially lowers the logits of words that have already appeared in the sequence.
Watch the bar for "The" drop dramatically when you increase the Repetition Penalty slider!
📈 Re-Normalization
If Top-K or Top-P chops off 20% of the distribution's probability mass, the remaining words no longer sum to 100%. The system must re-normalize the survivors, scaling them all up proportionally.
Notice the dashed green boxes in the Canvas! This shows exactly how much the kept words grew after the others were pruned.
✂️ Top-K Sampling
Sorts the vocabulary by probability and hard-caps the selection to the top $K$ words. All words from rank $K+1$ onwards are given exactly 0% chance.
Pro: Prevents catastrophic weird words. Con: Might arbitrarily cut off good words at the boundary.
⚛️ Top-P (Nucleus)
Creates a dynamic cutoff. It accumulates probability from left to right until the sum hits $P$ (e.g., 90%). It only keeps the "nucleus" of confident mass.
Pro: Automatically adapts. If the model is highly confident, it might only keep 2 words. If uncertain, it might keep 100.