Every time an AI picks the next word, it's not sure — it runs a race. It gives every possible word a score, turns the scores into percent chances, and the fastest one usually wins. But close races can flip. Let's start the engines.
This little AI read a bunch of sentences. Pick a starting word below and it lines up the words that could come next — longest bar = biggest chance. Change the word and the whole race re-ranks in a flash.
After “the”, the race for the next word looks like this:
Where do those percents come from? Each word first gets a raw score. Type any 3 scores — the machine turns them into chances that always add up to 100%. A bigger score wins more chance, but never all of it.
Some races are photo-finishes. After the word “the”, the top two words are almost tied. When it's that close, the model's pick is basically a coin flip — run it again and a different word can win.
The bars are hidden. You get a starting word and 4 possible next words. Which one has the highest chance? Tap your bet, then reveal the real race. Call it right 3 times to win the badge!
Our race lines up a handful of words. Real AI runs this exact race over a giant list of words — every single step. Here's how the scoring grew up:
Count how often each word followed another and pick the most common. ✅ Dead simple. ⚠️ Never saw a word pair? Score is zero — it just breaks.
Give every word a score, then softmax turns them into a tidy set of chances that add to 100%. ✅ Always a real probability. ✅ Handles ties gracefully.
A transformer outputs a logit for every token — tens of thousands of them — each step, then softmax picks. ✅ Fluent, flexible. ⚠️ Huge compute per word.
The race above used counts. This is SmolLM2, a real neural network small enough for a basic laptop, running on your device — every bar is a real probability it computed for the next token. Change the context and watch the whole race re-rank for real. (~180 MB one-time download; then offline.)
You learned the secret behind every word an AI writes: it scores every option (logits), turns the scores into percent chances (softmax), and picks a winner from that probability distribution. The favorite usually wins — but it's a race, not a sure thing.