Interactive lesson · Transformer
How a Transformer predicts
Decoder-only Transformer · live visual
← → navigate · Space play · R replay01 / 0701 · Goal
Rank possible next tokens
Heavyrainfell,soIcarriedan___
context7 tokens
Known context
Heavy rain fell, so I carried an ___
Next token
umbrella64%
jacket19%
bag8%
other9%
Example probabilities — not output from a real model