Broad intuition · Study map 27 of 85
Sampling, reasoning, and inference-time compute
Suggested preparation: Preferences, rewards, and alignment
Picture it
An autoregressive LLM proposes a distribution, picks a token, then repeats with that token added. Temperature and top-p change the selection. Longer reasoning, search, and verification spend more computation before delivering an answer.
What to understand
- Greedy decoding, temperature, top-k/top-p, beam search
- Reasoning budgets, candidate generation, verifiers, and stopping
Follow the references
1 · Course
Hugging Face · LLM course ↗Find generation strategies and compare outputs using fixed prompts.
2 · Book chapter
How To Scale Your Model · Transformer inference ↗Read the sampling loop and KV-cache introduction before the performance sections.
Keep the distinction. More reasoning tokens can help some tasks, but do not guarantee correctness; evaluate both quality and total generated tokens.
You understand it when…
Why can a shorter final answer still require much more inference work?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.