# Temperature, top-k and top-p, in plain words *The network inside a chatbot never picks a word. It hands back a score for every token it knows, and a sampler, ordinary code outside the network, turns the scores into chances and rolls dice. Temperature, top-k, top-p, min-p and the repetition penalty are the settings on that roll. This page works one made-up example through all of them by hand, says where each idea came from, what temperature zero does and does not promise, and what three engines shipped by default on the day we read them.* *Published 2026-11-13 (UTC) · A small (human) team and a fleet of AI agents.* **the short version:** A language model's last output is a list of scores, one for every token in its vocabulary: 262,144 of them on one model we run. A sampler turns the scores into chances and draws one token. Temperature divides every score before the draw: below 1 the favourite pulls ahead, above 1 the field levels out, and at zero the sampler just takes the top score. Top-k, top-p and min-p cut off the long tail of unlikely tokens in three different ways, and a repetition penalty marks down tokens the text has already used. Three engines' documented defaults disagree: temperature 0.8 in two and 1.0 in a third, as read on 2026-11-13 (UTC). Two of them apply the steps in different orders, and a model's maker can override the defaults. Temperature zero removes the dice. It does not promise the same answer every time, and the likeliest token is not a checked fact. 2,239 words · about 10 minutes (at 220 words/min) · 2 tables · data kit: no https://research.strata2signal.com/temperature-and-sampling-in-plain-words/ --- *If you're new here: [strata→signal](https://strata2signal.com) is a small workshop that runs its own machines and writes up what it finds. This page sits beside [Draft heads and speculative decoding, in plain words](https://research.strata2signal.com/draft-heads-and-speculative-decoding/) and [Mixture of experts, in plain words](https://research.strata2signal.com/mixture-of-experts-in-plain-words/). Nothing was measured for it: every figure is a paper's or an engine's own, read on 2026-11-13 (UTC), one a page of ours already prints, or our own arithmetic, marked as ours.* ## The model does not pick the word {#the-model-does-not-pick-the-word} A language model writes one token at a time; a token is a word or a piece of one. For each, the network's whole output is a list: one raw score, called a logit, for every token in its vocabulary. On gemma4:26b, a model we run, that is 262,144 scores for every token written ([The free speed wasn't free](https://research.strata2signal.com/the-free-speed-wasnt-free/#what-the-tax-actually-was)). There the network's work ends. Separate code, the sampler, turns the scores into chances and picks. The usual conversion is called softmax: raise e (about 2.718) to the power of each score, then divide each result by the total. A higher score gets a bigger share, and the shares add up to 100 %. The simplest sampler takes the biggest share every time: greedy decoding. The samplers on this page roll dice weighted by the shares instead, and the settings below reshape the list before the roll. So "the model chose that word" covers two steps: the model set the odds, and a sampler drew. ## Temperature, worked by hand {#temperature-worked-by-hand} **An illustration: the scores are made up, and the arithmetic is ours.** The text so far is "The lighthouse keeper poured a cup of", and the vocabulary has five tokens. Temperature is one number, T; the sampler divides every score by T before the softmax step. | token | score | share at T = 0.5 | share at T = 1 | share at T = 2 | |---|---|---|---|---| | tea | 3.0 | 86.5 % | 63.6 % | 42.9 % | | coffee | 2.0 | 11.7 % | 23.4 % | 26.0 % | | cocoa | 1.0 | 1.6 % | 8.6 % | 15.8 % | | soup | 0.0 | 0.2 % | 3.2 % | 9.6 % | | rain | −1.0 | 0.03 % | 1.2 % | 5.8 % | | tokens kept by top-k 3 | · | 3 | 3 | 3 | | tokens kept by top-p 0.9 | · | 2 | 3 | 4 | | tokens kept by min-p 0.1 | · | 2 | 3 | 5 | The T = 1 column by hand: e^3 = 20.086, e^2 = 7.389, e^1 = 2.718, e^0 = 1 and e^−1 = 0.368. They total 31.561, so tea's share is 20.086 ÷ 31.561 = 63.6 %. At T = 0.5 the scores become 6, 4, 2, 0 and −2; at T = 2 they become 1.5, 1, 0.5, 0 and −0.5. A rule to keep: at T = 1, one point of score makes a token about 2.7 times as likely. At T = 0.5 a point counts double (7.4 times) and the favourite pulls away. At T = 2 it counts half (1.65 times) and "rain" comes up about one time in 17. The order never changes: no temperature above zero makes "rain" likelier than "tea". Temperature decides how often the long shots come up. The name is from physics. In a 1985 paper on their [Boltzmann machine](https://doi.org/10.1207/s15516709cog0901_7), Ackley, Hinton and Sejnowski wrote of a parameter that "acts like temperature": run hot, the network "will ignore small energy differences". Thirty years later [a paper Hinton co-wrote](https://arxiv.org/abs/1503.02531) put it in words still in use: "T is a temperature that is normally set to 1. Using a higher value for T produces a softer probability distribution over classes." ## Three ways to cut the tail {#three-ways-to-cut-the-tail} A real vocabulary is mostly wrong answers, each with a tiny chance, and together they come up often. [Holtzman and colleagues](https://arxiv.org/abs/1904.09751) (2019) called it the "unreliable tail": "tens of thousands of candidate tokens with relatively low probability that are over-represented in the aggregate". Three cuts remove it before the roll; the survivors' shares are scaled back up to 100 %. The table's last three rows apply each cut to each column. **Top-k keeps a fixed number.** [Fan, Lewis and Dauphin](https://arxiv.org/abs/1805.04833) (2018) wrote stories by sampling "from the k = 10 most likely candidates". Top-k 3 keeps three tokens however sure the model is. **Top-p keeps a fixed share.** Holtzman's paper proposed nucleus sampling: keep "the smallest set" of top tokens whose shares reach p. With p = 0.9 at T = 1: 63.6 + 23.4 = 87.0, short of 90, so cocoa joins and three stay. The same rule keeps two in the confident column and four in the flat one, "expanding and contracting the candidate pool dynamically". **Min-p keeps what is close to the favourite.** The bar is a fraction of the top share. At min-p 0.1 and T = 1 it is 6.4 %, a tenth of 63.6, and three tokens clear it; the flat column keeps all five. Min-p came up from hobbyists: it reached [llama.cpp](https://github.com/ggml-org/llama.cpp) in [a pull request](https://github.com/ggml-org/llama.cpp/pull/3841) merged on 2023-10-31 (UTC), eight months before [its paper](https://arxiv.org/abs/2407.01082) of 2024-07-01 (UTC). The paper's contribution statement says one author, Andrew Baker, "conceived the original idea of min-p sampling, tested and shared it with hobbyists within the open-source LLM community". ## The repetition penalty {#the-repetition-penalty} Greedy text tends to loop. The [CTRL paper](https://arxiv.org/abs/1909.05858) (Keskar and colleagues, 2019) proposed "discounting the scores of previously generated tokens": divide a used token's score by a penalty, and about 1.2 "yields a good balance between truthful generation and lack of repetition". If "tea" was just written, its 3.0 becomes 2.5 and its share at T = 1 falls from 63.6 % to 51.5 % (e^2.5 = 12.182; the new total is about 23.66). The rule has a flaw: dividing a negative score moves it toward zero, so "rain" at −1.0 would become −0.83 and get likelier. [llama.cpp's source](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/src/llama-sampler.cpp) says dividing "would cause tokens with negative logits to become more likely, which is obviously wrong", and multiplies those instead. Nor is the penalty free. An [Ollama](https://ollama.com) [commit](https://github.com/ollama/ollama/commit/6a261db7d87b13d76c5197cec636a0a3951afb36) written on 2026-08-11 (UTC), and on Ollama's main branch the next day, changed the default from 1.1 to 1.0, which is off, for models that set none; an always-on penalty, it says, "distorts output that legitimately repeats tokens, such as code, JSON, and long reasoning traces". On our own machine, under one Ollama build, the same request ran at 209.37 tokens a second without the penalty and 136.13 with it ([The free speed wasn't free](https://research.strata2signal.com/the-free-speed-wasnt-free/#the-hunt-change-one-thing)). ## What temperature zero does and does not promise {#what-temperature-zero-does-and-does-not-promise} Nothing divides by zero, so engines treat zero as a switch: skip the dice, take the top score. [llama.cpp's source](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/common/common.h) reads "<= 0.0 to sample greedily"; [vLLM](https://github.com/vllm-project/vllm)'s [parameter list](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/vllm/sampling_params.py) reads "Zero means greedy sampling." Three things it does not promise. **The same answer every time.** Scores can differ in their last digits between runs. llama.cpp's [server manual](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/tools/server/README.md), describing a prompt cache that is on by default, warns that "the logits are **not** guaranteed to be bit-for-bit identical for different batch sizes". vLLM's [documentation](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/docs/usage/reproducibility.md) says it "does not guarantee the reproducibility of the results by default, for the sake of performance", and its [FAQ](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/docs/usage/faq.md) adds: "Once a different token is sampled, further divergence is likely." We have seen both outcomes: ten repeats of one question byte-identical on one server ([the version pair](https://research.strata2signal.com/the-free-speed-wasnt-free/#the-version-pair-and-a-twist)), and, in another probe, short answers matching on four repeats of four and long ones on none of four, tracking what had run just before ([It’s Not a Metaphor](https://research.strata2signal.com/compression-is-the-objective/#what-we-can-show-you)). **The same answer under other settings.** Zero switches off the dice, not the rest: in llama.cpp's default chain the penalties still run first, and only then is the top score taken. In that ten-repeat test two releases of one engine, each at its own defaults, each repeated itself perfectly and disagreed with the other: 128 tokens against 130 ([the same page](https://research.strata2signal.com/the-free-speed-wasnt-free/#the-part-that-isnt-about-speed)). The default repetition penalty had changed between them, but so had the whole release, and that page could not say which made the difference. **A true answer.** The top score marks the likeliest next token, not a checked fact. At temperature 0 a vision model of ours quoted a line that is not on the page, differently under each of two prompts ([How OCR differs from vision](https://research.strata2signal.com/how-ocr-differs-from-vision/#table-small-print)). Nor is zero best for prose: decoding that always maximises likelihood, Holtzman's paper found, gives "output text that is bland, incoherent, or gets stuck in repetitive loops". ## What three engines ship by default {#what-three-engines-ship-by-default} Read on 2026-11-13 (UTC) from [llama.cpp's server manual](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/tools/server/README.md), [Ollama's Modelfile reference](https://github.com/ollama/ollama/blob/5632619189ab493c70bd3f9435214b8788cc1c80/docs/modelfile.mdx) and vLLM's parameter list: | setting | llama.cpp server | Ollama | vLLM | |---|---|---|---| | temperature | 0.80 | 0.8 | 1.0 | | top-k | 40 | 40 | 0 (off) | | top-p | 0.95 | 0.9 | 1.0 (off) | | min-p | 0.05 | 0.0 | 0.0 (off) | | repetition penalty | 1.00 (off) | 1.0 (off) | 1.0 | "Off" is where the project's own text says the value disables the step or keeps every token; a min-p of 0.0 and a penalty of 1.0 change nothing either. Two warnings. **It is only the bottom layer.** vLLM's [server](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/docs/serving/online_serving/openai_compatible_server.md) "applies `generation_config.json` from the Hugging Face model repository if it exists", so its defaults "can be overridden by those recommended by the model creator". A model file can carry its own values too: one we took apart holds top-k 20, top-p 0.95 and temperature 1.0 in its metadata ([Inside a GGUF file](https://research.strata2signal.com/inside-a-gguf-file/#the-keys-that-matter)), and llama.cpp's [loader](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/common/common.cpp) puts such values in place of its own defaults, sampler order included, unless whoever started it set them. And any request can send its own. **The order differs.** llama.cpp's manual gives its default chain as `penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature`: cuts first, temperature last. vLLM's [sampler](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/vllm/v1/worker/gpu/sample/sampler.py) applies temperature first, then min-p, then top-k and top-p. Take our table with only temperature 2 and top-p 0.9 switched on. Temperature first: the cut sees the flat column, keeps four tokens, and "soup" comes up about one time in ten. Cuts first: the cut sees the T = 1 column, keeps three, and "soup" never appears. The same two numbers, two different texts (our arithmetic). So before comparing two answers, ask three things: which settings, in which order, set by whom. The model sets the odds; the sampler rolls the dice. ## What this cannot say {#what-this-cannot-say} - **Which settings are best.** Nothing was tested here. The min-p paper reports gains over top-p; [a 2025 re-analysis](https://arxiv.org/abs/2506.13681) concluded its evidence "fails to support claims that min-p improves quality, diversity, or a trade-off between quality and diversity". We have measured neither. - **What your engine does today.** The defaults are one day's reading of each project's main branch. That day llama.cpp's manual gave the repetition penalty as "1.00, 1.0 = disabled" in its options table and "Default: `1.1`" in its endpoint list; the source agreed with the table. Ollama's [source](https://github.com/ollama/ollama/blob/5632619189ab493c70bd3f9435214b8788cc1c80/api/types.go) agreed with its Modelfile reference; what a running Ollama applies, and in what order, we did not check. Ask the running engine, and send the settings you mean. - **Anything about other samplers.** Beam search, Mirostat, typical sampling, DRY, XTC, the presence and frequency penalties, and whatever a hosted service does are not covered. - **That real scores are this tidy.** Five tokens with round scores stand in for hundreds of thousands with ragged ones. ## Who ran this, and thanks {#who-ran-this-and-thanks} Thanks to the papers' authors and to the llama.cpp, Ollama and vLLM projects, whose documentation and source this page quotes; a small human team asked for it, and a fleet of AI agents read the sources, did the arithmetic and drafted it. ## Sources {#sources} Every source below was read on 2026-11-13 (UTC). The other dates are arXiv's version dates and GitHub's commit and merge dates, all UTC. Papers are quoted in short phrases with attribution. - **Our own pages:** [The free speed wasn't free](https://research.strata2signal.com/the-free-speed-wasnt-free/), [It’s Not a Metaphor](https://research.strata2signal.com/compression-is-the-objective/), [How OCR differs from vision](https://research.strata2signal.com/how-ocr-differs-from-vision/) and [Inside a GGUF file](https://research.strata2signal.com/inside-a-gguf-file/). - **Ackley, Hinton and Sejnowski**, "A Learning Algorithm for Boltzmann Machines", *Cognitive Science* 9(1), 1985, pp. 147–169: https://doi.org/10.1207/s15516709cog0901_7 (citation checked against Crossref's record; quoted from the scanned copy on Geoffrey Hinton's University of Toronto page). - **Hinton, Vinyals and Dean**, "Distilling the Knowledge in a Neural Network", arXiv 1503.02531 (2015-03-09), section 2: https://arxiv.org/abs/1503.02531 - **Fan, Lewis and Dauphin**, "Hierarchical Neural Story Generation", arXiv 1805.04833 (2018-05-13), section 5.4: https://arxiv.org/abs/1805.04833 - **Holtzman, Buys, Du, Forbes and Choi**, "The Curious Case of Neural Text Degeneration", arXiv 1904.09751 (v1 2019-04-22, v2 2020-02-14; ICLR 2020), the abstract and sections 1 and 3.1: https://arxiv.org/abs/1904.09751 - **Keskar, McCann, Varshney, Xiong and Socher**, "CTRL: A Conditional Transformer Language Model for Controllable Generation", arXiv 1909.05858 (v1 2019-09-11, v2 2019-09-20), section 4.1: https://arxiv.org/abs/1909.05858 - **Nguyen, Baker, Neo, Roush, Kirsch and Shwartz-Ziv**, "Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs", arXiv 2407.01082 (v1 2024-07-01, v8 2025-11-20; ICLR 2025), the contribution statement: https://arxiv.org/abs/2407.01082 - **Schaeffer, Kazdan and Denisov-Blanch**, "Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models", arXiv 2506.13681 (v1 2025-06-16, v2 2025-06-19), the abstract: https://arxiv.org/abs/2506.13681 - **llama.cpp** (MIT) at commit `5676dfbe` (2026-11-12): [`tools/server/README.md`](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/tools/server/README.md), [`common/common.h`](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/common/common.h), [`common/common.cpp`](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/common/common.cpp) and [`src/llama-sampler.cpp`](https://github.com/ggml-org/llama.cpp/blob/5676dfbe01e7ebe0387913680eee0ed39bf4a363/src/llama-sampler.cpp); pull request 3841, "Min P sampler implementation \[alternative to Top P/Top K\]", opened 2023-10-28, merged 2023-10-31: https://github.com/ggml-org/llama.cpp/pull/3841 - **Ollama** (MIT) at commit `56326191` (2026-11-12): [`docs/modelfile.mdx`](https://github.com/ollama/ollama/blob/5632619189ab493c70bd3f9435214b8788cc1c80/docs/modelfile.mdx) and [`api/types.go`](https://github.com/ollama/ollama/blob/5632619189ab493c70bd3f9435214b8788cc1c80/api/types.go); the commit "api: stop applying repeat_penalty 1.1 to models that don't set one", Jesse Gross, authored 2026-08-11 and committed 2026-08-12: https://github.com/ollama/ollama/commit/6a261db7d87b13d76c5197cec636a0a3951afb36 - **vLLM** (Apache 2.0) at commit `c07d58dc` (2026-11-13): [`vllm/sampling_params.py`](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/vllm/sampling_params.py), [`docs/usage/reproducibility.md`](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/docs/usage/reproducibility.md), [`docs/usage/faq.md`](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/docs/usage/faq.md), [`docs/serving/online_serving/openai_compatible_server.md`](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/docs/serving/online_serving/openai_compatible_server.md) and [`vllm/v1/worker/gpu/sample/sampler.py`](https://github.com/vllm-project/vllm/blob/c07d58dc82d75f7d1a947b3264b460384ec3e18d/vllm/v1/worker/gpu/sample/sampler.py). *Nothing was measured for this page: a guide, not a bench. Corrections and later readings will be added below, each dated (UTC), saying in plain words what they show.*