{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "temperature-and-sampling-in-plain-words",
 "title": "Temperature, top-k and top-p, in plain words",
 "dek": "The network inside a chatbot never picks a word: it scores every token it knows, and ordinary code outside it rolls the dice; this guide works the five settings on that roll by hand, says what temperature zero does and does not promise, and reads three engines' defaults as documented on one day.",
 "published": "2026-11-13",
 "series": [
  "guide"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/temperature-and-sampling-in-plain-words/",
  "md_url": "https://research.strata2signal.com/temperature-and-sampling-in-plain-words/index.md",
  "md_sha": "ec17cf614dc06466942ff7f311b404a757a6d64569c4fe620586dc45c1c8128d",
  "html_sha": "d5f3a1543d3f248ec7d692d00d9ca33844aea0396967794c4f48f991341aa553",
  "anchors_sha": "8c6a597f37028bc38cc2fdbc994429624a40e6a91d0da2fd65b603706dc697aa"
 },
 "short": {
  "paragraph": "A language model's last output is a list of scores, one for every token in its vocabulary: 262,144 of them on one model we run. A sampler turns the scores into chances and draws one token. Temperature divides every score before the draw: below 1 the favourite pulls ahead, above 1 the field levels out, and at zero the sampler just takes the top score. Top-k, top-p and min-p cut off the long tail of unlikely tokens in three different ways, and a repetition penalty marks down tokens the text has already used. Three engines' documented defaults disagree: temperature 0.8 in two and 1.0 in a third, as read on 2026-11-13 (UTC). Two of them apply the steps in different orders, and a model's maker can override the defaults. Temperature zero removes the dice. It does not promise the same answer every time, and the likeliest token is not a checked fact.",
  "from_dek": false,
  "counts": {
   "words": 2239,
   "minutes": 10,
   "tables": 2,
   "kit": false
  },
  "bullets": []
 },
 "sections": [
  {
   "id": "the-model-does-not-pick-the-word",
   "heading": "The model does not pick the word",
   "level": 2,
   "span": [
    2246,
    3305
   ],
   "chunks": [
    [
     2246,
     3305
    ]
   ],
   "chars": 1059,
   "digest": "a language model writes one token at a time by producing a list of raw scores called logits. a separate sampler then converts these scores into chances using softmax to pick a token.",
   "digest_skipped": null
  },
  {
   "id": "temperature-worked-by-hand",
   "heading": "Temperature, worked by hand",
   "level": 2,
   "span": [
    3305,
    5191
   ],
   "chunks": [
    [
     3305,
     5191
    ]
   ],
   "chars": 1886,
   "digest": "temperature is a number used by the sampler to divide scores before the softmax step. at T = 1, one point of score makes a token about 2.7 times as likely, while higher temperature values make long shots more frequent.",
   "digest_skipped": null
  },
  {
   "id": "three-ways-to-cut-the-tail",
   "heading": "Three ways to cut the tail",
   "level": 2,
   "span": [
    5191,
    6962
   ],
   "chunks": [
    [
     5191,
     6962
    ]
   ],
   "chars": 1771,
   "digest": "three methods remove low-probability tokens before the roll: top-k keeps a fixed number of candidates, top-p keeps a set of tokens whose shares reach a specific value, and min-p keeps tokens that are close to the favourite.",
   "digest_skipped": null
  },
  {
   "id": "the-repetition-penalty",
   "heading": "The repetition penalty",
   "level": 2,
   "span": [
    6962,
    8534
   ],
   "chunks": [
    [
     6962,
     8534
    ]
   ],
   "chars": 1572,
   "digest": "to prevent looping, a repetition penalty can divide the scores of previously generated tokens. however, dividing negative scores can make them more likely, and applying a penalty can distort output like code or JSON.",
   "digest_skipped": null
  },
  {
   "id": "what-temperature-zero-does-and-does-not-promise",
   "heading": "What temperature zero does and does not promise",
   "level": 2,
   "span": [
    8534,
    11480
   ],
   "chunks": [
    [
     8534,
     11480
    ]
   ],
   "chars": 2946,
   "digest": "temperature zero acts as a switch for greedy sampling, but it does not guarantee the same answer every time, the same answer under different settings, or a true answer.",
   "digest_skipped": null
  },
  {
   "id": "what-three-engines-ship-by-default",
   "heading": "What three engines ship by default",
   "level": 2,
   "span": [
    11480,
    13992
   ],
   "chunks": [
    [
     11480,
     13992
    ]
   ],
   "chars": 2512,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "what-this-cannot-say",
   "heading": "What this cannot say",
   "level": 2,
   "span": [
    13992,
    15208
   ],
   "chunks": [
    [
     13992,
     15208
    ]
   ],
   "chars": 1216,
   "digest": "this text does not determine which settings are best, what a specific engine does today, or cover other samplers like beam search or Mirostat.",
   "digest_skipped": null
  },
  {
   "id": "who-ran-this-and-thanks",
   "heading": "Who ran this, and thanks",
   "level": 2,
   "span": [
    15208,
    15503
   ],
   "chunks": [
    [
     15208,
     15503
    ]
   ],
   "chars": 295,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "sources",
   "heading": "Sources",
   "level": 2,
   "span": [
    15503,
    19882
   ],
   "chunks": [
    [
     15503,
     19882
    ]
   ],
   "chars": 4379,
   "digest": "this section lists the papers and code repositories read on 2026-11-13 (UTC) to provide the information in the article.",
   "digest_skipped": null
  }
 ],
 "chips": [
  {
   "id": "c-23cee075",
   "q": "how does a language model select its next token?",
   "a": "The network produces a list of raw scores called logits for every token in its vocabulary. A separate component called the sampler then converts these scores into probabilities using a process like softmax and picks a token based on those chances.",
   "cites": [
    "the-model-does-not-pick-the-word"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-26b32912",
   "q": "how does temperature affect token probability distributions?",
   "a": "Temperature divides every score by a value T before the softmax step. A higher T produces a softer distribution, making long shots more likely, while a lower T makes the favorite pull away by increasing the impact of score differences.",
   "cites": [
    "temperature-worked-by-hand"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-0d0f7fe0",
   "q": "how does a repetition penalty influence token scores?",
   "a": "A repetition penalty divides the score of a previously generated token to reduce its likelihood. However, dividing negative scores can incorrectly make them more likely, so some implementations multiply negative logits instead.",
   "cites": [
    "the-repetition-penalty"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-e4e1a2c3",
   "q": "what happens when temperature is set to zero?",
   "a": "Engines treat zero as a switch to skip the dice and simply take the top score, known as greedy sampling. This does not guarantee bit-for-bit identical results or factual accuracy, and it can lead to bland or repetitive prose.",
   "cites": [
    "what-temperature-zero-does-and-does-not-promise"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-601a37ba",
   "q": "what limitations exist regarding the best sampling settings?",
   "a": "The article does not determine which settings are best, as no testing was performed. It notes that while min-p is reported to show gains, some re-analysis suggests the evidence for its improvement in quality or diversity is lacking.",
   "cites": [
    "what-this-cannot-say"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "reading-is-fast",
   "why": "shares ground with § The model does not pick the word · § The number that argued back"
  },
  {
   "slug": "how-an-embedding-model-works",
   "why": "shares ground with § The model does not pick the word · § Task prefixes: telling the model what a text is for"
  },
  {
   "slug": "elo-ratings-explained",
   "why": "shares ground with § Temperature, worked by hand · § How a rating moves"
  }
 ],
 "thanks": "## Who ran this, and thanks {#who-ran-this-and-thanks}\n\nThanks to the papers' authors and to the llama.cpp, Ollama and vLLM projects, whose documentation and source this page quotes; a small human team asked for it, and a fleet of AI agents read the sources, did the arithmetic and drafted it.",
 "kit": null,
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 0,
  "digests_written": 8,
  "digests_kept": 7,
  "chips_written": 7,
  "chips_kept": 5,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 0,
   "figure": 0,
   "length": 0,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 0,
   "profanity": 0,
   "identifier": 0,
   "bare_figure": 2,
   "house_voice": 0,
   "editorial": 0,
   "hand-edit": 4
  },
  "new_noun_tokens_checked": [
   "0.8",
   "0.80",
   "0.9",
   "0.95",
   "1",
   "1.0",
   "144",
   "2.7",
   "2026-11-13",
   "262",
   "26b",
   "40",
   "JSON",
   "Mirostat",
   "T",
   "UTC",
   "gemma4"
  ],
  "new_noun_tokens_withheld": 0,
  "figure_definition": "v3",
  "figure_boundary": "guarded",
  "drops": [
   {
    "kind": "bullet",
    "reason": "bare_figure",
    "item": "on gemma4:26b, a model produces 262,144",
    "detail": "the cited section writes this figure as '262,144 scores' and the bullet prints '262,144' alone"
   },
   {
    "kind": "bullet",
    "reason": "bare_figure",
    "item": "at T = 1, one point of score makes a token about 2.7",
    "detail": "the cited section writes this figure as '2.7 times' and the bullet prints '2.7' alone"
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 44,
    "text_sha": "20e0bbe30c06",
    "figure": "40",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-11-13",
    "why": "The defaults are one day's reading of each project's main branch.",
    "detail": "removed from this pack by hand on 2026-11-13 (UTC), after the writer ran and after the gates had counted: The defaults are one day's reading of each project's main branch. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 181,
    "text_sha": "e0ffac1482ba",
    "section": "what-three-engines-ship-by-default",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-11-13",
    "why": "Three engines' documented defaults disagree: temperature 0.8 in two and 1.0 in a third, as read on 2026-11-13 (UTC).",
    "detail": "removed from this pack by hand on 2026-11-13 (UTC), after the writer ran and after the gates had counted: Three engines' documented defaults disagree: temperature 0.8 in two and 1.0 in a third, as read on 2026-11-13 (UTC). Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 274,
    "text_sha": "a19010664e4e",
    "chip_id": "c-21562773",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-11-13",
    "why": "The bar is a fraction of the top share.",
    "detail": "removed from this pack by hand on 2026-11-13 (UTC), after the writer ran and after the gates had counted: The bar is a fraction of the top share. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 240,
    "text_sha": "bfa57522c117",
    "chip_id": "c-f3e65405",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-11-13",
    "why": "The defaults are one day's reading of each project's main branch.",
    "detail": "removed from this pack by hand on 2026-11-13 (UTC), after the writer ran and after the gates had counted: The defaults are one day's reading of each project's main branch. Not a gate drop, and recorded here so the ledger adds up."
   }
  ]
 },
 "generated_utc": "2026-11-13T04:22:24Z"
}