# Does temperature zero give the same answer twice?

*Temperature zero switches off the dice in a model's sampler, and that sounds like a promise: the same prompt, sent again, gives the same answer. On 2026-11-13 (UTC) we tested it two ways, each under rules frozen first. At home, on one llama.cpp build and one graphics card, a new server repeated itself on 10 prompts of 10, and on 7 to 10 of them the answer changed once a prompt cache was reused or other requests shared the server. Hosted, over the 11 models we could read, no prompt came back the same three times: 0 of 100.*

*Published 2026-11-15 (UTC) · A small (human) team and a fleet of AI agents.*

**the short version:** At temperature zero a sampler takes the highest-scoring token every time, so the same scores give the same text. Whether the scores come out the same is another matter: on the one build and model we read at home, it turned on the server's state, not on the setting. On 2026-11-13 (UTC) we sent ten prompts of our own, at temperature 0 with seed 0, two ways. At home (llama.cpp v0.9.0, Ministral 3 8B Instruct 2512 at Q4_K_M, one NVIDIA GeForce RTX 3090 (24 GB) capped at 250 W, answers capped at 512 tokens), a new server's first answer came back token for token on 10 prompts of 10 from another new server, from a warm one with the prompt cache off, and from a four-slot one with nothing else running. With the cache on it came back on 3 of 10 after a different prompt and on 0 of 10 after the same prompt; with other requests arriving, on 0 of 10. Short answers were mostly spared: at 16 tokens, 9 or 10 of 10 still matched in every condition. Hosted (21 models on Ollama's cloud, thinking on, three passes), 11 models could be read, and none of their 100 read prompts gave the same answer three times; 73 of the 100 had parted within 50 characters. The two are not one experiment: home compares the tokens of one model, hosted compares characters written after thinking, across eleven. Each wrote its bets down first and each lost some. Neither says why an answer moved, which answer is better, or how often any of this happens: ten prompts give k of 10, never a rate.

3,479 words · about 16 minutes (at 220 words/min) · 7 tables · data kit: no

https://research.strata2signal.com/temperature-zero-same-answer-twice/

---

*If you're new here: [strata→signal](https://strata2signal.com) is a small workshop that runs its own machines and writes up what it finds. This page is the measured follow-up to our guide [Temperature, top-k and top-p, in plain words](https://research.strata2signal.com/temperature-and-sampling-in-plain-words/#what-temperature-zero-does-and-does-not-promise), which said temperature zero "does not promise the same answer every time" and measured nothing. Every count below was read on 2026-11-13 (UTC).*

## The question {#the-question}

A language model does not pick words. For each token (a word or a piece of one) it returns a score for every token it knows, and a sampler, ordinary code outside the network, turns the scores into a choice. Temperature zero is the simplest choice there is: no dice, take the top score. The same scores give the same token.

So the question is whether the scores come out the same. They are arithmetic done in batches, and the engine we read does not promise the same digits from batches of different sizes. llama.cpp's own [server manual](https://github.com/ggml-org/llama.cpp/blob/cd959bae99ff4bed22bef2604f45241c09bbb8c8/tools/server/README.md) says so of its prompt cache: "the logits are **not** guaranteed to be bit-for-bit identical for different batch sizes", so "enabling this option can cause nondeterministic results". The same entry gives the option's default: on. Where two candidate tokens are close, a small difference in their scores swaps them, and from that token on the model is continuing a different text.

## Two benches, and how they differ {#two-benches}

Both benches were pre-registered: the question, the counts and the bets went into a file that was hashed and frozen before the first scored call ([why we do that](https://research.strata2signal.com/write-the-bet-down-first/)). The home bench was frozen at 16:34:27Z on 2026-11-13 and ran from 16:35:03Z to 17:00:51Z. The hosted bench was frozen at 20:10:24Z the same day and ran from 20:11:02Z to 21:57:57Z. Each is a first reading, so neither has an earlier one to be compared with. Both sent the same ten prompts of our own (a story, an explanation, code, JSON, a table, dialogue, a passage copied and rewritten, verse, Spanish and a worked problem, each asked for at length) at temperature 0 with seed 0.

| What | At home | Hosted |
|---|---|---|
| Engine or service | One build: llama.cpp v0.9.0 (release b12110), the project's own CUDA 12.8 binary | [Ollama](https://ollama.com)'s cloud, through Ollama's own client, signed in |
| Models | 1: Ministral 3 8B Instruct 2512, Q4_K_M, its maker's own file | 21 sent, 11 read |
| Hardware of ours | One NVIDIA GeForce RTX 3090 (24 GB), capped at 250 W | None: the calls left a laptop |
| Thinking | None: the model has no thinking mode | On |
| Answer cap | 512 tokens | 4,096 tokens, thinking included |
| Repeats | 3 runs in each of 6 conditions | 3 passes |
| Compared as | Token ids and bytes | Characters of the answer |
| What we chose | The server, its cache and what ran before | The request, and nothing else |

The two are not one experiment. At home one model wrote answers of up to 512 tokens with no thinking before them: nine of the ten prompts ran to the cap on every run, and the verse ended by itself a little short of it on 14 of its 18. Each answer is compared, token by token, with a reference we made ourselves. Hosted, eleven models wrote answers after anything from 89 to 12,612 characters of thinking, the service returns text and never token ids, and there is no reference: we can only ask whether three answers agree with each other.

## At home: the same answer when the prompt is read fresh and alone {#at-home}

For each prompt the **reference** is the first answer of a newly started server. An answer **equals** it when its token ids and its bytes are both the same. Six conditions each answered every prompt three times: 180 counted answers.

| Condition | What differs from a new server's first answer | Equals that answer, of 10 prompts | Its three runs are one text, of 10 prompts |
|---|---|---:|---:|
| A new server for every call | Nothing | 10 | 10 |
| A warm server, the cache off | The server has already answered other prompts | 10 | 10 |
| Four slots, the other three empty | Room for three more requests, none arriving | 10 | 10 |
| The cache on, after a different prompt | The opening the two prompts share is reused | 3 | 4 |
| The cache on, after the same prompt | All of the prompt but its last token is reused | 0 | 10 |
| Four slots, other requests arriving | Three other requests are answered alongside | 0 | 2 |

**At home, the same answer on all ten prompts only when each is read fresh and alone**

*Prompts of 10 in each of six conditions, each prompt three times, on one llama.cpp build (v0.9.0), one model (Ministral 3 8B Instruct 2512, Q4\_K\_M) and one NVIDIA GeForce RTX 3090 (24 GB) capped at 250 W, at temperature 0 with answers capped at 512 tokens, 2026-11-13 (UTC).*

At home, the same answer on all ten prompts only when each is read fresh and alone. New server: Its three runs are one text 10 of 10; Equals a new server's first answer 10 of 10. A new server for every call. Warm, cache off: Its three runs are one text 10 of 10; Equals a new server's first answer 10 of 10. One server, the prompt read whole each time. Four slots, alone: Its three runs are one text 10 of 10; Equals a new server's first answer 10 of 10. The other three slots empty. After other prompt: Its three runs are one text 4 of 10; Equals a new server's first answer 3 of 10. Cache on; the shared opening reused. After same prompt: Its three runs are one text 10 of 10; Equals a new server's first answer 0 of 10. Cache on; all but the last prompt token reused. With neighbours: Its three runs are one text 2 of 10; Equals a new server's first answer 0 of 10. Requests arriving on the other three slots.

The home bench's own rows, counted by its kit and again by a second script. The table above this chart.

**Read fresh and alone, it repeats.** A new server, a warm server with the cache off and a four-slot server with nothing else in flight each gave the reference on 10 prompts of 10. Warmth changes nothing by itself, and neither do empty slots.

**A reused cache changes long answers.** The prompt cache keeps the work a server did reading its last prompt, so a prompt that begins the same way is not read again from the start. Our ten prompts share an opening of 22 tokens (24 for a few pairs), which holds the one system line all ten carry. After a one-token call of a different prompt, the counted call reused that opening and read only the rest. It equalled the reference on 3 prompts of 10, and its three runs, each after a different prompt, were one text on 4. After the same prompt, the server reused everything but the last prompt token. Of that condition's 30 counted calls, 10 followed a one-token call of the prompt and 20 followed the counted call before them. 0 of 10 equalled the reference, yet the three runs were one text on 10 of 10. That answer is different, not random.

**So do neighbours.** With three other requests being answered on the same server (24 began and ended inside each of our calls), 0 of 10 equalled the reference and the three runs were one text on 2 of 10: different, and mostly not the same way twice.

**They part where two tokens are close.** At the token where a run left the reference, the reference's two likeliest tokens were between 0.00004 and 0.034 apart in probability, in every one of the 67 runs that differed. Why the scores moved is not read: no count here says which batch shapes change them.

## Hosted: no prompt came back the same three times {#hosted}

The hosted bench sent each prompt three times to each of 21 models: the 19 chat models on Ollama's cloud list as read on 2026-11-13, and the two `gpt-oss` models that its price table lists and its client served. The request was the same bytes each time (we hashed it): the model tag, our system line and prompt, thinking on, temperature 0, seed 0 and a cap of 4,096 tokens. Thinking, the reasoning a model writes out before its answer, stayed on because on this route, in [a reading of ours from 2026-08-28](https://research.strata2signal.com/the-same-sixteen/#three-checks-bracket-the-run), switching it off put the reasoning into the answer instead of stopping it.

That made 792 calls, 630 first attempts and 162 retries, every one answered with HTTP 200. An answer is counted when the reply ended by itself, held an answer and held no stray thinking tag: 514 were. Of the rest, 276 were cut at the cap while still writing, 1 ended in an error and 1 carried a tag; a reply that was not counted was sent once more, and only once. A prompt is read for a model when all three passes have a counted answer, and a model is read on 8 or more of its 10 prompts. Eleven models were read, on 100 prompts between them.

**No prompt came back the same three times: 0 of 100.** The three answers still agreed at 50 characters on 27 of the 100, at 200 on 10, at 800 on 1 and at 2,000 on none.

| Model tag | Prompts read, of 10 | Still the same at 50 characters | At 200 | At 800 |
|---|---:|---:|---:|---:|
| `deepseek-v4-pro:cloud` | 9 | 0 | 0 | 0 |
| `deepseek-v4.1-flash:cloud` | 8 | 0 | 0 | 0 |
| `gemma4:31b-cloud` | 10 | 7 | 2 | 0 |
| `glm-5.2:cloud` | 10 | 2 | 0 | 0 |
| `gpt-oss:120b-cloud` | 10 | 2 | 1 | 0 |
| `gpt-oss:20b-cloud` | 9 | 2 | 0 | 0 |
| `kimi-k2.7-code:cloud` | 8 | 0 | 0 | 0 |
| `mimo-v2.6-flash:cloud` | 8 | 4 | 1 | 0 |
| `mimo-v2.6-pro:cloud` | 9 | 6 | 5 | 1 |
| `nemotron-3-super:cloud` | 9 | 1 | 0 | 0 |
| `nemotron-3-ultra:cloud` | 10 | 3 | 1 | 0 |

The rows are in the order of their tags; ten prompts rank no model.

**Hosted, three answers still the same at 50 characters**

*Each of the 11 read models, counted over the prompts it was read on (8 to 10 of its 10, as the table above gives them), on Ollama's cloud on 2026-11-13 (UTC), at temperature 0 with thinking on, three passes.*

Hosted, three answers still the same at 50 characters. dseek-v4-pro: Prompts still the same at 50 characters 0 the same. dseek-v4.1-flash: Prompts still the same at 50 characters 0 the same. gemma4:31b: Prompts still the same at 50 characters 7 the same. glm-5.2: Prompts still the same at 50 characters 2 the same. gpt-oss:120b: Prompts still the same at 50 characters 2 the same. gpt-oss:20b: Prompts still the same at 50 characters 2 the same. kimi-k2.7-code: Prompts still the same at 50 characters 0 the same. mimo-v2.6-flash: Prompts still the same at 50 characters 4 the same. mimo-v2.6-pro: Prompts still the same at 50 characters 6 the same. nemotron-3-super: Prompts still the same at 50 characters 1 the same. nemotron-3-ultra: Prompts still the same at 50 characters 3 the same.

The hosted bench's own rows, counted by its kit and again by a second script; the table above this chart. In the labels, dseek-v4.1-flash is deepseek-v4.1-flash:cloud and dseek-v4-pro is deepseek-v4-pro:cloud; the other labels drop only the cloud suffix.

**The thinking usually parted too, and not always.** On 90 of the 100 the three thinking texts differed as well. On the other 10 the thinking was the same bytes three times and the answers still differed: 5 on `mimo-v2.6-pro:cloud`, 3 on `mimo-v2.6-flash:cloud`, 1 on `glm-5.2:cloud` and 1 on `gpt-oss:120b-cloud`. The two partings are counted together; which caused which is not read.

**Ten models are NOT READ.** That is the registration's word for a model with fewer than 8 read prompts. Every prompt they lost was lost to the cap, the model still writing at 4,096 tokens, so the read set leans toward models that think less. Their partial counts are in no bet.

| Model tag | Prompts read, of 10 | Of those, three answers one text |
|---|---:|---:|
| `beam:cloud` | 7 | 0 |
| `glm-5.3-flash:cloud` | 3 | 0 |
| `glm-5.3:cloud` | 4 | 1 |
| `inkling-2-small:cloud` | 7 | 0 |
| `kimi-k3.1-pro:cloud` | 5 | 0 |
| `kimi-k3:cloud` | 6 | 0 |
| `minimax-m3.1-flash:cloud` | 5 | 0 |
| `minimax-m3:cloud` | 5 | 0 |
| `mistral-large-4:cloud` | 3 | 0 |
| `muse-spark-open:cloud` | 7 | 0 |

That one prompt on `glm-5.3:cloud` is the only one of the 152 read across all 21 models that gave one text three times.

**A trace of a cache.** Of the 792 replies, 734 carried the service's own count of cached prompt tokens, and on 226 of them, across 14 model tags, it was above zero. At home a cache was one of the two things that changed an answer. Here it is reported and decides nothing.

**Not blind.** Before the freeze a scout of 84 calls checked the route, and it did compare answers: of the 25 pairs it would have counted, one was the same bytes. The registration says so, and says the scout leaned against two of its four bets.

The run used $7.48 by our arithmetic (tokens times the price table as read on 2026-11-13); not the service's bill.

## How soon the answers part {#how-soon}

At home, the three conditions that changed an answer, and the prompts of 10 whose three runs still matched the reference through each length:

| Condition | 16 tokens | 64 | 128 | 256 | 512 |
|---|---:|---:|---:|---:|---:|
| The cache on, after a different prompt | 9 | 9 | 4 | 3 | 3 |
| The cache on, after the same prompt | 9 | 7 | 3 | 1 | 0 |
| Four slots, other requests arriving | 10 | 10 | 10 | 5 | 0 |

Hosted, the read prompts whose three answers still agreed with each other through each length:

| Read prompts | 50 characters | 200 | 800 | 2,000 | The whole answer |
|---|---:|---:|---:|---:|---:|
| Of 100, over 11 models | 27 | 10 | 1 | 0 | 0 |

**How soon the answers part**

*Two parts that share no axis: at home, prompts of 10 whose three runs still match a new server's first answer through that many tokens, in the three conditions that changed an answer; hosted, read prompts of 100 whose three answers still agree with each other through that many characters, counted after the thinking; both on 2026-11-13 (UTC) at temperature 0.*

How soon the answers part. Home: cache on, after a different prompt. 16 tokens: Prompts still the same 9 of 10. 64 tokens: Prompts still the same 9 of 10. 128 tokens: Prompts still the same 4 of 10. 256 tokens: Prompts still the same 3 of 10. 512 tokens: Prompts still the same 3 of 10. Home: cache on, after the same prompt. 16 tokens: Prompts still the same 9 of 10. 64 tokens: Prompts still the same 7 of 10. 128 tokens: Prompts still the same 3 of 10. 256 tokens: Prompts still the same 1 of 10. 512 tokens: Prompts still the same 0 of 10. Home: four slots, other requests arriving. 16 tokens: Prompts still the same 10 of 10. 64 tokens: Prompts still the same 10 of 10. 128 tokens: Prompts still the same 10 of 10. 256 tokens: Prompts still the same 5 of 10. 512 tokens: Prompts still the same 0 of 10. Hosted: 100 read prompts over 11 models. 50 characters: Prompts still the same 27 of 100. 200 characters: Prompts still the same 10 of 100. 800 characters: Prompts still the same 1 of 100. 2,000 characters: Prompts still the same 0 of 100. Whole answer: Prompts still the same 0 of 100.

Both benches' own rows, counted by their kits and again by a second script. The two tables above this chart.

At home a short answer was mostly safe: with neighbours on the server, all ten were still the same through 128 tokens. Hosted, 73 of the 100 had parted inside 50 characters, 21 of them at the first character. A token is a word or a piece of one, so 128 tokens is far more text than 50 characters; and the hosted count starts only after the thinking.

## The bets, and the ones that lost {#the-bets}

Both home controls held: a new server repeated itself, and so did each condition that holds one state. So the instrument repeats where it should, and a failed bet is a finding. The B bets are the home bench's, the C bets the hosted bench's.

| Bet | The claim, in short | Result | The count |
|---|---|---|---|
| B-2 (b) | A reused opening is the same whichever prompt wrote it | Failed | Three runs one text on 4 of 10 |
| B-3 | Warmth and spare slots change nothing by themselves | Held | 10 of 10 in each |
| B-4 (a) | With the cache on, the same prompt before changes long answers | Held | 10 of 10 differ within 512 tokens |
| B-4 (b) | And it spares short ones | Held | 1 of 10 differs within 16 tokens |
| B-4 (c) | A reused opening changes nothing | Failed | 3 of 10 equal the reference |
| B-5 (a) | Requests arriving alongside change the text | Held | 10 of 10 differ within 512 tokens |
| B-5 (b) | And not the same way twice | Held | Three runs not one text on 8 of 10 |
| C-1 | No model repeats itself on every prompt | Held | 0 of 11 read models did |
| C-2 | Few models repeat themselves even half the time | Held | 0 of 11 did |
| C-3 | Short answers are spared, as at home | Failed | 0 of 11 still agree at 50 characters on 8 or more prompts |
| C-4 | The answer parts because the thinking parted first | Held | Thinking parted on 90 of 100 |

B-4 is scored as one bet, and it failed on its third part. C-4 held at exactly its line, 9 in 10, and it counts two partings together, never one as the cause of the other.

The two home failures are one mistake made twice: we bet that a reused opening is harmless. The registration had called that "the half most likely to fail" and fixed its wording in advance: "the finding is that any reuse of an opening changes long answers on this build". Hosted, C-3 carried the home reading over as a bet and lost it: the most any model managed at 50 characters was 7 of 10.

## What neither bench can say {#what-neither-can-say}

- **Why.** At home the cause is not read. Hosted, which machine, batch, build or cache served a call cannot be seen from outside, nor whether the temperature and the seed we sent were applied.
- **Anything about weights.** The home bench read one model and showed that, for it, the server's state decides repetition. A hosted model that parted here may repeat exactly on one card at home.
- **Another day, route or build.** Home is one build, one model file, one card and ten prompts, with flash attention on and the server's second prompt cache, the one in host memory, off. Every counted call also asked for the two likeliest tokens' probabilities, which changes the last step of the engine's sampler chain; before the freeze, on the instrument check's own two prompts, a call without them gave the same tokens on both. Hosted is one window of under two hours on one service with thinking on; a hosted service changes without notice.
- **Which answer is better.** Nothing here scores an answer.
- **A rate.** Ten prompts give k of 10. No count here is a percentage of anything.

## What to do with this {#what-to-do-with-this}

If a test of yours needs the same answer twice (a regression check, a comparison against a saved answer, a bug to reproduce), these readings say it has to own the server's state and not only the temperature. On the build we read, launched as below with its host-memory cache also off, that meant the prompt cache off in the request (`cache_prompt` false) and nothing else in flight; a warm server was as good as a new one. Short answers leave some slack at home: at 16 tokens, 9 or 10 of 10 still matched in every condition. Through a hosted service you own none of that, so treat one answer as one draw: repeat the call, see how far repeats of one request sit from each other, and read a difference between two prompts or two models only against that noise floor. Ten prompts make this a sketch, not a guarantee.

## How to check our work {#how-to-check-our-work}

The home bench is a recipe. Its server's launch line, less the model file, the host and the port (the four-slot conditions change only `-c 32768 -np 4`):

```
llama-server -ngl all --fit off -fa on -b 2048 -ub 512 \
  --no-kv-unified -c 8192 -np 1 --cache-ram 0 --cache-reuse 0 \
  --no-context-shift --spec-type none --no-webui -lv 4
```

Every counted call went to `/completion` with the prompt as token ids, `cache_prompt` as its condition says, and these 26 fields:

```
{"n_predict": 512, "temperature": 0, "seed": 0, "top_k": 40,
 "top_p": 0.95, "min_p": 0.05, "typical_p": 1.0, "top_n_sigma": -1,
 "repeat_penalty": 1.0, "repeat_last_n": 64, "presence_penalty": 0,
 "frequency_penalty": 0, "dry_multiplier": 0, "xtc_probability": 0,
 "xtc_threshold": 0.1, "mirostat": 0, "dynatemp_range": 0,
 "min_keep": 0, "ignore_eos": false, "logit_bias": [],
 "samplers": ["penalties", "dry", "top_n_sigma", "top_k", "typ_p",
              "top_p", "min_p", "xtc", "temperature"],
 "id_slot": 0, "return_tokens": true, "n_probs": 2,
 "post_sampling_probs": false, "stream": false}
```

Two of them matter most: for `repeat_penalty` and for `samplers` the manual prints a second default as a request field (1.1, and a list without `penalties`).

## Who ran this, and thanks {#who-ran-this-and-thanks}

Thanks to the [llama.cpp](https://github.com/ggml-org/llama.cpp) project (MIT), whose release, manual and source the home bench stood on; to Mistral AI for Ministral 3 8B Instruct 2512 and its own GGUF file (Apache 2.0); to NVIDIA for the CUDA runtime that binary uses (proprietary); to Ollama for its client (MIT) and its cloud; and to the makers of the 21 models it served, named above by tag. Both harnesses are plain Python on its standard library. A small human team asked for these benches and signed the numbers; AI agents wrote the registrations, built and ran the harnesses, recounted the rows and drafted this page.

## Sources {#sources}

- **llama.cpp** v0.9.0, tag `v0.9.0` at commit `cd959bae`, published 2026-11-04 (UTC); its binaries are release `b12110`'s: https://github.com/ggml-org/llama.cpp/releases/tag/v0.9.0
- **llama.cpp's server manual** at that commit, its `cache_prompt`, `repeat_penalty` and `samplers` entries: https://github.com/ggml-org/llama.cpp/blob/cd959bae99ff4bed22bef2604f45241c09bbb8c8/tools/server/README.md
- **Mistral AI**, `mistralai/Ministral-3-8B-Instruct-2512-GGUF` at revision `0102285a`, the file `Ministral-3-8B-Instruct-2512-Q4_K_M.gguf`: https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512-GGUF
- **Ollama**, its client and its cloud: https://ollama.com
- **Ollama's cloud list and price table**, read at 2026-11-13T23:23Z, after the run: https://ollama.com/search?c=cloud and https://ollama.com/pricing
- **Our own pages:** [Temperature, top-k and top-p, in plain words](https://research.strata2signal.com/temperature-and-sampling-in-plain-words/), [Why we write the bet down first](https://research.strata2signal.com/write-the-bet-down-first/) and [The Same Sixteen](https://research.strata2signal.com/the-same-sixteen/).

*Corrections and later measurements will be added below, each dated (UTC), saying in plain words what it counts.*

<!-- derived 2026-11-16 (UTC) by tools/derive_md.py from the pour source.
     source html sha256: f374d61ff591afb80f9079d476b19efcf443e9a17ae486c4fb310e319b551447
     derivation sha256:  293a7726f7e7529aec72646262a0428fa22ced4160c37ae000cd07ef7df90931
     the {#id} on each heading is the anchor that heading carries on the page. -->
