{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "ollama-or-vllm-which-to-run",
 "title": "Ollama or vLLM: which to run, and how",
 "dek": "Ollama and vLLM both run an open language model on your own computer. This page says which of the two to run for the case you bring, the two or three settings that decide it, who can reach it once it is running and what it sends out of your machine by default, every line pointing at the measurement or the dated source it rests on.",
 "published": "2026-10-08",
 "series": [
  "guide"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/ollama-or-vllm-which-to-run/",
  "md_url": "https://research.strata2signal.com/ollama-or-vllm-which-to-run/index.md",
  "md_sha": "5a7ce20e1ef701f4c655fad8864979c25a5790398ee7147174634f7355c9798a",
  "html_sha": "7630c10c6314aeed5afebb20282ab0808fb8ff8364cd8f659d52150f752df57a",
  "anchors_sha": "5d190d24d500ce87faadcfc066783134d46a416757e0e42523df9439f9770b34"
 },
 "short": {
  "paragraph": "Ollama and vLLM are two programs that run an open language model on your own graphics card, and which one to run depends mostly on how many requests reach it at once. Every figure here is our bench page's, from one RTX 3090 (24 GB) with the same 4-bit Gemma 4 12B weights on both, where each reading sits beside the prediction we wrote down before measuring it; speeds are in tokens a second, a token being a word or a piece of one, and Ollama 0.34.4 is set to serve sixteen requests at once in every comparison here (at its default it serves one at a time). One request at a time, run either: once an answer had started, vLLM 0.30.0 wrote it at 82.693 tokens a second and Ollama at 85.007 tokens a second, inside the 10 per cent we had predicted, though Ollama's wait before the answer started was more than twice vLLM's, 441.866 ms against 212.999 ms. Two to four at once, run vLLM, or Ollama set to serve more at once if another case below keeps you on it: vLLM was ahead on every speed and every wait the bench page prints at two and at four, its total across both requests 1.2077 times Ollama's at two, though at four Ollama's repeated runs disagreed widely. Eight to sixteen at once, run vLLM: at sixteen requests released together it wrote 506.829 tokens a second across them and Ollama 142.327 tokens a second, 3.561 times as much, and Ollama's total stayed flat from four requests on. Whichever you run, the two did not always give the same answer to the same test question, so check the answers on your own prompts. On this page, each of these cases links the section of our bench page, Ollama and vLLM on one RTX 3090, that holds its figures.",
  "from_dek": false,
  "counts": {
   "words": 8568,
   "minutes": 39,
   "tables": 1,
   "kit": false
  },
  "bullets": []
 },
 "sections": [
  {
   "id": "who-this-page-is-for",
   "heading": "Who this page is for",
   "level": 2,
   "span": [
    3554,
    6904
   ],
   "chunks": [
    [
     3554,
     6904
    ]
   ],
   "chars": 3350,
   "digest": "this page is for someone with a graphics card who has met two names. it defines terms like token, slot, window, runner, and total speed, while explaining the bench's method involving predictions, arms, levels, and spreads.",
   "digest_skipped": null
  },
  {
   "id": "which-to-pick-when",
   "heading": "Which to pick when",
   "level": 2,
   "span": [
    6904,
    23816
   ],
   "chunks": [
    [
     6904,
     23816
    ]
   ],
   "chars": 16912,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "the-settings-that-matter",
   "heading": "The settings that matter",
   "level": 2,
   "span": [
    23816,
    37796
   ],
   "chunks": [
    [
     23816,
     37796
    ]
   ],
   "chars": 13980,
   "digest": "this section details adjustable defaults for both servers. it covers memory budgets, slot and window settings, model loading, and how to manage communication and reasoning features to ensure the model stays on the card or handles long chats.",
   "digest_skipped": null
  },
  {
   "id": "what-each-calls-home",
   "heading": "What each calls home",
   "level": 2,
   "span": [
    37796,
    41916
   ],
   "chunks": [
    [
     37796,
     41916
    ]
   ],
   "chars": 4120,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "who-can-reach-it",
   "heading": "Who can reach it, and what to put in front",
   "level": 2,
   "span": [
    41916,
    48082
   ],
   "chunks": [
    [
     41916,
     48082
    ]
   ],
   "chars": 6166,
   "digest": "this section describes how each server handles network access and authentication. it explains that one server checks no key and the other checks a part of its surface, recommending a reverse proxy for security.",
   "digest_skipped": null
  },
  {
   "id": "getting-it-running",
   "heading": "Getting it running, and keeping it running",
   "level": 2,
   "span": [
    48082,
    51618
   ],
   "chunks": [
    [
     48082,
     51618
    ]
   ],
   "chars": 3536,
   "digest": "this section explains how to install and run both servers. it notes that one is a single download while the other is a Python package for Linux, and advises reading release notes before upgrading.",
   "digest_skipped": null
  },
  {
   "id": "what-to-take-with-you",
   "heading": "What to take with you",
   "level": 2,
   "span": [
    51618,
    53683
   ],
   "chunks": [
    [
     51618,
     53683
    ]
   ],
   "chars": 2065,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "what-it-does-not-say",
   "heading": "What it does not say",
   "level": 2,
   "span": [
    53683,
    55695
   ],
   "chunks": [
    [
     53683,
     55695
    ]
   ],
   "chars": 2012,
   "digest": "this section lists the limitations of the bench, noting it used a 24 GB card, one model family, and one operating system. it clarifies that it did not measure other servers, different request shapes, or bit-identical weights.",
   "digest_skipped": null
  },
  {
   "id": "how-to-check-our-work-and-see-it-live",
   "heading": "How to check our work — and see it live",
   "level": 2,
   "span": [
    55695,
    57032
   ],
   "chunks": [
    [
     55695,
     57032
    ]
   ],
   "chars": 1337,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "the-rest-of-the-seminar",
   "heading": "The rest of the seminar",
   "level": 2,
   "span": [
    57032,
    57966
   ],
   "chunks": [
    [
     57032,
     57966
    ]
   ],
   "chars": 934,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "who-ran-this-and-thanks",
   "heading": "Who ran this, and thanks",
   "level": 2,
   "span": [
    57966,
    59127
   ],
   "chunks": [
    [
     57966,
     59127
    ]
   ],
   "chars": 1161,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "sources",
   "heading": "Sources",
   "level": 2,
   "span": [
    59127,
    63476
   ],
   "chunks": [
    [
     59127,
     63476
    ]
   ],
   "chars": 4349,
   "digest": null,
   "digest_skipped": null
  }
 ],
 "chips": [
  {
   "id": "c-1adeb12a",
   "q": "how do ollama settings affect memory usage?",
   "a": "Each slot reserves a whole window, so the runner is started with window × slots. For example, sixteen slots of 5,120 tokens requires the runner to hold 81,920 tokens.",
   "cites": [
    "the-settings-that-matter"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-1e6d7eba",
   "q": "what limitations exist regarding the model size?",
   "a": "The article does not cover models larger than the card or the use of two or more cards.",
   "cites": [
    "what-it-does-not-say"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "ollama-and-vllm-on-one-rtx-3090",
   "why": "shares ground with § The settings that matter · § What each holds, and how fast it starts"
  },
  {
   "slug": "six-cards-the-same-questions",
   "why": "shares ground with § The settings that matter · § What to take with you"
  },
  {
   "slug": "how-a-vision-model-sees",
   "why": "shares ground with § What it does not say · § The bench, in one paragraph"
  }
 ],
 "thanks": "## Who ran this, and thanks {#who-ran-this-and-thanks}\n\nThis page measures nothing of its own: every reading is the bench page's, taken on one EVGA GeForce RTX 3090 XC3 Ultra (24 GB) capped at 300 W, and every dated fact but the safety paragraph's is one of the two histories', read from the project's own release notes, documentation and source at the tags named there. **Google** made Gemma 4 and its QAT (quantization-aware training) weights and released them under the Apache 2.0 licence; **Ollama**'s team and contributors built one server (MIT) on **llama.cpp** (MIT), begun by **Georgi Gerganov** and kept by the ggml authors; the **vLLM** project (Apache-2.0), begun in UC Berkeley's **Sky Computing Lab** and hosted by the **PyTorch Foundation**, built the other, on **PyTorch** (BSD 3-clause). On this card both run over **CUDA** (NVIDIA, proprietary, the one closed piece in either stack). None of them owed us anything.\n\nA small human team asked for this page and for the bench it draws on, chose what each would and would not claim, and signed off on every figure; a fleet of AI agents ran the bench and wrote this page under that team's rulings.",
 "kit": null,
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 0,
  "digests_written": 9,
  "digests_kept": 5,
  "chips_written": 8,
  "chips_kept": 2,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 0,
   "figure": 2,
   "length": 0,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 0,
   "profanity": 0,
   "identifier": 0,
   "bare_figure": 1,
   "house_voice": 0,
   "editorial": 0,
   "hand-edit": 10
  },
  "new_noun_tokens_checked": [
   "1.2",
   "10",
   "120",
   "13.0",
   "212.999",
   "24",
   "3.561",
   "32",
   "441.866",
   "5",
   "768",
   "81",
   "920",
   "CUDA",
   "Docker",
   "GB",
   "ID",
   "Linux",
   "NVIDIA",
   "Ollama's",
   "Python",
   "R580",
   "VLLMAPIKEY"
  ],
  "new_noun_tokens_withheld": 0,
  "figure_definition": "v3",
  "figure_boundary": "guarded",
  "drops": [
   {
    "kind": "bullet",
    "reason": "bare_figure",
    "item": "the window for a 24 GB card is 32,768",
    "detail": "the cited section writes this figure as '32,768 tokens' and the bullet prints '32,768' alone"
   },
   {
    "kind": "bullet",
    "reason": "figure",
    "item": "the bench page's two serve lines are the receipts",
    "detail": "terminal token carries no figure"
   },
   {
    "kind": "chip",
    "reason": "figure",
    "item": "For two or more requests at once, vLLM is faster, with its total speed reaching 1.2 times Ollama's at two requests and 3.561 times Ollama's at sixteen requests.",
    "detail": "figures absent from cited spans: ['3.561']"
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 45,
    "text_sha": "bf2044b802a5",
    "figure": "212.999",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the line names no server: 212.999 ms is the first word on vLLM 0.30.0 at one request, which the cited sentence sets against 441.866 ms on Ollama 0.34.4 with sixteen slots; printed alone at the top of a page that compares the two servers, the figure has no owner, and the Ollama posture the comparison was read on is gone with it",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the line names no server: 212.999 ms is the first word on vLLM 0.30.0 at one request, which the cited sentence sets against 441.866 ms on Ollama 0.34.4 with sixteen slots; printed alone at the top of a page that compares the two servers, the figure has no owner, and the Ollama posture the comparison was read on is gone with it. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 257,
    "text_sha": "391736085e35",
    "section": "which-to-pick-when",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the digest says the section gives measured or sourced evidence for needs including coding agents, where the coding agent row reads not measured at its shape and the list under the table says nothing here measured it; that case has neither a measured nor a sourced reading",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the digest says the section gives measured or sourced evidence for needs including coding agents, where the coding agent row reads not measured at its shape and the list under the table says nothing here measured it; that case has neither a measured nor a sourced reading. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 202,
    "text_sha": "98dc2baa3e6b",
    "section": "what-each-calls-home",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the digest gives model lists as data the servers send out by default; the section says the Ollama server fetches two things, the catalogue of its cloud models and a list of recommended models, signs each request with the key the installation generated, and that nothing on that list is your prompt; the model lists come in, they are not sent",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the digest gives model lists as data the servers send out by default; the section says the Ollama server fetches two things, the catalogue of its cloud models and a list of recommended models, signs each request with the key the installation generated, and that nothing on that list is your prompt; the model lists come in, they are not sent. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 208,
    "text_sha": "91deb7e94c67",
    "section": "what-to-take-with-you",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the digest has the section pick one server for single requests and the other for many at once; the section says at one request pick on everything but writing speed, decode inside the ±10 per cent band and the first word sooner on vLLM, so either server; and the switches it says to set are the ones that stop the default calls out, which the digest calls security switches",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the digest has the section pick one server for single requests and the other for many at once; the section says at one request pick on everything but writing speed, decode inside the ±10 per cent band and the first word sooner on vLLM, so either server; and the switches it says to set are the ones that stop the default calls out, which the digest calls security switches. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 200,
    "text_sha": "d63dc1b45afb",
    "section": "sources",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the digest drops the condition the section puts on the source files: vLLM source and documentation were read at the tags the last bullet names, for the safety paragraph only; every reading on the page is from the bench page, every other dated fact is from one of the two histories, and the page measures nothing of its own",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the digest drops the condition the section puts on the source files: vLLM source and documentation were read at the tags the last bullet names, for the safety paragraph only; every reading on the page is from the bench page, every other dated fact is from one of the two histories, and the page measures nothing of its own. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 256,
    "text_sha": "bab828532064",
    "chip_id": "c-9a05d08c",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the answer gives decode as the speed of writing one answer and drops after that, the words that put decode after the first word; without them decode reads as including the wait, which the section keeps apart; a glossary chip on the bench page left its pack for the same loss on 2026-10-07",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the answer gives decode as the speed of writing one answer and drops after that, the words that put decode after the first word; without them decode reads as including the wait, which the section keeps apart; a glossary chip on the bench page left its pack for the same loss on 2026-10-07. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 212,
    "text_sha": "37456bed5386",
    "chip_id": "c-5b089ec6",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the answer gives 441.866 ms as the first word on Ollama at one request and drops the posture it was read on, sixteen slots; the section names the default of one slot as the Ollama posture for this case and prints its first word only behind a long shared prompt; it also drops the versions benched, vLLM 0.30.0 and Ollama 0.34.4",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the answer gives 441.866 ms as the first word on Ollama at one request and drops the posture it was read on, sixteen slots; the section names the default of one slot as the Ollama posture for this case and prints its first word only behind a long shared prompt; it also drops the versions benched, vLLM 0.30.0 and Ollama 0.34.4. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 252,
    "text_sha": "dd923cbeeafd",
    "chip_id": "c-11918be3",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the answer states a vLLM default, the usage report and what it carries, in the voice of this pack with no version; the section ties that list to the version benched, saying that at v0.30.0 the repeat every ten minutes carries nothing more, and the vLLM history it links reads the source at v0.30.0; it also drops that the architecture goes as a class name rather than the name of the model, and the section names no cloud as where the report goes",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the answer states a vLLM default, the usage report and what it carries, in the voice of this pack with no version; the section ties that list to the version benched, saying that at v0.30.0 the repeat every ten minutes carries nothing more, and the vLLM history it links reads the source at v0.30.0; it also drops that the architecture goes as a class name rather than the name of the model, and the section names no cloud as where the report goes. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 221,
    "text_sha": "3ee6252b8b6f",
    "chip_id": "c-9c97a76e",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the answer gives what the vLLM security guide says the key does not cover with no version; the section reads the guide at v0.30.0, and the bench called the routes the guide names on vLLM 0.30.0 only, so the words of the guide hold at that version and the line must carry it",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the answer gives what the vLLM security guide says the key does not cover with no version; the section reads the guide at v0.30.0, and the bench called the routes the guide names on vLLM 0.30.0 only, so the words of the guide hold at that version and the line must carry it. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 173,
    "text_sha": "919a616ad917",
    "chip_id": "c-d368a287",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-08",
    "why": "the answer makes an NVIDIA driver of the R580 branch or newer a requirement of vLLM itself, by pip or Docker, in the voice of this pack with no version; the section says the packages pip installs are built for CUDA 13.0 since v0.20.0, by its release notes, which ask for that driver, that its installation page at v0.30.0 still says CUDA 12.9, and says nothing of a driver for the Docker image",
    "detail": "removed from this pack by hand on 2026-10-08 (UTC), after the writer ran and after the gates had counted: the answer makes an NVIDIA driver of the R580 branch or newer a requirement of vLLM itself, by pip or Docker, in the voice of this pack with no version; the section says the packages pip installs are built for CUDA 13.0 since v0.20.0, by its release notes, which ask for that driver, that its installation page at v0.30.0 still says CUDA 12.9, and says nothing of a driver for the Docker image. Not a gate drop, and recorded here so the ledger adds up."
   }
  ]
 },
 "generated_utc": "2026-10-08T10:00:28Z"
}