Reading is fast, writing is slow

Four words this page leans on. A token is the unit a language model actually reads and writes: a word or a piece of one — "fishing" might be one token, "villagers" might be two. A prompt is everything you hand the model before it starts answering. A model's weights are the billions of numbers that make it up — about 17.7 gigabytes on disk for the model on this page, at the compression our products run. The card is the GPU, the chip that holds those weights and does the model's arithmetic.

The mystery, stated honestly

Every chatbot you have ever used does both of these things, seconds apart:

• Reads in bulk. Your five-paragraph question, hundreds of tokens, is absorbed almost at once. Ten pages costs seconds, not minutes — and is still far faster than writing ten pages back would be.
• Writes at typing speed. The answer arrives word by word, and no amount of paying for better hardware makes it arrive as one block.

Same model, same card, two speeds. On our machine, measured for this page, reading ran forty times faster than writing. Why?

Reading is a group photo

Here is the trick that makes reading fast. In the kind of model behind every chatbot you have used (the trade calls them causal models, because they only ever look backwards), the math for each token depends on the tokens before it, and on nothing after it. The word "robber" in your question is understood in light of the words leading up to it, never the words that follow.

That one fact has a glorious consequence. If the model already has your whole prompt in hand, every token's math can happen at the same time. Token fifty needs the forty-nine tokens before it — and you already typed them; they are sitting right there. It never has to wait for them to be processed, only to exist, and they already do. So nothing waits on anything: the card, a machine built to do thousands of small calculations simultaneously, takes the entire prompt in one frame. A group photo, everyone in the picture at once. One click for the whole crowd — and the cost of that click is one trip of the model's weights through the card for the entire prompt, however long it is. The click gets slower as the crowd grows, but it never becomes one click per person.

The trade calls this the prefill: the phase where the model reads everything you gave it, in parallel, before writing a single word back.

Writing is one at a time

Now the answer — and the trick stops working.

The model chooses its reply one token at a time, and each choice depends on everything so far, including the tokens of its own answer. It cannot pick word three until word two exists, because word three has to make sense after word two. There is no group photo to take: the future of the sentence genuinely does not exist yet.

So writing is a single-file line. Choose a token. Run the model again to choose the next. Again. If a group photo costs one click for everyone, writing is a portrait sitting per word, and each sitting is expensive: choosing even one token means hauling the weights across the card — nearly eighteen gigabytes of numbers pulled through its arithmetic for one word's worth of decision. Writing speed is mostly set by how fast the card can pour those weights through itself, over and over, once per token. The trade calls this phase the decode, and its shorthand for the whole asymmetry is worth carrying: reading is limited by how much math the card can do; writing, by how fast it can move memory. Two different bottlenecks, one machine.

That is the mystery, dispatched: reading is parallel because the text already exists; writing is serial because it doesn't yet. For one conversation on one card, doing ordinary word-at-a-time generation, a bigger GPU widens the photo and shortens each sitting, but the line stays single-file. (There is a lawful way to partly cheat that — and before this page ends, we will catch our own card doing it.)

The notes in the margin

One more piece completes the machine. Choosing each new token requires knowing what came before. So wouldn't word four hundred of an answer have to re-read the three hundred ninety-nine words before it, every time?

It would, except the model keeps notes. As it reads your prompt, it retains what it computed about every token — not a summary, its own exact workings, kept rather than redone. The trade calls this the KV cache; call it the notes. When writing begins, each new token doesn't redo the reading. It glances over the notes, adds its own line, and moves on.

Be precise here, because two different costs are in play. Every written token still pays the trip through the weights — the notes never save that. (One lawful exception turns up later on this page.) What the notes save is re-doing the reading: without them, each new word would re-process the entire conversation from scratch, and answers would slow catastrophically as they grew. With them, a long conversation slows only gently — each new token glances over a longer set of notes, so the pace drifts as context grows, though our own published measurement shows that drift can be swamped by other effects, a wrinkle this page returns to. And the notes are not free themselves: they claim real memory on the card beside the weights, growing with the conversation.

A measurement, from our own card

Time to make the machine show its work. We asked qwen3.8:27b — the same model that screens questions on our rules helper — to read a 528-token passage and write a 128-token reply, ten times over, on the same card that serves our products. The runtime (ollama, the program that actually runs the model — version 0.32.13 here) reports the two phases separately on every reply; medians and ranges are across the ten runs, and the raw run records are published beside this page.

Measured 2026-08-18T06:20Z · qwen3.8:27b at Q4_K_M (the compression named at the top of this page; it averages under 5 bits per parameter, some tensors kept wider) · temperature 0 · num_predict 128 · weights already resident on the card · n=10 · rates computed from the runtime's nanosecond durations. At temperature zero all ten runs produce the identical 128 tokens — the same work ten times over — which is why the writing range is so tight, and why the reading spread is the honest scatter.

Reading ran forty times faster than writing on this workload. The ratio belongs to the workload as much as the machine — longer prompts and shorter answers stretch it, the reverse shrinks it — but for any prompt longer than a handful of tokens the direction never flips. The wall-clock version is the plainest way to feel it: reading 528 tokens took 97 milliseconds; writing 128 tokens took 947 milliseconds. A quarter of the tokens, ten times the time.

For scale, exhibit fourteen's stopwatch leg measured this same model's decode across three prompt sizes: about 116 tokens/s at its registered primary (a 1,000-token prompt), 133.5 at 8,000, and 127 at 32,000 — with the odd ordering (8k faster than 1k) flagged there as unexplained. This page's 135.3 on a 528-token prompt sits above that 1k primary, which by the mechanism taught above is backwards twice over. Hold that thought — it is about to matter.

The number that argued back

Here is the part we didn't plan. While checking this page, we did the arithmetic its own mechanism demands. This model has 27.3 billion parameters — those are the weights from the top of the page — and it is dense, meaning every one of those numbers is used for every token. At 17.7 gigabytes, writing 135 tokens per second would mean moving about 2.4 terabytes of weights through the card every second. That is more than the memory of any card in this machine's class can physically move. Our own measurement was too fast for our own explanation.

The resolution was sitting in the model's served configuration: a setting telling it to guess four tokens ahead (draft_num_predict 4). This model ships with a draft head — a small, fast guesser wired to the big model — and our runtime enables it by default. The guesser proposes the next few tokens; the big model checks all the proposals in a single parallel pass. A tiny group photo. Accepted guesses arrive several-at-a-time for one trip through the weights; where a guess is wrong, the big model corrects it and the line resumes. At the settings we measured — temperature zero — the check is exact: every accepted token is precisely the one the big model would have chosen alone. It is reading's speed, borrowed to hurry writing, running quietly on our card the whole time. And the arithmetic closes, because the runtime's own logs say exactly how well the guessing goes. Measured while checking this page (2026-08-18, greedy decoding, reproduced bit-identically across three separate runner processes): the big model accepted 49% of the guesser's proposals — an average of 2.95 tokens riding each trip through the weights, 128 tokens written in 43 trips instead of 128. The four guess slots earn their keep unevenly — the first lands 86% of the time, then 56%, then 37%, then just 16% — which is why four is the shipped depth and not eight. Roughly three tokens per haul brings the required traffic back inside what such a card can move.

So the honest statement of our own measurement: 135 tokens per second is the single-file line already partly cheated, and the impossible division is the receipt that the cheating works. Without the guesser, writing would be slower still — the asymmetry this page teaches is, if anything, understated. The clean with-and-without comparison — the same tiers with the guesser switched off — turns out to require restarting the model's runner per configuration, so it waits for a declared maintenance window rather than running beside live products; the acceptance figures above are read from the runtime's own logs, which cost the residents nothing. And the strange ordering in fourteen's speed rows — the 8k tier reading faster than the 1k primary, which that page flags as unexplained and leaves for a re-run — now has a leading suspect: a guesser whose luck varies with the text it is guessing. A page that teaches "the line is physics" owed you this footnote: physics rewards the clever, and our card had been clever the whole time without telling us.

What this explains, once you see it

Why answers stream word-by-word. The words genuinely arrive that way. Streaming is the single-file line, shown honestly; waiting silently until it finishes would be essentially the same total time with worse manners.

Why the first word takes longest. Before any writing starts, the whole prompt must be read and the notes written. Long prompt, longer pause, then the words flow. The trade measures that pause as time to first token. Exhibit fourteen prints this model's as 393 milliseconds and calls it a proxy: it was computed as total time minus writing time on calls that did not stream, so it sweeps setup and queueing in with the reading. Treat it as a ceiling on the reading bill, mostly made of reading. (And don't divide a prompt length by it to get a rate — reading rates climb steeply with prompt size, because a short prompt's bill is mostly fixed overhead. Our 5,415 tokens/s on a 528-token prompt is a floor for this card, not a ceiling.)

Why AI pricing has two numbers. Input tokens (reading) and output tokens (writing) are priced differently, chiefly because they cost differently: reading fills a card efficiently and batches well across users; writing occupies a line per user. When exhibit thirteen priced one read of a 30.7-megabyte documentation file at about $80 — call it eight million tokens by the source's own rule of thumb, four bytes to a token, at $10 per million input tokens — that was input cost alone, before a word of reply.

Why RuleSage's answers take the time they take. Exhibit fifteen closed on a live ruling that took 2.7 seconds: 88 milliseconds finding the right rulebook passages, and 2.6 seconds of the answering model at work — itself a reading bill first (the eight retrieved passages must be prefilled) and then the single-file writing of the answer. The finding is cheap. The writing is the wait.

Why this workshop keeps models warm. Two different warmths, worth separating. Keeping the weights on the card avoids a multi-second load from disk — fourteen's postscript tells of a production deploy that died against exactly that cold load, 17.7 gigabytes arriving slower than a timeout. The notes are warmth within a single conversation: built during the reading, consulted through the writing, gone when it ends. Both are the same lesson at different scales: never pay the expensive first read twice.

And why a "faster model" claim needs its phase named. A model can read faster and write slower than its sibling, or the reverse. A speed number without its phase is a number without its meaning — the same law this series applies to every figure it prints: carry the denominator, and carry the phase.

What this page does not say

It does not say the ratio is a constant — forty-to-one is this workload on this card; your prompt and your answer set your own. It does not say the line can never be hurried: the draft head above is one lawful trick, and batching — many users' lines interleaved so the card never idles — is another, which is part of why big providers' economics differ from a single card's. And it does not say the reading is free: long-enough prompts make even the group photo expensive, which is why Three librarians and a careful reader spends its whole length on handing the model less.

What to take with you

Five things, each with its number from this page:

• Reading is parallel because your text already exists; writing is serial because the answer doesn't yet. On our card, same model, same second: 5,415 tokens/s reading against 135 writing — forty to one.
• The first word costs the reading; every word after costs a full trip through the weights — nearly eighteen gigabytes hauled through the card, shared by however many tokens ride the trip. Streaming isn't theater; it's the honest shape of the work.
• The notes (the KV cache) save re-reading, never the trip. A long conversation slows only gently instead of collapsing — our own rows read 133.5 tokens/s at an 8,000-token prompt and 127 at 32,000 — and the notes claim real memory of their own beside the weights.
• AI pricing's two numbers are these two phases wearing dollar signs. Reading a 30.7-megabyte file — call it eight million tokens — for ~$80 was all prefill; your answer's cost is mostly the single-file line.
• Your card may already be cheating the single-file line, lawfully. Ours was: a draft head guessing four tokens ahead, 49% of guesses accepted, ~3 tokens riding each trip — found because our own measurement beat our own physics.

How to check our work — and see it live

Everything above is one command away on your own machine. If you run a local model under ollama:

curl -s http://localhost:11434/api/generate -d '{
  "model": "qwen3.8:27b", "stream": false,
  "prompt": "<a few hundred tokens of anything>",
  "options": {"num_predict": 128, "temperature": 0}}'

The reply carries prompt_eval_count and prompt_eval_duration (the reading) and eval_count and eval_duration (the writing), to the nanosecond. Run a long prompt with a short answer and watch the ratio stretch; run the reverse and watch it shrink. Check whether a draft head is quietly helping you (ollama show <model> — look for draft_num_predict; that is the parameter's name on the model, and the runtime hands it to its runner as --spec-draft-n-max, the form our receipt prints), and do the division against your card's memory bandwidth the way this page did. Our raw runs are published beside this page, with the guesser's report card next to them — check our arithmetic, or better, produce your own. The weights measured here are 17.7 GB — whatever card you use has to hold those plus the notes, so size accordingly.

The rest of the seminar

This piece is the machines' journey beginning. Its sibling, Three librarians and a careful reader, walks a question's journey from a typed ask to a cited page — the two pages explain each other's timing numbers, and the stopwatch rows this page leans on live in exhibit fourteen, the audit that seated this very model. The $80 read that opens the pricing story is exhibit thirteen's measurement. Ahead, no dates promised, tracked in a register we keep of owed work: what those Q4_K_M tags actually mean (a model shrunk like a compressed photograph — and our bench that found the shrinking cost nothing), what a token costs and why, and how a rulebook becomes a checklist that refuses to say what it cannot prove. The whole shelf holds every bench behind every claim, failures included. If one of those is the piece you want next, say so — the suggestion box is read.

Licence: CC BY 4.0, the whole page — name the source and link to it.