What 150 Watts Buys

The dial, and who notices

Somewhere on a walking tour, someone asks the guide a question and the first word of the answer is back in about a fiftieth of a second, the rest streaming behind it. Somewhere else, a message written inside a game world goes to a moderation seat first, which decides whether it is safe to pass on. Both answers come off the same graphics card, in the same room, drawing the same electricity. This page is about turning that card's power dial down by a quarter — from 600 watts to 450 — and finding out which of the two notices.

One of them didn't. The other paid two percent. The interesting part is why the bill landed where it did, and what the card's own gauges said while it was happening — because one of those gauges lied, and we counted how.

Five words this page leans on

• Power limit (cap) — a ceiling you set on how many watts the graphics card may draw. The card obeys by lowering its clock speed until the draw fits. It never refuses work.
• SM clock — the speed the card's compute cores run at, in MHz. The lever a power cap actually pulls.
• Dense vs MoE — a dense model uses all its parameters for every token; a mixture-of-experts (MoE) wakes only a fraction. Dense models work the card harder — which is how our two lanes landed on opposite sides of the cap. What the cap actually reads is draw, not sparsity: these two arms also differ in engine, quantisation and batching, and the image lane further down shows the same law inside one engine.
• Busy window — we average only the power samples where the card reported ≥50% utilisation, so idle gaps between requests don't flatter the number. Every percentage on this page that counts samples counts them inside this window, and says so.
• tok/s per watt — one stream's tokens per second over the whole card's busy-window watts: the efficiency number.

The moment this page is about

Our 96 GB workstation card defaults to 600 W. Our rack's power budget — a UPS that has to carry the whole pod through an outage — wants it at 450 W. The question is what that costs. Not in folklore ("undervolting is free!") and not in a synthetic loop — on the two models our products actually serve, with the same frozen prompt as the serving bench these two models were measured on (2026-08-25; its own write-up is still in the workshop).

Measured on our 96 GB workstation card at driver generation 595, vLLM 0.27.1 and Ollama 0.32.15. The versions matter: the instrument finding further down is specific to this driver generation, and a reader on the next one should expect to re-check it.

What we ran, and the rule behind every number

Two workloads, chosen because they sit on opposite sides of the cap — and refusing to average them is the first honest act of this page:

• gemma4:26b (MoE) served by vLLM at FP8 — the candidate fast lane the serving bench measured; production still runs Ollama, so this is the kitchen we're evaluating, not the one currently cooking. Sparse: 413 W busy-window mean at 600 W (averaged over both sustained repeats). It brushes 450 W and never sits on it — 16 of 386 busy samples, 4.1%, counted below.
• mistral-small3.2:24b (dense, Q4) served by Ollama — the live moderation lane, and the hottest arm of the serving bench: 594.6 W mean and 87 °C across its own eight-minute leg (2026-08-25, 10:48–10:57Z), the highest either figure went in that whole two-hour run. It lives directly underneath the cap. Served one request at a time in our production configuration — the daemon's parallel-request setting is 1, its context window 32768: at concurrency 16, fifteen requests queue — so every per-stream figure below is one stream's decode rate, and the aggregate is the same ~94 tok/s, not sixteen times it. The efficiency number carries the same rule: one stream's tokens over the whole card's watts.

The design, told per arm because they differ: the MoE arm ran the serving bench's burst cells (3 waves, concurrency 1 and 16) at each cap, plus two-minute sustained legs interleaved 600/450/600/450 so drift can't masquerade as a cap effect — its same-cap repeat pair is our measured noise floor (the two repeats differed by 1.0% and 1.8% on the aggregate, and by 0.1% on per-stream decode). The dense arm ran the burst cells only, once per cap — no sustained leg, no repeat. Its −2.0% is therefore read against the MoE arm's floor: a borrowed denominator, and we say so rather than bury it. What makes the dense verdict credible anyway is the mechanism and the margin — both its concurrencies moved the same direction with large clock drops behind them, and 2.0% is twenty times the 0.1% per-stream floor. Every rung logs per-sample watts, temperature, SM clock, and the card's own throttle flags, so the mechanism can be checked rather than inferred — and every one of those logs is in the kit beside this page.

The answer, in one paragraph

On the dense arm — the one actually pinned to the ceiling — the 600 → 450 W cap costs 2.0% of throughput and buys back 138 W, 11 °C, and a 28% jump in efficiency. At concurrency 16: 94.6 → 92.7 tok/s per stream, busy-window draw 586.5 → 448.4 W, peak temperature 83 → 72 °C, efficiency 0.161 → 0.206 tok/s per watt. The 2.0% is a median of 48 requests at each cap (three waves of sixteen, one warm-up wave discarded; temperature 0, seed 20260825, 256 tokens out), and the two sets do not overlap — the slowest 600 W request, 94.31 tok/s, still beat the fastest 450 W one, 93.35. Latency moved too, and stayed small in absolute terms: time-to-first-token p50 20.6 → 22.2 ms and p95 20.8 → 23.8 ms — three requests at concurrency 1, on a client machine that was not idle, so we quote the milliseconds rather than the percentage. On the MoE arm the cap clips only the top of the power distribution — across the two uncapped (600 W) sustained legs, 16 of 386 busy-window samples (4.1%) reached 450 W, peaking at 461 W on the second repeat — and costs no measurable throughput: sustained decode 170.24 → 170.09 tok/s (−0.09%, averaged over both interleaved repeats, inside the noise floor). The worst case across both arms: about two percent — of throughput.

The mechanism, checked rather than inferred

A cap works by pulling the clock down, so the proof that a cap bit is a clock that moved — and the watts belong beside the clocks so the reader can check both halves:

* the vLLM row averages the two interleaved repeats at each cap.

Busy-mean power on the sparse arm repeats to about 10 W between identical runs, so the 412.8 → 419.0 W in its sustained row is inside its own spread — the cap did nothing there, and the number says so.

That lopsided middle row is the whole finding: the dense model gave up nearly a fifth of its clock and lost two percent of its speed. Q4 decoding is memory-bandwidth-bound — the cores spend most of each token waiting on memory, so slowing the cores costs almost nothing while watts (which follow the clock) fall off a cliff. The live receipt: on the dense arm's 450 W run, the 119 samples where the card's own power-cap flag read Active drew 449.1–452.3 W against a 450.0 W limit (118 of the 119; one sample at the leg's edge read 317 W) — board power limits are enforced against a rolling average, so individual samples read a few watts either side of the limit — at 2137–2400 MHz, 360–630 MHz below the 2764 MHz the same cell ran at 600 W. There is no reading of that in which the card is not power-limited. Which brings us to the instrument that was supposed to say exactly that.

The instrument that half-failed, counted rather than asserted

The card exposes a flag that says "I am power-capped right now," and a cumulative counter that is supposed to total the microseconds spent capped. Across our twelve rungs they told opposite stories, and the counts are the receipt. The flag worked: on the rung that was actually clamped it read Active in 119 of 275 in-window samples (43.3%), at exactly the clamped clocks (2137–2400 MHz). The same arm's single-stream rung, which only brushed the ceiling, read Active once in 24. Every one of the other ten rungs — including all six at 600 W — read zero. It discriminated correctly in both directions. The counter froze: through ~60 seconds of flagged clamping it did not advance one microsecond — the same value before and after every rung, twelve of twelve — and then resumed normal accumulation twenty minutes after the bench ended. The flag saw it; the counter that is supposed to total it did not. The practical rule we take away, and publish with the numbers in the kit: on this card and driver generation, a zero flag is evidence; a zero counter is not.

An earlier internal write-up of this bench called both halves blind — that was a windowing error against local-time log stamps, found in this page's own audit, and the bench records now carry the dated correction. The serving bench this page's prompt and comparison rows come from has not been published yet; when it lands, it will carry that correction from its first line rather than as an addendum, because we found this before it shipped.

What this page does not say

• Two points, not a curve. 600 and 450. The dense arm's trade (−19% clock → −2% speed) hints there's more room below 450 worth measuring; that's a separate bench, and a sibling page is planned for our 24 GB consumer cards at an 80% cap.
• The dense arm's result is one unrepeated pair of runs at one prompt shape. A long-context or prefill-heavy workload shifts the compute/bandwidth balance and could move the 2% either way.
• MoE is not inherently cap-proof. This one, at this prompt size and batch, sits below 450 W; a longer-context or prefill-heavy load, a larger batch, or a bigger paging pool could put it against the ceiling, and we have not measured that regime — our own products feed this model ~6,500-token packs, which is exactly the regime to test next.
• One apparent result was noise, and we name it rather than hide it: a single MoE burst pair came back −13.7% per-stream at 450 W — with identical 2812 MHz clocks at both caps and neither leg reaching 450 W, which is a scheduling artifact, not a cap effect. It is why the interleaved sustained probe exists, and the sustained result (−0.09%) is the verdict.
• No quality claim. A cap changes clocks, not arithmetic; we did not re-run quality exams and make no claim about output content.
• Client-side noise existed and is named in the kit's notes (an unrelated CPU bench loaded the measuring client during part of the window); it touches client-measured latency, not the card's clocks or watts, which is where every verdict lives.

What it means

On the one dense arm we measured — Q4, ~512-token prompts, one stream at a time — a 600 → 450 W cap traded two percent of throughput (and a couple of milliseconds of first-token latency) for 138 fewer watts and 11 °C off the peak, with efficiency up 28%. The reason to believe that generalises to workloads like it is the mechanism, not the sample size: memory-bandwidth-bound decoding barely uses the clock the cap takes away. If your workload is a sparse MoE that never reaches the ceiling, the cap is a no-op today — set it anyway; it costs nothing and bounds your worst case, but re-check when the workload's prompt shape changes. The rack budget that wanted this number closed with room to spare — priced at two percent of one lane.

Same dial, other lane — measured on the production card, the day this page shipped

Measured 2026-08-27, 16:19–18:14Z (UTC), on the same 96 GB card every number above came from — after this page was written. We liked the serving answer enough to ask the question it doesn't cover: this card also paints. Language models mostly wait on memory; an image model mostly computes. So before setting the dial for good, we walked it — 600 → 500 → 450 W — under the image lane, live.

The setup, honestly: live traffic was failed over to another box for the sitting, so nothing competed with the ladder — the two language models stayed resident on this card (~34 GB) and we probed them at every rung; each rung rendered the same 139 sealed cells — real game subjects, verbatim prompts, sealed seeds, five image models across three resolutions — drawn from a bench this shelf ran days earlier, selected by that bench's own selection code re-run. Render times are the render engine's own execution stamps; power, clocks and temperature were sampled twice a second throughout, every sample carrying the limit it was taken under (all 3,767 samples read their arm's cap — no rung is contaminated); the card's power-cap flag was read once inside each lane's own block, which matters below. One wrinkle we measured rather than assumed: the earlier sitting predates this card moving onto a sine-wave UPS, so the fresh 600 W rung is the only denominator below — the old-vs-new comparison publishes separately as its own small finding about what a changed room costs.

The result is not "images pay more." It is "the model that fills the card pays; the ones that don't, don't."

* the two SDXL rows read as noise around zero: they never approach either cap, and a −1% "speed-up" under a cap is measurement scatter, printed rather than rounded away.

The painter's bill grows with the picture: +12.0% at 512 px, +19.2% at 768, +20.4% at 1024 — the row above pools them.

The ordering of the slowdowns is the ordering of how hard each model drives the card — the mechanism section's prediction, running in production. The heavy painter draws 599 W uncapped; capped to 500 it sits pinned at 499.9 W with its clock dragged from 2392 to 1927 MHz, and at 450 to 1796 MHz. At 500 W the power-cap flag read Active under load on that lane; at 450 W a single instantaneous flag read is luck of the draw — it caught the cap on a lighter lane and missed it on the painter — so there the power trace, pegged at 450.0 W, is the binding evidence, and the arm notes in the kit say so. The lightest lanes never reach either cap and pay nothing real.

And the serving lane, probed at every rung through the same two models the top of this page measured — this time single warm calls against the production Ollama seats (the median of 6–7 per rung, 128 tokens out) rather than the batched vLLM candidate above, which is why gemma reads 207 tok/s here and 170 per stream under batching at the top of the page. gemma read 207.3 / 207.1 / 207.2 tok/s at 600 / 500 / 450 W — flat to a tenth of a percent — and mistral 95.0 / 94.6 / 93.5 at the same three; the arms ran 500, then 450, with the fresh 600 W control last. The watts go where the pixels are.

One more small finding, kept honest: the same 88 cells with sealed 600 W times from the earlier sitting (2026-08-23, same card, before the UPS) re-ran today at 600 W. The pegged painter reproduced its old times exactly (median ratio 1.003); the two lighter lanes with sealed history read ~6% slower today (klein 1.064, albedobase 1.059). Four things changed between sittings — the sine-wave UPS, the render server's host and version, and ~34 GB of language models now resident beside the renders — so this is an observation, not a controlled result; we publish it because hiding a 6% environment drift while quoting 4–19% cap effects would be exactly the kind of bookkeeping this page exists to refuse. (Two sealed rows were excluded from that comparison with cause: each was the first render of its old block, and its sealed time carries the model's cold load — 4.87 s recorded where every identical-shape sibling reads 0.45–0.64 s.)

Where the dial lives now, ruled the day of the measurement (2026-08-27): 500 W. The serving lanes barely notice — gemma is flat to a tenth of a percent at every rung, and mistral pays about 1.5% at 450 W, the same dense-lane bill the top of this page priced at 2.0%, measured again two days later on a different load; the heavy painter pays its 10% only in the minutes it is actually running; and the UPS budget that opened this page gets 100 W of standing headroom back. A cap is one line to set and one line to undo; the numbers above are what each line costs, and this page now describes the card it was measured on.

Addendum, 2026-08-28 (UTC), one day after the ruling above. The dial this page is about — the 96 GB card's power limit, set to 500 W on 2026-08-27 when this page was written — moved again on 2026-08-28, to 450 W, as part of an estate-wide decision to run every card at roughly 80% of its board power. The reasoning is this page's own dense-arm rung: at 450 W the heaviest serving lane measured −2.0% of throughput for −138 W and −11 °C, the sparse lane didn't notice, and the image lane's cost at 450 was measured above (+19% on the one painter that fills the card, nothing real on the rest). Less power, less heat, longer-lived hardware, at very little cost — the operator's words, and the numbers on this page are the receipt they rest on.

What to take with you

Six things, in the page's own words:

• A power cap is a clock cap wearing a different hat. The card never refuses work; it slows its cores until the draw fits. So the cost of a cap is exactly the cost of a slower clock to your workload — and for decoding a Q4 model, that is nearly nothing.
• Dense pays, sparse doesn't. The dense model sat on the ceiling and gave up a fifth of its clock for two percent of its speed; the mixture-of-experts model brushed 450 W in 4% of its busy samples and lost nothing we could measure. Which side of the cap your model lives on is the whole result.
• Two percent bought 138 watts, 11 °C and a 28% better tok/s per watt. On the dense lane at concurrency 16. Latency moved by a couple of milliseconds at the first token.
• The dial reads per-model on the image lane too. The painter that fills the card gave up 19% at 450 W; the ones that idle under the cap gave up nothing — the same law as dense-vs-MoE on serving: which side of the cap a workload lives on is the whole result.
• Trust the flag, not the counter. On this driver generation the card's "power-capped right now" flag discriminated correctly on all twelve rungs; the counter that should have totalled the capped time stayed frozen through a minute of flagged clamping. A zero flag is evidence when the flag is sampled continuously, as it was on those twelve rungs; read once, it is luck of the draw (the cap ladder shows it missing a pinned painter). A zero counter is not evidence either way.
• We caught our own earlier claim and say so. The first internal write-up called both gauges blind — a local-time windowing error. The correction is dated in the records, and the bench this page borrows its prompt from will ship with it already applied.

Corrections, 2026-08-28 (UTC) — before this page went further afield, a reader-lens pass over it and its kit found sixteen things to tighten, and they are fixed in place rather than footnoted: the serving-probe figures are now labelled with the rung they belong to (they had been printed in run order under a heading that read the other way); the cap-ladder text no longer claims the power-cap flag fired at 450 W for the painter (the kit's own arm notes say it did not — the power trace is the evidence there); the decode settings, request counts and the statistic behind the 2.0% are stated; the sparse lane's "never asked for 450 W" became the counted "brushes it in 4% of busy samples"; a p95 from three requests is quoted in milliseconds; the ladder table names its statistic and the painter's cost by resolution; one kit row that a redaction rule had broken is repaired and the kit's gate now parses every file; and the takeaways count six things, which is how many there are.

How to check our work

The kit is beside this page: the twelve rungs' per-request records, the per-sample power/clock/temperature/flag traces (power.csv, throttle.csv per rung), the interleaved repeat pairs, the noise-floor table the verdicts are read against, the half-failed instrument's raw streams, the driver and server versions, and a README that states the busy-window rule and what the raw files needed before anything could be counted: the harness stamped them on the bench machine's own clock, not UTC. The kit's copies are converted, and the README says exactly what moved — get that conversion wrong and every count on this page comes out the way ours first did. The frozen prompt is the serving bench's own p512 — the published ruling body for 7 Wonders, 2nd edition:

p512    90eedd0c53f9554ae3837674504fcb7090013d9a432a653352183c0f25a7ce5c
harness 96a8f38dd1bbb89ce24f89339d81019fd91435e06e7be21b8476f76ba9ce857b

The cap ladder rides in the same kit under data/ladder/ — its registration with all four dated amendments, the cell registry, every render row, the per-rung power traces and arm notes. If a number here does not reproduce from the kit, say so at the contact desk, where a person reads every message.

The rest of the seminar

This page is a bench, and it sits with the others: the same 96 GB card is the subject of The Typist and the Developer (what happens when a chat model and a painter share it), the consumer-card end of the same question is A Rig Your Friend Already Owns, and how the card arrived in the room at all is The Move.

The whole shelf holds the benches behind the claims we publish, failures included. If there is a piece of the machinery you want opened next, say so — the suggestion box is read.