The bench — a new picture model on a game’s own subjects and style, judged blind by one person against the models it paints with
Qwen-Image-2.1-Turbo, judged blind in a game's own painted style
exhibit eighty-six The bench
Published 2026-10-18 (UTC)
A small (human) team and a fleet of AI agents.
Qwen-Image-2.1-Turbo is a text-to-image model released on 2026-10-09 by Qwen, the Alibaba Cloud team behind the Qwen family of AI models; it draws a picture in 8 steps, each one pass that cleans the picture up a little. We set it beside two models we already paint with, FLUX.1 [dev] and FLUX.2 [klein] 4B, on six real subjects from our game in its own oil-painted sketch style, and one person picked the best picture of each without knowing which model made it. The Turbo was picked for none: by the rules we wrote down first, not a win.
ask about this page → assistant.strata2signal.com · in beta, still being tested
hardware on this page: RTX PRO 6000 Blackwell (96 GB) → research.strata2signal.com/hardware/ · the roster is in beta, still being tested
the short version
Qwen-Image-2.1-Turbo painted six real subjects from our game RealKeep in the game's own sketch style, stylized oil-painted realism in the manner of the video game Dishonored, beside FLUX.1 [dev] at 32 steps and FLUX.2 [klein] 4B at 8. One person picked the best picture of each from unlabelled sheets. It is not a win. The Turbo, at either of its two sizes, was picked for 0 of the 6 subjects; FLUX.1 [dev], one of the models the game paints with today, for 4; klein 4B for 2. The Turbo did show every item on each subject's check list on 5 of the 6, as klein 4B did (FLUX.1 [dev]: 4), but a win needed both. One person's blind eye on six subjects in one style is not a ranking: it says only that the Turbo has not earned a bigger test. klein 4B was also the fastest (0.98 seconds a picture; the Turbo 1.41 at the same size, on a different clock); speed decided nothing. The Turbo's licence allows research and evaluation only, so it was never going into the game.
- The hottest of the eight logged readings for the card was 62 °C (see How it ran)
2,844 words, about 13 minutes to read.
The summary is this page’s own; the receipt lines were drafted by a model on this workshop’s network and every figure in them is in the article, checked before this page went out — what was dropped, and why, is in this page’s receipt file.
strata→signal is a small workshop that runs its own machines and writes up what it measures. The rules and method below were written before the first picture; the run's readings, the results, the pictures and the data-kit call were added after the blind picks were in. Every web page cited was read on 2026-10-14 (UTC).
What this page asks
Qwen-Image-2.1-Turbo turns a sentence into a picture. Its makers, Qwen, the team behind a family of AI models "developed by Alibaba Cloud" [14], call it "an accelerated checkpoint of Qwen-Image-2.1 for text-to-image generation and image editing with 8 denoising steps" [1]. Qwen-Image-2.1 came out on 2026-09-20 with 40 steps as its default [3]; a Turbo (or distilled) model is trained to copy a slower one's results in fewer steps, here a fifth as many.
We run a small role-playing game, RealKeep, that paints its own pictures on our own machines [13]. Its in-play sketches for its fishing-village world, Sorrowmoor Cove ("the cove" below), are painted in one house style that the game's files spell out word for word. So the question is narrow on purpose:
Does the Turbo paint the cove's style, on the cove's real subjects, at least as well as FLUX.1 [dev], judged blind by one person, without losing more subjects? FLUX.2 [klein] 4B ran beside them as a third column.
For anyone choosing a picture model for their own computer:
- Will it fit my card? Its three parts (a 7.12-billion-parameter picture generator, an 8.77-billion-parameter text encoder that reads your sentence, a 0.34-billion-parameter decoder) weigh 32.44 GB. Loaded whole, it took 31,692 MiB of memory on the 96 GB card we ran it on; by arithmetic, not measured, a 24 GB card can run it one part at a time.
- May I use what it makes? Its licence allows research and evaluation only (quoted below). Qwen's first Qwen-Image came under Apache 2.0 [4]; the 2.1 line does not.
The three models
- Qwen-Image-2.1-Turbo, from Qwen, at revision d65dbc9a7e8f6b5479e33dee6030eaab2a906509, with its own saved 8-step schedule. Its makers' presets are all near 2048 by 2048 pixels [1]. Licence: the Qwen Research License Agreement [2].
- FLUX.1 [dev], from Black Forest Labs, a 12-billion-parameter model [10], one of the models RealKeep paints with today. Licence: the FLUX.1 [dev] Non-Commercial License [10].
- FLUX.2 [klein] 4B, also from Black Forest Labs: a 4-billion-parameter model "available for commercial use" under Apache 2.0 [11], the model this workshop's print lab paints with.
The Turbo ran in Diffusers 0.41.0, the Python library its makers point to [7]; the FLUX models in ComfyUI v0.21.1, the copy installed beside this card, which predates ComfyUI's Qwen-Image-2.1 support [9]. When this bench was planned, on 2026-10-09, no ComfyUI file of the Turbo existed yet [8]. So: two engines, two clocks.
The style, and the six subjects
The style is the cove's sketch style, verbatim from the game's campaign file:
stylized oil-painted realism, loose quick brushwork, in the style of Dishonored game art, unsigned, no text, no letters, no signature, no watermark
Dishonored is "a 2012 action-adventure game developed by Arkane Studios and published by Bethesda Softworks" [12]; no picture from it was given to any model. Every prompt is the style clause, a comma, then one subject, as the game builds them; none was rewritten, and no model got a negative prompt.
The six subjects are the game's own words: two foes, two scenes, two portraits. The last column is the judge's check on each picture, fixed in advance.
| Subject | The subject clause, verbatim | What must show |
|---|---|---|
| Foe · the Reef-Runner | a hard-faced smuggler in dripping oilskins wading a flooded sea-cave, a gaff hook in one hand and a hooded lantern in the other | a gaff hook · a hooded lantern |
| Foe · the Salt-Father | a towering humanoid monster built of jagged translucent white salt crystal, looming full-figure, smaller dry husk-people half-merged into its glittering salt mass, no face | built of salt crystal · no face |
| Scene · the sea-fog breakwater | REGION VIGNETTE: A thick curtain of sea-fog swallowing the stone breakwater and the distant silhouette of forty huddled rooftops. | the stone breakwater · huddled rooftops in the distance |
| Scene · the stone chapel | REGION VIGNETTE: The weathered stone chapel with its cracked bell tower glowing under a dim lantern light. | a cracked bell tower |
| Portrait · Harbormaster Haldane | close-up head-and-shoulders portrait of a precise middle-aged harbormaster in a peaked cap, skeptical fair eyes, his pen poised mid-entry, the ledger counter a blur below, amber on grey | a peaked cap · a pen |
| Portrait · Wrenna Sault | close-up head-and-shoulders portrait of a quiet devout lampwright, wick-trimming scissors in hand, warm amber glow on her face, night fog-blue behind | wick-trimming scissors |
"No face" asks a model to leave something out, the hardest kind of instruction on the page.
What counts as a win, written down first
The rules were registered on 2026-10-11 at 03:13:29Z and amended three times, each dated, the last at 01:24:19Z on 2026-10-13; the run began at 07:38:07Z on 2026-10-15. No rule changed after a picture was made.
- G0 · Is the picture valid? A mechanical check on every picture: the registered model, steps, settings and size, the exact prompt. A failed picture is void and counts against its model.
- G2 · Does it show the subject? The judge ticks every "what must show" item a picture shows. A model holds a subject when every item is ticked. Passes if the better Turbo column holds at least as many subjects as FLUX.1 [dev].
- G3 · Which reads best? One pick per subject: "Which option best reads as the cove's style (stylized oil-painted realism, loose quick brushwork, Dishonored game art) while showing the subject named above?" Passes if the Turbo (either column) is picked for at least 4 of the 6.
- G4 · Speed is reported only, as a tie-breaker; G5, the grid pattern users report on Qwen-Image-2.1's faces [6], is checked after the reveal, for a next test only.
(G1, a pixel-art test, was dropped when the workshop retired pixel art.)
A WIN needs G0, G2 and G3, and would have bought only a bigger test (the full Qwen-Image-2.1 at 40 steps, painting at 2K, or the Turbo inside ComfyUI), never a place in the game. A NO-WIN closes this line of testing, with no table of "what it would have needed". INCOMPLETE (fewer than 4 subjects with a valid judged picture from FLUX.1 [dev], or from both Turbo columns) keeps a fault in our own harness from passing as a result. klein 4B decided none of it.
How it ran
One card, nothing else on it: the NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB), its enforced power limit read at the start as 420 W, as registered. The run took just under five minutes, 07:38:07Z to 07:43:05Z on 2026-10-15 (UTC).
| Column | Model, 16-bit (bf16) | Size | Steps | Engine |
|---|---|---|---|---|
| A | FLUX.1 [dev] | 768 × 768 | 32, guidance 3.5 | ComfyUI v0.21.1, the game's own high-quality FLUX recipe |
| B | FLUX.2 [klein] 4B | 768 × 768 | 8, cfg 1 | ComfyUI v0.21.1, the maker's example recipe at 8 steps (it ships 4) |
| C | Qwen-Image-2.1-Turbo, whole on the card | 768 × 768 | 8 on its saved schedule, cfg 1 | Diffusers 0.41.0 |
| D | the same | 1024 × 1024 | the same | the same |
768 × 768 is the game's default sketch size, 1024 × 1024 the size it pre-paints its world at; column D asked whether a size nearer the Turbo's own changes the answer. Seeds 101, 202 and 303 (a seed fixes the starting noise; across two programs it is only a label), one picture each per subject and column: 72 in all.
Two clocks. ComfyUI's seconds run between its own "started" and "finished" stamps: steps, decoding, writing the file, and on each subject's first seed, reading the prompt. The Turbo's run around the picture call, the card synchronised at both ends: steps and decoding only. The fairest line is the median over seeds 202 and 303, where neither reads the prompt.
Isolation. A, then B, then the card emptied (2 MiB in use), then C and D. The Turbo loaded whole in 12.75 seconds, every part in bf16, from a copy checked against Hugging Face's checksums. The network was off throughout. No registered stop fired: every column made 18 of 18, and the hottest of the card's eight logged readings was 62 °C (the stop was 83).
The blind pick. Each subject's seed-101 pictures went onto one sheet: four options in random order, unlabelled, all at 768 × 768, nothing naming a model, size or time. The key was locked away, its checksum printed on every sheet, so it could not change unseen. The sheets went to one person, the member of our small team who directs the game's art. Six lines came back on 2026-10-15 (UTC), a pick and the ticks under each option; we then asked the judge to enter the ticks under the options not picked once more, still blind. Only then was the key opened, the same day; its checksum matched the seal on all six sheets.
What the judge picked
Not a win. The Turbo was picked for none of the six subjects, at 768 × 768 or at 1024 × 1024. FLUX.1 [dev] was picked for 4, klein 4B for 2. G3 fails (the Turbo needed 4). G2 passes: both Turbo columns held 5 of 6 subjects, as klein 4B did, against FLUX.1 [dev]'s 4. A win needed both.
The bench was COMPLETE: all 72 pictures passed G0, none void, every column judged on all 6 subjects.
| Subject | What must show | Picked (option on its sheet) | Held it (every item ticked) |
|---|---|---|---|
| Foe · the Reef-Runner | a gaff hook · a hooded lantern | 4: A · FLUX.1 [dev] | all four columns |
| Foe · the Salt-Father | built of salt crystal · no face | 2: B · klein 4B | none: "no face" was ticked under no picture; D · Turbo 1024 had neither item |
| Scene · the sea-fog breakwater | the stone breakwater · huddled rooftops | 1: A · FLUX.1 [dev] | all four columns |
| Scene · the stone chapel | a cracked bell tower | 2: A · FLUX.1 [dev] | all four columns |
| Portrait · Harbormaster Haldane | a peaked cap · a pen | 4: A · FLUX.1 [dev] | all four columns |
| Portrait · Wrenna Sault | wick-trimming scissors | 2: B · klein 4B | B, C, D; the scissors not ticked under A · FLUX.1 [dev] |
| Of 6 | A 4 · B 2 · C 0 · D 0 · none 0 | A 4 · B 5 · C 5 · D 5 |
So the Turbo lost no more subjects than anyone and was still never the picture that read best as the cove's style. Painting it at 1024 × 1024 changed nothing: column D was picked as often as column C, never, and held the same count.
The pictures
These are klein 4B's two picks, as the judge saw them; klein 4B is Apache 2.0, and our licence ledger marks it "published". The other models' pictures stay off this page: FLUX.1 [dev]'s row there reads "held — shown only as research receipts (ruled 2026-09-04)", on the pages it names, and this is not one; the Turbo's licence is silent on who owns its pictures, and no ruling of ours covers them.
Foe · the Salt-Father, picked (option 2 of 4): FLUX.2 [klein] 4B, 768 × 768, 8 steps, seed 101, 1.11 s by ComfyUI's clock (a first seed, prompt reading included). Salt crystal, yes; but the giant has a face. In the lower-left corner sits a scrawl like a signature, though the style clause says "unsigned, no signature".
Portrait · Wrenna Sault, picked (option 2 of 4): FLUX.2 [klein] 4B, 768 × 768, 8 steps, seed 101, 1.11 s by ComfyUI's clock (a first seed, prompt reading included).
Seconds per picture, by each engine's own clock
| Column | Median seconds per picture (seeds 202, 303; 12 pictures) | Range of those 12 | Clock |
|---|---|---|---|
| A · FLUX.1 [dev], 768², 32 steps | 6.42 | 6.21–6.55 | ComfyUI's stamps, file write included |
| B · klein 4B, 768², 8 steps | 0.98 | 0.97–1.03 | ComfyUI's stamps, file write included |
| C · Turbo, 768², 8 steps | 1.41 | 1.397–1.413 | synchronised wall, no file write, no prompt reading |
| D · Turbo, 1024², 8 steps | 2.68 | 2.662–2.693 | synchronised wall, no file write, no prompt reading |
klein 4B was the fastest even though its clock counts writing the file and the Turbo's does not. Outside its clock, the Turbo read each subject's prompt once: 0.593 s for the first, 0.068–0.085 s for the other five. Its median step took 0.164 s at 768² and 0.310 s at 1024².
The grid pattern (G5) had not been read when this page was written; it would only inform a next test, and this result earns none.
What this result does and does not mean
- It means the Turbo, run this way, did not earn a bigger test, and this line of testing closes. Nothing here changes what the game paints with.
- One person's eye, blind, once: six subjects, one picture each, one style. The right judge for this game's art, and still one judge. klein 4B's two picks are reported as found; by the rules, klein decided nothing.
- Not other styles. A model that loses here may win at photographs, posters or text in pictures, as its makers' examples show [1].
- Not the Turbo at home. Its presets are near 2048 × 2048, and its makers say "other schedules have not been evaluated for this checkpoint" [1].
- Not the full Qwen-Image-2.1, its editing mode, a 24 GB card, a smaller file format, the Turbo inside ComfyUI, the game's live sketch settings, or a speed ranking finer than two clocks allow.
Why this page is an evaluation
The Turbo comes under the Qwen Research License Agreement, "Release Date: September 20, 2026" [2]. §1(i): "'Non-Commercial' shall mean for research or evaluation purposes only." §2(b): "You shall not use the Materials for any commercial purpose without obtaining a separate commercial license from us."
No clause says who owns a picture the model makes. A public discussion on the Qwen-Image-2.1 model page quotes a post it attributes to Qwen's developer account on X saying users "retain the rights to images and other content they generate using the model"; we have not read that post, and on 2026-10-14 the discussion was still open with no reply from Qwen [5]. This page gives no legal reading.
What we did fits a use the licence names: we evaluated it. We downloaded it once, ran it with the network off on our own card, and removed the copy after the run. Nothing it made went into the game or any other tool. FLUX.1 [dev] is non-commercial too and ran here for evaluation (our history of Black Forest Labs quotes its licence); klein 4B is Apache 2.0 [11].
How to check our work
No data kit ships: the run's files name our machines and private paths, and a kit holding FLUX.1 [dev] or Turbo pictures could not carry our kits' open licence. The prompts, settings, rules, results and timings above are what a repeat needs. Every web page below was read on 2026-10-14 (UTC).
- Qwen, Qwen-Image-2.1-Turbo model card, Hugging Face (read at fc13ba1986fcb905854dc8de2f52eee395f49b5b, which differs from the pinned revision only in an example picture size). https://huggingface.co/Qwen/Qwen-Image-2.1-Turbo
- Qwen Research License Agreement, Release Date: September 20, 2026 (7,831 bytes; sha256 8dc973f024ff95966bea25866efa443fd16776dcb1001e681e3d467ea572b28d). https://huggingface.co/Qwen/Qwen-Image-2.1-Turbo/blob/main/LICENSE
- QwenLM, Qwen-Image-2.1 README, GitHub. https://github.com/QwenLM/Qwen-Image-2.1
- Qwen, Qwen-Image model card, Hugging Face (apache-2.0). https://huggingface.co/Qwen/Qwen-Image
- "Commercial Use of Qwen-Image-2.1 Outputs — Clear Answer Needed", discussion #40, Hugging Face. https://huggingface.co/Qwen/Qwen-Image-2.1/discussions/40
- "Diamond grid pattern caused by VAE", discussion #12, Hugging Face. https://huggingface.co/Qwen/Qwen-Image-2.1/discussions/12
- "Diffusers 0.41.0: QwenImage 2.1 pipeline and more", GitHub release, 2026-10-06. https://github.com/huggingface/diffusers/releases/tag/v0.41.0
- Comfy-Org, Qwen-Image-2.1 repository, Hugging Face (commit history). https://huggingface.co/Comfy-Org/Qwen-Image-2.1
- ComfyUI v0.37.0 release notes, GitHub, 2026-09-21. https://github.com/Comfy-Org/ComfyUI/releases/tag/v0.37.0
- Black Forest Labs, FLUX.1 [dev] model card. https://huggingface.co/black-forest-labs/FLUX.1-dev
- Black Forest Labs, FLUX.2 [klein] 4B model card. https://huggingface.co/black-forest-labs/FLUX.2-klein-4B
- Wikipedia, "Dishonored". https://en.wikipedia.org/wiki/Dishonored
- RealKeep. https://realkeep.strata2signal.com/
- Wikipedia, "Qwen". https://en.wikipedia.org/wiki/Qwen
- Who makes FLUX: Black Forest Labs and its licences, model by model.
- The same eighteen pictures, card by card: FLUX.1 [dev] and klein 4B timed on six graphics cards.
- The licence ledger: every model this workshop runs, its licence read first-hand.
Who ran this, and thanks
Thanks to Qwen, to Black Forest Labs, and to the people whose public reports on the Qwen-Image-2.1 pages told us what to look for. The Turbo ran in Diffusers 0.41.0 and Transformers 5.19.0 (Apache 2.0) on PyTorch 2.13.0 (BSD-style); the FLUX columns in ComfyUI v0.21.1 (GPL-3.0); all under CUDA (NVIDIA, proprietary), the one closed piece. Arkane Studios' Dishonored is named only for its style. A small human team asked for this test, set its rules first, made the blind picks and signed the numbers; a fleet of AI agents read the sources, built and ran the harness, and drafted this page under that team's rulings.
Corrections and later measurements will be added below, each dated (UTC), with a window at both ends where one applies, saying in plain words what it counts.