Every pre-registered test, failures included.
Each experiment was pre-registered and committed before it ran. Every message and model call is recorded, and each results file carries a hash. On real hosts: 41 predictions, 25 passed and 16 failed.
Host for A1–A5: planner → worker → checker, all claude-haiku-5-5, 14 change
requests on a 30-line shared state, 3 injected drifts per task. Channel tokens use a regex count; every compactness
pass also holds in BPE tokens (o200k). Source of every number: whitepaper v2 §7–§9, the A3c, A4 and A5 reports, and the files in
results/.
A5 · The model bill against the market's best setup
Target: 10× below the best setup the market could build for the same agents. Phase 0 built that
baseline honestly first: it found that most of the earlier bill was the command-line tool's own context on every call,
fixed it, and priced every call from the official price table. Then the planner writes RSOS operations and the runtime
executes and checks them; in the assembly line (X5) all 14 requests of a task are translated in one call and audited in
one call. 192 live tasks, 4,917 calls. sha256 9cd1eb92…
| S1 market best | S1 assembly line | S2 market best | S2 assembly line | |
|---|---|---|---|---|
| Exact / every copy true | 16 / 16 | 16 / 16 | 15 / 15 | 16 / 16 |
| Model calls per task | 43.1 | 2.4 | 42.1 | 2.2 |
| Cost per task (API prices) | $0.004598 | $0.001233 | $0.003815 | $0.001352 |
| Ratio to market best (95% CI) | 1 | 0.268 [0.240, 0.300] | 1 | 0.354 [0.336, 0.374] |
| Ratio as billed | 1 | 0.173 | 1 | 0.200 |
| Time per task | 106 s | 12 s | 99 s | 12 s |
| Drift healed with 0 calls · wrong values from drift | 48/48 · — | 48/48 · — | 48/48 · 0 | 48/48 · 0 |
2 of 6 pass no drift-traced errors (K5), offline baseline estimate within 12% (K6). K1 failed the registered arm (X3) reached 0.58× / 0.83×, not 0.1×; the assembly line reached 0.27× / 0.35×. K2, K3 failed per-step arms kept 22–28 calls and thought more than the baseline. K4 failed X3 missed one task per set on auditor errors. The audit approved 0 wrong steps in 192 tasks; one wrong step went through unsampled.
What changed and what limits it. A normal agent pipeline needs one model round trip per request, because each request reads what the last one wrote. Once the model writes a program the runtime executes, that dependency moves to the runtime, and the cost stops growing with the number of steps. The remaining bill is the model's thinking: 78–79% of the assembly line's cost is output, 86–87% of it thinking. Against our own earlier runs the cost per task fell about 12×; about 2.4× of that is the harness fix anyone can make.
A4 · Does hidden drift become wrong answers?
A3 left 40 of 48 corruptions latent in the anchor protocol's copies. A4 makes the worker compute from
its own copy (8 of 14 steps per task). 80 tasks, 4,000 calls. sha256 c2f41b80…
| H-dragon | P · palace | D · anchors | E · deltas | A · JSON | |
|---|---|---|---|---|---|
| Exact success | 15/16 | 16/16 | 0/16 | 1/16 | 1/16 |
| Wrong values traced to drift | 0 | 0 | 93 of 96 | 14 | 7 |
| Drift healed with 0 model calls | 48/48 | 48/48 | 0/48 | 0/48 | 0/48 |
| Channel tokens (BPE) | 575 | 2,231 | 1,038 | 2,374 | 13,736 |
3 of 4 pass the anchor protocol collapses (L1), no drift-traced errors for the integrity protocols (L3), H-dragon stays smallest (L4). L2 failed one worker model error on a true, healed copy (seed 407). The test is a worst case by design: every corrupted line is read later.
A3c · The prose floor
A3b missed 5× because the planner's free prose must be restored word for word. A3c codes it against a
reference both sides hold: a deterministic predictor with no neural network (H-pred), and a labelled arm with a local
language model (H-local). sha256 1d797624…
| D · anchors | H-dragon | H-pred (0 model calls) | H-local (local model) | |
|---|---|---|---|---|
| Channel tokens per task (BPE) | 1,860 | 1,023 | 262 | 197 |
| × D | 1 | 0.55 | 0.141 | 0.106 |
| Exact / every copy true | 16/16 / 0/16 | 16/16 / 16/16 | 15/16 / 15/16 | 16/16 / 16/16 |
| Drift healed with 0 model calls | 0/48 | 48/48 | 48/48 | 48/48 |
| Prose bits per character | — | — | 0.60 | 0.41 |
2 of 4 pass H-local reaches 0.106× with full integrity (C3); the White Dragon predicted the size within 5% (C4; in A3b it was 66% off). C1, C2 failed H-pred's one miss was a checker rejecting a correct deletion three times. H-local needs about 1 GB of weights on both sides and ran about 15× slower on CPU.
A3b · The hologram palace
The White Dragon designed a compact protocol in one call, with its own predicted size (H-dragon).
A hand-built control used the anchor protocol's formats plus the green dot (H-copy). 16 tasks per arm, 1,374 new
calls; palace, D and E were reused from A3 after all 48 of their runs replayed with 0 mismatches.
sha256 0ae0d031…
| Palace | D · anchors | H-copy | H-dragon | |
|---|---|---|---|---|
| Channel tokens per task (BPE) | 2,699 (3,131) | 1,468 (1,860) | 1,290 (1,695) | 932 (1,023) |
| × D, regex / BPE | 1.84 / 1.68 | 1 / 1 | 0.88 / 0.91 | 0.64 / 0.55 |
| Exact success | 16/16 | 16/16 | 12/16 | 16/16 |
| Every copy true | 16/16 | 0/16 | 12/16 | 16/16 |
| Drift healed with 0 model calls | 48/48 | 0/48 | 47/48 | 48/48 |
| Channel tokens per heal | 37.9 | — | 2.0 | 0 |
| Truncations silently accepted | 0 (A3) | 13/1,150 | 0/1,175 | 0/1,140 |
5 of 7 pass for H-dragon: smaller than D (B1), full integrity (B3), success ≥ D (B4), better than H-copy (B5), heal cost (B7). B2 failed target 0.2× D, reached 0.64×. B6 failed the engine predicted 321 tokens per task; 932 were measured. H-copy: 2 of 5 pass; its four inexact tasks were model edit errors that the checker approved.
Why 5× was missed, measured. Where the engine landed, cost fell close to the floor: verdicts 854 → 32 tokens per task, edits 196 → 44, forwards 187 → 49, heal 114 → 0. 87% of what remains is the planner's free prose (808 tokens), which must be restored word for word. The engine had named this as its main uncertainty: right about where, wrong about how much.
A3 · Head to head with compact protocols
D and E are re-implementations from the published descriptions of Slipstream (shared concept
anchors) and ACCP (deltas with sequence numbers), not the original systems. 16 tasks × 4 arms, 3,200 calls.
sha256 ff058345…
| Palace (A1) | JSON | D · Slipstream-style | E · ACCP-style | |
|---|---|---|---|---|
| Exact success | 16/16 | 4/16 | 16/16 | 1/16 |
| Every copy true at the end | 16/16 | 4/16 | 0/16 | 1/16 |
| Channel tokens per task | 2,699 | 17,442 | 1,468 | 2,773 |
| Drift healed with 0 model calls | 48/48 | 0/48 | 0/48 | 0/48 |
| Drift left in some copy | 0 | 0 | 40 | 6 |
| Truncations silently accepted | 0 | 0 | 13/1,150 | 99/1,955 |
2 of 4 pass healing (R2), success (R3). R1 failed the palace was 1.84× the size of D, from formats inherited from A1, not from its integrity layer. R4 failed on channel tokens against E (38 vs 96 per repair); counting model work, the palace's repair costs 0.9% of E's. The anchor protocol stays right only because no agent reads the drifted copy: 40 of 48 corruptions stayed latent. A4 tests whether they turn into wrong answers.
A2 · The knowing White Dragon redesigns a weak host
Host: A1's summaries-plus-retrieval workflow, 5.83 wrong lines per task. Given only its parts,
weaknesses and a cross-domain library, the engine landed four changes in order. The first was a memory journal with
per-line check and restore, drawn from DNA mismatch repair and ledger reconciliation. 6 tasks per stage, 5,687 calls.
sha256 8916f4de…
| Arm | Wrong lines after 4 landings | Change | Billed tokens |
|---|---|---|---|
| Knowing | 2.17 | −63% | +20% |
| Blind | 8.17 | worse | +95% |
| Weighted (F1b rule) | 3.33 | −43% | +72% |
4 of 4 pass knowing beats blind and weighted; gain on 3 of 4 landings; predicted total gain within 16% of measured; the map's order beats a shuffled order. The engine re-derived the palace's green dot and heal on its own, from other domains, and placed it first.
A1 · Palace against three strong baselines
8 tasks per arm, 1,522 calls. sha256 d224abbf…
| Palace | JSON full state | Structured output | Summaries + retrieval | |
|---|---|---|---|---|
| Exact success | 7/8 | 5/8 | 5/8 | 0/8 |
| Channel tokens per task | 2,563 | 17,311 | 24,425 | 4,561 |
| Drift caught | 24/24 | 24/24 | 23/24 | 0/24 |
| Per repair: channel tokens / model tokens / time | 38 / 0 / 0.15 ms | 875 / 6,894 / 13.8 s | 1,607 / 10,214 / 13.0 s | — |
5 of 6 pass P2 failed a checker that remembers the last approved state also caught all drift. Against full-state baselines the difference is the cost and determinism of repair, not the detection rate; against the memory pattern common in production, detection itself differs: 24/24 vs 0/24.
HT1 · Human input
The author entered six of his own models as typed symbols and as text. Symbols were faster on 0 of 5 timed tasks while also at least as accurate, so K1 failed. Mean accuracy was 0.88 for symbols and 0.96 for text; where symbols were faster, an element had been dropped.
The text was written as shorthand, so HT1 compares two compressed forms by the same person.
sha256 e1f8266b…
What followed
Typing was the bottleneck, so an RSOS keyboard layer was built: one key or chord emits a whole component or operator. An automated check keyed 12 targets and decoded them blind from the code alone: 12/12 after one bug fix, 88 of 89 fields recovered (0.989). HT2 runs after a learning phase with a pre-registered plateau gate.
Open the keyboard →The failure map
All 31 pre-registered failures to date are listed with their cause and the next test. Of the first 20, 14 are failures of blind choice: fixed rules, random growth, resending, or typing by hand. A3c answered two more (the prose floor and the engine's size prediction). Every remaining failure of the integrity protocols is a single model error on a true copy, never a codec or integrity failure; the cost failures of A5 point to thinking tokens.
Read the full failure mapEarlier results: simulation and code
| Test | Result |
|---|---|
| C1 palace messages | 280/280 decoded exactly; drift detected 366/366 |
| T7 structure vs names | meaning carried by structure 98.3% vs names 3.3%; with 10% relation noise: 47.5% (failed) |
| T8 one program, two runtimes | a person's session and an agent from the same program (agent accuracy 0.488 → 0.762) |
| V1 sense keys (simulation) | one rich sense channel opens the anchored state 0.90–0.92 vs a name key 0.18 |
| H3c evolve-healing (simulation) | quality −0.19 → +0.67; harmful copies 4.95% → 0.60% |
| E1, F1b White Dragon growth (simulation) | deficit targeting 2.5×; entry reinforcement +0.02 (failed) |
Limits
- One model family (Haiku-class, one provider). The cross-family test is open.
- 8–16 tasks per arm: integrity and token differences are large; success differences of one or two tasks are not established.
- One task family: structured state editing. Free-text collaboration is untested.
- Competitors are re-implemented from public descriptions; our lossless quantiser favours D.
- The zero-token heal assumes a second, untouched copy; with both corrupted it costs 18.5 tokens per drift (offline).
- A2 and A3b each used one engine design call, so the variance between calls is unmeasured.
- Human evidence: one person, the inventor. No effect on people has been shown.
- Three of five ledger instruments tested did not track their claims and are being rebuilt.