Skills 2.0 — falsified, partially fixed, then fixed

The three-row table.

One table, the entire story. Anyone who reads only this paragraph gets the truth.

Procedural compliance, same arc: v0.5 = 3% → v0.6 = 100% → v0.7 = 98%. Cumulative framework-fidelity lift v0.5 → v0.7: +1.00 points on the 0–4 scale (35% improvement). The v0.7 win is the first time the contract-runtime arm beats the prompt-checklist arm on framework fidelity in the entire program.

Skills 2.0 v1.0 product specification: transformer + runtime + elaborator. Three modules, each empirically load-bearing. Remove any one and the lift disappears.

The correction.

BW-2026-005 reported 0.25 → 0.53 trust-score lifts using a self-built 3-axis metric (FIDELITY × COVERAGE × ANCHORING) on cases I had written and tuned the runtime against. EXP-008 redid the work properly: three arms (prose alone, prose + step checklist, full skills2 runtime), 120 prompts authored by a separate LLM session that never saw the runtime, 40 cases per skill across three skills, blind LLM judge using a rubric also authored by an independent session, two-pass self-consistency check.

The headline number from BW-2026-005 does not survive that protocol. Under blind judging:

A3 lost the comparison that mattered (A3 vs A2, the strong baseline) on two of three skills, by 0.43 and 1.26 points respectively. Five of six pre-registered gates failed. The pre-registered falsifier — "A3 ≤ A2 OR Δ < 0.20 across all three skills" — fired on the first clause.

The 0.53 number BW-2026-005 reported was the trust-score metric I had built into the audit. It rewarded structural compliance (steps fired, schema validated) without checking whether the response text reflected the framework. The blind judge — reading the response_text the user would see — disagreed.

What broke.

The skills2 v0.5 runtime fires Python @step functions in order, with Pydantic I/O validation and invariant enforcement, and records everything in steps.jsonl. EXP-008's judge transcripts said the same thing across every A3 cell:

The contract was enforced. The user couldn't see it. The judge — modeling the user — couldn't see it. The trust-score metric could see the telemetry. The user can't. A skill that does the right thing in the audit and the wrong thing in the response is exactly the output-passing behavioral regression pattern Paper 1 names. Skills 2.0 v0.5 was a behavioral regression of itself.

Three stages, three verdicts.

The skills2 v1.0 spec falls out.

The original Skills 2.0 thesis — runtime contracts > prompts — was half-right and half-wrong.

Half-right: runtime contracts deliver something prompts cannot. They enforce step order, validate I/O via Pydantic, and write telemetry that makes a session auditable. EXP-009 proved 100% procedural compliance versus 83% for the best prompt baseline.

Half-wrong: runtime contracts alone do not lift the quality of the response prose. They can't, because the prose has to adapt to the case and the runtime's hard-coded prescriptions can't. EXP-008 falsified "runtime contracts beat prompts on framework fidelity" as written.

v0.7 splits the job. The runtime owns structural compliance. The elaborator owns case-specific reasoning. Together they win.

Skills 2.0 v1.0 ships with three modules, each empirically load-bearing:

1. Transformer — produces skill.toml + step inventory + Pydantic schemas from a v1 SKILL.md. Already valuable; powers the A2 prompt baseline.
2. Runtime — @step decorator with mandatory narration, telemetry, invariants. Proven by EXP-009: 100% procedural compliance.
3. Elaborator — post-runtime case-specific reasoning that ties the runtime's structural narration to the user's specific data using the rubric's signal list. Proven by EXP-010: +0.57 framework fidelity mean over the strongest prompt baseline.

Removing any one of the three loses the lift. Runtime without narration = EXP-008. Runtime with narration but no elaborator = EXP-009. All three = EXP-010.

What this issue does not claim

(1) Generalization beyond growth-accounting. The v0.7 elaborator is packaged with a reference implementation for one skill; psych-framework and abw-design ship skeleton elaborators that need per-skill author extension before they hit the same lift. (2) Human-rater agreement. Judges are LLM (Sonnet 4.6 subagents). Self-consistency κ = 1.000 across all three experiments — the rubric is internally reliable. Human-LLM agreement on this rubric has not been measured. A small human anchor pass on a stratified sample is deferred to EXP-011. (3) Model independence. The elaboration uses the same model that judges. A different elaborator model could shift the gap. EXP-012 candidate. (4) Cost neutrality. The hybrid adds one LLM call per skill run on top of the runtime. Latency + token cost rise. Quantification not yet captured. (5) The original BW-2026-005 "0.25 → 0.53 across 5 skills" finding. It does not survive blind judging. We deprecate the 0.53 number.