Skills 2.0 — making Anthropic's prose skills a Python contract

What a prose contract leaks.

Anthropic shipped Agent Skills in late 2025 as a SKILL.md file with YAML frontmatter and progressive disclosure: three levels of loading, scripts and references and templates as siblings, a description string Claude matches against the user's prompt. The spec is excellent for distribution. Forty skills cost about 1,500 tokens of overhead.

The spec is also a prose contract. There is no I/O schema. There is no proof a step ran. There is no telemetry. There is no evaluation gate. The skill describes its decision flow in the body and trusts Claude to follow it. When Claude does not, no one finds out until the output is wrong.

Seven specific symptoms, each one you have probably seen, none caught by output-passing evals:

You can ignore each one for a while. The cost compounds. By skill 20, you have 40 prose contracts the model honors with one degree of freedom each.

The five layers.

Skills 2.0 preserves the v1 prose verbatim. SKILL.md is unchanged, still loaded by Claude exactly as before. Five Python layers sit next to it.

1. Manifest — skill.toml

Every step gets a machine-readable contract.

[[steps]]
id = "compute_single_period_identity"
order = 4
required = true
fn = "runtime.steps:compute_single_period_identity"
invariants = ["runtime.validators:identity_closes"]

2. Runtime — the @step decorator

Each step is a Python function the runtime wraps with order checking, Pydantic I/O validation, invariant enforcement, and append-only telemetry to ~/.claude/skills2/runs/.

@step(
    order=4,
    requires=["pick_decomposition_form"],
    invariants=["runtime.validators:identity_closes"],
    output_schema=IdentityCheck,
)
def compute_single_period_identity(prev_mau, new, resurrected, churned, current_mau):
    implied = prev_mau + new + resurrected - churned
    residual = current_mau - implied
    return IdentityCheck(..., identity_closes=abs(residual) <= 1.0)

The invariant — prev_mau + new + resurrected − churned = current_mau — closes or it does not. There is no "the model thinks it does."

3. MCP server — every step a tool

Each @step function auto-registers as an MCP tool named mcp__skills2__{skill}__{step_id}. Claude calls them natively. Pydantic-model arguments bridge transparently to JSON Schema at the tool boundary, so FastMCP can introspect the surface without seeing the skill's internal types.

For growth-accounting, this exposes fifteen tools — ten step functions plus __start_run, __end_run, __audit_run, and two global listing tools.

4. Hooks — pre-injection, post-audit

A single dispatcher reads stdin JSON from Claude Code and routes:

• UserPromptSubmit — match the prompt against installed-skill triggers; if a skill activates, inject the step checklist into Claude's next-turn context.
• Stop — spawn a fully-detached audit worker that scores the most recent run on three axes and appends to ~/.claude/skills2/ledger/audits.jsonl. The hook itself exits in under 200ms. Fail-open. Same pattern as the existing beast-mode-stop.py.

Installation into ~/.claude/settings.json is idempotent. Existing hooks are identified by a substring marker and preserved.

5. Audit — three-axis trust score

Three dimensions, weighted:

Threshold default 0.80. Below threshold the post-hook surfaces missing-step and failed-invariant lists in the transcript. The next turn sees the audit; the model corrects. No hard blocking — Claude Code's hook protocol does not currently support "reject and rewrite," and inventing it would couple Skills 2.0 to a contract that might change.

6. Evals — cases.jsonl

Each case has must_call, must_mention, must_not, and (for tests of failure modes) expected_audit: fail. The eval runner returns a pass-rate. CI gates version bumps on regression.

Five skills, 48 rows, one number.

Five skills, transformed once, run under two versions:

• v1 — null runner. Claude answers the prompt with only the SKILL.md prose in context. No step functions fire. The audit runs over an empty telemetry log.
• v2 — full step chain. For growth-accounting, every step is hand-implemented. For the other four, the transformer-generated stubs run as-is.

Three readings.

The v1 baseline at 0.25 is the ANCHORING component alone. The prose teaches the model the vocabulary, the model uses it, nothing else can be scored because nothing else fires. That is the ceiling on Skills-as-prompts. Not low because the model is dumb; low because there is nothing else to measure.

The 0.53 v2 number across four skills is what skills2 transform produces with zero manual code. The stubs return {}. The schema validates trivially. The invariants are empty. Coverage rises from 0 to 0.7 because the steps fire in order. Anchoring holds. This is the floor of Skills 2.0 — the per-skill effort is running one CLI command.

The 1.00 on growth-accounting is what the runtime can reach when the step bodies actually compute the quantities the prose names. The MAU identity closes or the invariant raises. The quick-ratio classifier output matches the threshold cuts or the consistency check fails. The dominant-flow function gets called or COVERAGE drops below threshold and the hook tells the model on the next turn.

What this issue does not claim

(1) That five skills generalize to the full corpus of 52 — the headline number is the median across the chosen sample, not extrapolated. (2) That generic eval cases — three per skill, generated by the model from the "When to use" section — substitute for hand-tuned cases. (3) That Skills 2.0 makes a bad skill good: if the SKILL.md prose is wrong about the framework, the contract enforces the wrong thing more reliably. (4) That Skills 2.0 eliminates the model — the orchestration is still Claude. The contract bounds the failure modes the model can produce. It does not bound whether the model is the right tool for the work. (5) That the MCP and hooks layers will continue to work unchanged through 2026 — they depend on Claude Code's hook protocol staying stable. Fail-open is the right default.