Experiment · Paper 1 · Study 6 · Gate E
EXP-007 — Cross-runtime (Codex CLI)
Preregistered replication of the EXP-004 contract-vs-baseline comparison under the Codex CLI runtime, with Claude Code as the cross-runtime reference. Tests whether contract dominance generalizes across runtime. Runs pending Codex install and API key.
Preregistered hypothesis
Under the Codex CLI runtime on the OPBR-Bench cross-runtime partition, behavior contracts will exceed the best non-contract baseline by ΔF1 ≥ 0.30 with paired McNemar p < 0.01 (Holm-corrected across the baseline family). The result will hold regardless of the direction of the contract-vs-output ranking under Claude Code.
Why this matters
EXP-004 establishes contract dominance under Claude Code. Without a second runtime, the result is consistent with a confound: contracts could be exploiting runtime-specific trace structure rather than the underlying behavioral regularity. Gate E of the Paper 1 acceptance protocol requires ≥ 2 runtimes. This experiment closes that gate. A null or overturned result on Codex would be reported in full, with the corresponding scope tightening of Paper 1's claims.
What this experiment is not
This is not a vendor comparison. The cross-runtime test is methodological — the question is whether the contract mechanism generalizes across the runtime layer, not whether one runtime is better than another. Result reporting will be runtime-symmetric.
Replication (after execution)
abw run --experiment EXP-007 --runtime codex
Requires Agent Behavior Workbench ≥ 0.4 with Codex adapter. Requires OPENAI_API_KEY and a Codex CLI install. Corpus auto-downloads from OPBR-Bench v0 cross-runtime partition.
Artifacts
Every load-bearing claim on this page traces back to one of the artifacts below. They are the canonical citation targets — not the prose.
exp-007-protocol.json— frozen preregistration: hypothesis, falsifier, method, detectors, primary metric, preregistered gates, claim boundary.- Agent Behavior Workbench — open-source code and OPBR-Bench v0 corpus. Required to run
abw run --experiment EXP-007. - Paper 1 · Study 6 · Gate E — Closes Gate E of the Paper 1 acceptance protocol (>= 2 runtimes).
Cite as: Sanchez de la Sierra, A. (2026). EXP-007 — Cross-runtime (Codex CLI). Raising Agents Lab. https://raisingagents.is/lab/exp/exp-007
Related
- Paper 1: Output-Passing Behavioral Regressions · Study 6 (Gate E)
- Same-design under Claude Code: EXP-004 — Held-out repair
- Confirmatory: EXP-006 — Confirmatory E1+E2