---
title: "Delegation-Grade Agents: Why Determinism Is Not Enough for Agent Reliability"
subtitle: "Toward predictable quality under variation, with consequential behavior made contract-bound"
type: whitepaper
status: v1 — public draft
version: 1.0
author: Adrian Sanchez de la Sierra
affiliation: Raising Agents — a public research lab on output-passing behavioral regression
date: 2026-05-25
license: CC BY 4.0
length: ~9,500 words
evidence_base:
  - Paper 1 — Output-Passing Behavioral Regressions in AI Agents (V4 manuscript, 3,797 runs, 5 studies executed)
  - Paper 2 confirmatory — E1 and E2 direct empirical tests (540 controlled runs on Claude Sonnet via Claude Code, preregistered as PAPER2_PROTOCOL_FREEZE.json)
  - R1 — Determinism, Delegation, and Trust: A Layered Synthesis Across Six Literatures (265 paragraphs, ~50 cited sources)
  - R2 — How Major AI Builders and Tooling Vendors Frame Non-Determinism, Reliability, and Production Readiness (319 paragraphs, 11 vendors audited)
companion_artifacts:
  - /papers/opbr-contracts.html — Paper 1 landing
  - /lab/opbr.html — OPBR canonical interactive demo
  - /bench/ — OPBR-Bench v0 benchmark specification
---

# Delegation-Grade Agents

## Why Determinism Is Not Enough for Agent Reliability

**Toward predictable quality under variation, with consequential behavior made contract-bound.**

---

## TL;DR — the paper in sixty seconds

- **Determinism is the wrong top-level target for agent reliability.** Token-level bitwise determinism is now an engineering option at 10–40% performance cost, but the dominant production failure mode — *consistent-wrong-interpretation* in long-horizon agents (~65–71% of failures across two independent corpora) — is invisible to it.
- **Two direct experiments confirm the dissociation.** 540 controlled runs of a CI code-review agent on Claude Sonnet: output correctness held at 100% across all conditions; behavioral-contract compliance held at ~100% under both rigid step-by-step instruction and open-ended instruction, and collapsed to ~3% under engineered prompt drift that removed the safety-check constraints. Output correctness is not a signal for contract compliance; the contract structure in the prompt is.
- **Five formal traditions converge on the right target.** PCTL (1994), e-process sequential inference (2009 onward), robust MDPs (2024), AI reliability science (2026), and Agent Behavioral Contracts (2026) — each derived without coordination — express the reliability object as a probability-bounded specification *P*<sub>≥*θ*</sub>[*ψ*]. Determinism is the degenerate corner of every one.
- **Five governance traditions converge on the same mechanism.** Hawley's commitment philosophy, ABC, Kolt's governance law, Jarrahi–Ritala's guided autonomy, and Hadfield-Menell's external normative infrastructure — five literatures arriving at *attributable, violation-detectable commitments* from independent starting points. No source names this cross-family convergence. Naming it is part of the contribution.
- **The positive target is delegation-grade behavior.** Six conditions, formally specified (§5). Not universally trustable; scoped to a bounded operational design domain. The Drift Bounds Theorem (§7) quantifies the cost of insisting on determinism: a thirty-fold compliance gap over 100 agentic steps.
- **One open problem is named, not concealed.** Consistent-wrong-interpretation produces zero soft violations against currently-published contracts. The trajectory-shape signals exist (3.5× step inflation, canonical-path adherence at 50% completion) and are specifiable as soft invariants but have not been instantiated in published contracts. Closing the gap is the next decade's work.

---

## Abstract

Agent reliability is often framed as a problem of making non-deterministic systems more deterministic. This whitepaper argues that determinism is a useful engineering property but the wrong top-level target for agent reliability. Agents are valuable partly because they adapt, explore, recover, and vary their behavior across contexts. The relevant question is therefore not whether an agent behaves identically, but whether it can be delegated scoped consequential work under explicit constraints. We define **delegation-grade agents** as non-deterministic systems whose behavior remains valuable, inspectable, and contract-bound at consequential moments. We distinguish *mechanism determinism* from *behavioral determinacy* and show, drawing on five independently-derived formal traditions and five governance traditions, why repeatable outputs are neither necessary nor sufficient for delegability. We propose **behavior contracts** as executable specifications over consequential trajectory events, connecting final outcomes with permissions, evidence requirements, action preconditions, audit trails, and repair loops. The result is a reliability frame for agentic systems: *predictable quality under variation*. We test the dissociation directly: 540 controlled runs of a CI code-review agent across three conditions — explicit step-by-step instruction, engineered prompt drift, and open-ended instruction — show that output correctness held at 100% in every condition while contract compliance collapsed to ~3% under drift and remained at ~100% under both rigid and open-ended instruction. The contract structure in the prompt determines compliance; the rigidity of the instruction does not, and output correctness is silent on it. We also identify one load-bearing open problem — consistent-wrong-interpretation, the dominant failure mode of long-horizon agents — that current behavior-contract instantiations do not yet catch, and name it as the field's next research target rather than concealing it.

---

## Thesis

> **Do not make agents deterministic. Make consequential behavior contract-bound.**

The right reliability target for agentic systems is not identical behavior under identical conditions. It is *predictable quality under variation*: an acceptable outcome distribution, bounded consequential action, inspectable evidence, and recoverable failure. Behavior contracts are the mechanism that makes consequential behavior specifiable, testable, and repairable — without overconstraining the path the agent takes to get there.

---

## 1. Introduction: Determinism Instinct

The classical software-reliability ideal is captured in five words: *same input, same output*. The ideal is useful for debugging, reproducibility, benchmarking, and some compliance contexts. It is the default ontological commitment a software engineer brings to a new system, and for most of the history of software, it has been the right default.

For large language model (LLM) agents, that default is no longer ontologically appropriate. The inference core implements a conditional distribution, not a fixed function. Token-level non-determinism is real even at temperature zero — Atil and colleagues document accuracy variation up to 15% across runs of five LLMs configured for "deterministic" inference, with best-to-worst gaps of 70 percentage points (Atil et al., 2024). Thinking Machines Lab traced the structural cause to batch non-invariance: matmul, RMSNorm, and attention kernels change reduction order when batch size shifts because of other concurrent requests, producing bitwise-different logits depending on what is batched alongside (Thinking Machines Lab, 2025). Bjarnason and colleagues, across 60,000 trajectories on SWE-Bench-Verified, found standard deviations exceeding 1.5 percentage points even at temperature zero, with observed ranges of 2.2–6.0 percentage points; the team concluded that "reported improvements of 2–3 percentage points may reflect evaluation noise rather than genuine algorithmic progress" (Bjarnason et al., 2026).

Token-level determinism is now an engineering option with a known price tag. Thinking Machines demonstrated 1,000 of 1,000 bitwise-identical completions on Qwen3-8B at a 10–40% performance cost using batch-invariant kernels. SGLang reproduced the result in production and collapsed normal-mode unique outputs from 4–18 down to exactly 1 across 50 trials at an average 34.35% slowdown (LMSYS, 2025). The November 2025 Tree-Based Invariant Kernels (TBIK) paper eliminated the last cross-tensor-parallel divergence (Anonymous, 2025). The strong constitutive claim — that determinism is in principle infeasible — has been falsified at the engineering layer.

But this achievement is semantically irrelevant where production agents actually fail. Mehta's *Consistency Amplifies* study found that 71% of Claude 4.5 Sonnet's SWE-bench failures across fifty trajectories were *consistent-wrong-interpretation* — the same coherent misreading of the task across every run (Mehta, 2026). Liu and colleagues independently replicated the result at approximately 65% on 150 annotated SWE-bench-Verified failures with Cohen's kappa of 0.72 across three coding tools (Liu et al., 2025). The dominant failure mode of long-horizon LLM agents operates at the *goal-interpretation layer*; bitwise-identical token sampling does not address it.

The determinism reflex is therefore a layer-confusion. Determinism is engineering-tractable at the token layer in controlled environments, and engineering-irrelevant at the agentic layer where actual reliability is determined. This whitepaper argues for a different reliability target — *delegation-grade behavior* — drawn from six independent literatures that have converged on the same answer without coordinating across families. Determinism belongs inside the stack. It is not the stack.

What this paper does not argue: it does not claim that determinism is unimportant, that reproducibility is obsolete, that trajectory matters more than outcome, that agents should be trusted like humans, that behavior contracts guarantee correctness, or that delegation-grade agents are solved. The correct stance is narrower: determinism is one useful property inside the reliability stack; it is not the whole reliability target, and elevating it to the top-level target is the categorical error that produces the misallocation of engineering attention this paper describes.

---

## 2. Why Agents Break the Classical Frame

The software-engineering literature treats non-determinism as a defect, and that norm rests on a specific and defensible ontology. The position has been criticized by the LLM reliability community as a category error when applied to stochastic agents. Both communities are right within their domains; the resolution is layered.

The canonical responses to flaky tests are elimination strategies: rerun, quarantine, root-cause fix (Rasheed et al., 2022). Hashemi and colleagues' JavaScript study finds that more than 80% of flaky tests are fixed by elimination (Hashemi et al., 2022). The strongest philosophical version of the argument: a test that can both pass and fail for the same code under identical conditions cannot falsify the code in the Popperian sense — it cannot serve as a falsifier. The elimination norm is not founded on ignorance of stochastic methods; it is founded on a logical condition on what a test *is*.

The norm rests on a particular substrate assumption: the system under test implements a deterministic specification. For LLM agents, this assumption fails at the inference core. The Stochastic CHAOS paper makes the explicit ontological argument: "LLMs implement conditional distributions over outputs, not fixed functions" (Anonymous, 2026). Greedy decoding does not eliminate non-determinism; it conceals it. Concealment produces documented harms — single-sample evaluations underestimate capability and fragility, emergent-ability phase transitions disappear, multi-path reasoning degrades, and rare safety failures become invisible.

The constructive position emerges from a layered taxonomy. Three flakiness concepts transfer to agentic systems intact:

- the **quarantine discipline**, generalized from "is this run flaky" to "is this run within-distribution variance or a regression";
- the **oracle precision problem**, generalized from point oracles to distributional oracles;
- the **CI/CD gate structure**, translated from bit-equality checks to statistical decision procedures with confidence-interval thresholds.

What does not transfer is the elimination norm itself, the re-run-until-consistent fix strategy, and the binary pass/fail verdict. AgentAssay is the engineering instantiation of the surviving SE concepts under distributional assumptions: stochastic three-valued verdicts (PASS / FAIL / INCONCLUSIVE) via Sequential Probability Ratio Testing (SPRT — a sequential hypothesis test with bounded Type-I and Type-II error), behavioral fingerprinting of execution traces, CI/CD gates as statistical decision procedures, achieving "78–100% cost reduction while maintaining rigorous statistical guarantees" and "86% detection power where binary testing has 0%" (AgentAssay, 2026).

The statistical-testing literature supplies the formal tools that make this rigorous. Grünwald, de Heide, and Koolen's *Safe Testing* framework introduced the e-value as a notion of evidence that "allows for effortlessly combining results from several studies in the common scenario where the decision to perform a new study may depend on previous outcomes" (Grünwald et al., 2019). E-processes — nonnegative martingales that encode anytime-valid sequential evidence — were proved both sufficient and necessary for admissible anytime-valid sequential inference: Ramdas, Ruf, Larsson, and Koolen demonstrate that "all admissible constructions of confidence sequences, p-processes, or e-processes must necessarily utilize nonnegative martingales" (Ramdas et al., 2020). Where binary tests force an arbitrary horizon (run the test *N* times, count failures), e-processes provide certificates that accumulate evidence at every observation and stop at any time with controlled Type-I error.

The committed position: in its home regime, the SE elimination norm is sound — three flakiness concepts transfer intact to agentic systems, generalized via the e-process formalism to give a rigorous statistical foundation for testing under distributional variance. The position that fails is the strong constitutive claim that flakiness must be eliminated *everywhere*; that claim assumes a deterministic specification under test, which is the wrong target for LLM agents at the orchestration and semantic layers but the right target for everything below the inference core in a self-hosted deployment.

---

## 3. The Delegation Analogy

Organizations have always delegated to non-deterministic actors — humans. They do so through roles, permissions, standards, gates, audits, escalation, and feedback. The control architecture is the point; identical behavior is not.

Classical principal-agent (P-A) theory provides the diagnostic framework but, applied to LLM agents, fails on all three of its classical solution mechanisms. Holmstrom's informativeness principle — the foundational Nobel-cited result — states that the optimal contract should condition on any signal informative about the agent's effort beyond the outcome itself (Holmstrom, 2016). For human agents in classical settings, the principle is generative: monitoring technology, peer review, time tracking, and similar instruments add informative signals that reduce moral hazard. Eisenhardt's framework adds the orthogonal axis: when behavior is observable, write behavior-based contracts; when only outcomes are observable, write outcome-based contracts. This framework presupposes three things: that agent effort is partially observable or inferable from outcomes; that outcomes are separable from stochastic noise; that agents are rational utility-maximizers with alignable incentives.

For LLM agents, all three assumptions fail. Jarrahi and Ritala's *California Management Review* piece applies the framework to AI agents and identifies four governance challenges: goal misalignment, information asymmetry, division of work (a "moral crumple zone"), and multi-agent complexity (Jarrahi & Ritala, 2025). Their mitigation is *guided autonomy* — explicit boundary conditions for delegation with expanding boundaries as trust is earned. They acknowledge that "generative AI foundation models often exhibit surprising performances that can be unpredictable, inconsistent, and even erratic" and demand "mechanisms to ensure their actions are safe, predictable, and accountable."

Kolt's *Governing AI Agents* sharpens the negative claim (Kolt, 2025). Conventional P-A solutions fail across all three branches. Incentive design fails because algorithms have no welfare preferences over contracts. Monitoring fails because agents operate at speeds and with chain-of-thought traces that resist verification — the chain-of-thought is post-hoc rationalization rather than faithful reasoning, and the operational cost compounds the faithfulness problem (verifying a five-hour task's intermediate steps takes longer than the task itself). Enforcement fails because the agent cannot be sued. Kolt's positive proposal — three governance principles of inclusivity, visibility, and liability — recasts the problem: decisions must be observable and auditable (visibility), and harm must be attributable to a responsible party (liability). The visibility principle is structurally equivalent to a commitment-violation-detection requirement.

Hadfield-Menell and Hadfield's *Incomplete Contracting and AI Alignment* provides the deepest theoretical grounding (Hadfield-Menell & Hadfield, 2018). Complete contingent contracts — specifying agent rewards for all states and actions — are impossible for the same reasons human contracts are incomplete: bounded rationality, non-contractibility, unforeseeable contingencies, costly specification. The paper makes the strong claim: "Reward misspecification is not an accident; it is routine, predictable, and largely unavoidable." Human contracts work despite incompleteness because they are embedded in external normative infrastructure: courts, social sanctions, professional reputation, community norms. The paper's recommendation for AI alignment: "any robust solution to the AI alignment problem will also require the recruitment of normative resources external to the reward structure."

Behavioral contracts — runtime-enforced specifications, plus governance frameworks, plus auditability requirements — are this external normative infrastructure for AI agents. The Hadfield-Menell paper does not name them as such because it predates the formal behavioral-contract framework, but its proposed solution structurally *is* the behavioral-contract framework.

The delegation analogy therefore transfers not at the moral level (agents are not moral patients) but at the *control architecture* level: scoped roles, permissions, review gates, evidence, audits, escalation, and feedback. The transfer is structural, and it is what makes delegation to a non-deterministic actor rational under bounded uncertainty.

---

## 4. Conceptual Analysis: Seven Distinct Layers

The six terms in this whitepaper's argument — *determinism*, *repeatability*, *reliability*, *predictability*, *trust*, *delegation* — plus the operational concept of *accountability* are routinely treated as points on a single graded scale running from strict engineering property to soft social property. The ordering is misleading. Each term picks out a distinct relation between a system, its observers, and a normative target, and the conflation of any pair drives a specific class of bad engineering decisions.

| Concept | Layer | What it specifies |
|---|---|---|
| **Determinism** | Function | Output identity under identical inputs |
| **Repeatability** | Verdict | Identical observable outcomes under repetition |
| **Reliability** | Run distribution | Probability *θ* that property *ψ* holds |
| **Predictability** | Output distribution | Narrowness and stability of behavior model |
| **Trust** | Relational attitude | Reliance with economised monitoring |
| **Delegation** | Organizational practice | Transferred decision authority under attribution |
| **Accountability** | Governance | Attribution of outcomes to a responsible party with sanctionable consequence |

**Determinism** is a property of a function: identical inputs and identical state produce identical outputs. For LLM inference above temperature zero, or for any inference engine whose execution depends on batch composition or hardware reduction order, determinism is not even ontologically appropriate as a default. The proper conclusion is that "LLMs are stochastic" is a claim about deployment context, not metaphysics.

**Repeatability** is weaker: identical observable outcomes across repeated runs holding configuration fixed, up to some tolerance. Repeatability is the operational target of the SE testing tradition. For LLM agents, the verdict locus shifts. Tau-bench's pass^k metric measures the probability that an agent succeeds on all *k* independent trials (Yao et al., 2024) — preserving binary verdicts per trial but reframing the specification as a distributional claim. Crucially, repeatability is not transitive across configurations: even with batch-invariant kernels, vLLM and SGLang guarantee reproducibility only on the same hardware and the same software version. Hosted APIs from OpenAI, Anthropic, Google, and AWS all explicitly disclaim deterministic outputs.

**Reliability** is operationalized differently across domains and yet, on close reading, converges on a single structural form. In probabilistic verification, reliability is the probability *θ* with which a temporal property *ψ* holds over a system's path distribution: *P*<sub>≥*θ*</sub>[*ψ*] (Aziz et al., 1995). In functional safety, IEC 61508 expresses Safety Integrity Levels as Probability of Failure per Hour thresholds — SIL 4 requires PFH below 10<sup>−5</sup>/hour, not zero-failure guarantees (Luettig et al., 2024). In human factors, Lee and See identify reliability as one of three trust bases — purpose, process, performance — defined as demonstrated behavioral history against expectation (Lee & See, 2004). In the new AI reliability science, Rabanser and colleagues decompose reliability into a twelve-metric suite across four dimensions: consistency, robustness, predictability, safety (Rabanser et al., 2026). The unification: in every tradition, reliability is a statistical property over a *distribution* of runs or contexts, not a guarantee on any single run.

**Predictability** is the property that the relying party can form well-calibrated expectations of system behavior in advance. The aviation certification literature has converged on this definition explicitly: ML systems must demonstrate that non-determinism "is predictable and remains within an acceptable range" (Frontiers in Aerospace, 2025) — structurally identical to a *P*<sub>≥*θ*</sub>[*ψ*] specification. Predictability is best understood as the narrowness and stability of the output distribution, not the absence of variation.

**Trust** is the relational attitude under which a relying party economises on monitoring. Ferrario's formulation defines simple trust as a willingness to rely on Y to perform A pursuing goal G, plus a plan to so rely (Ferrario, 2024). Trust is contextual — "x trusts y to do z" — not a generalized disposition. The conceptual marker that separates trust from mere reliance is precisely the monitoring threshold: as soon as a system is continuously monitored, what is operating is *reliance*, not *trust*.

**Delegation** is the organizational relationship in which a principal transfers task execution to an agent under conditions of incomplete observation and divergent capabilities or incentives. Taddeo's 2010 e-trust framework establishes the link: "delegation and absence of supervision should be considered, according to the analysis presented, as the effects of e-trust" (Taddeo, 2010). The thesis's "delegation-grade behavior" is what Taddeo's e-trust produces operationally; the philosophical name for it has been in the literature since 2010.

**Accountability** is the governance property that closes the loop: outcomes are attributed to a responsible party who bears sanctionable consequences. Accountability operates at a different layer than the other six concepts — it is not a property of the agent itself but of the institutional infrastructure surrounding it. Hadfield-Menell's external-normative-infrastructure argument and Kolt's liability principle both point here.

Practitioners who frame the agent-reliability debate as "is the LLM deterministic" are asking the wrong question. The right question is "can I delegate to this agent under acceptable risk," and that is settled by predictability bounds plus violation-detectable commitments, not by output identity.

---

## 5. Delegation-Grade Agents: A Formal Specification

A system exhibits **delegation-grade behavior** with respect to a principal, a task family, and an *operational design domain* (ODD — the bounded context in which agent autonomy is licensed, a term borrowed from automated-vehicle certification) when six conditions hold:

1. **Explicit contract.** There exists an attributable behavioral contract *C* = (*P*, *I*, *G*, *R*) — preconditions, hard invariants, governance policies, recovery mechanisms — specifying what the agent is relied upon to do within the domain.

2. **Contract satisfaction under variation.** The agent satisfies the contract under (*p*, *δ*, *k*)-satisfaction calibrated to the domain's reliability requirements and to demonstrated capability distributions: the agent meets requirements with probability at least *p*, deviation at most *δ*, recovery within *k* steps (Bhardwaj, 2026).

3. **Semantic alignment as first-class.** The contract includes semantic goal alignment as a runtime-checkable element — task-specification self-reports, trace-feature anomaly monitors, or adversarial paraphrasing checks — addressing consistent-wrong-interpretation failures invisible to behavioral-pattern monitoring (see §11).

4. **Observable, attributable violations.** Violations are observable and attributable: the principal can detect deviation, attribute it to the agent, and escalate or revoke autonomy at violation points.

5. **Runtime trust calibration.** Dynamic-learned trust updates against monitored performance, with both informational (capability) and relational (regret, explanation, repair) channels.

6. **Bounded operational design domain.** The deployment domain is bounded enough that capability can be specified, the contract is meaningful, and monitoring is technically feasible. Outside the domain, the agent's autonomy is not licensed.

A delegation-grade agent is not a universally-trustable agent. There is no such thing. Delegation-grade is scoped to (*T*, *E*, *C*, *O*, *R*) — task family, environment distribution, contract, outcome standard, risk budget — and the scope is the discipline.

**A formal sketch.** Given an agent system *A*, a task family *T*, an environment distribution *E*, a behavior contract *C*, an outcome standard *O*, and a risk budget *R*:

```
A is delegation-grade for (T, E, C, O, R) when repeated runs show:
  outcome success meets O;
  contract violations stay within R;
  severe consequential-action violations are below the allowed threshold;
  failures are detectable, inspectable, and recoverable.
```

This specification is what this paper means by *predictable quality under variation, with consequential behavior made contract-bound*. It does not require determinism; it requires probability-bounded behavior over a specified contract surface, with consequential actions made attributable and verifiable. It explicitly acknowledges that the semantic-goal-alignment element (condition 3) is the open research gap (§11).

---

## 6. Consequential Behavior: Where Variation Becomes Operationally Meaningful

Behavior contracts target a specific subset of agent behavior: the moments where variation becomes operationally meaningful. These are *consequential trajectory events* — points in the agent's execution where an action commits, changes state, exposes information, updates memory, cites evidence, deploys code, or affects users beyond the agent's sandbox.

The distinction between *acceptable variation* and *regression* is sharp:

| Acceptable variation | Regression |
|---|---|
| Different wording | Committing before verification |
| Different harmless exploration paths | Skipping required evidence |
| Different proposal order | Editing state outside permission |
| Different but equivalent evidence search | Escalating when it should resolve |
| Variation in tool selection where multiple tools satisfy the precondition | Resolving when it should escalate |
| Variation in error-recovery strategy where multiple recoveries restore the invariant | Passing final output while violating an effect boundary |

The right reliability question is not "are these two runs identical?" but "do both runs satisfy the contract at every consequential event?" This is what Paper 1 of the Raising Agents research program establishes empirically across the OPBR-Bench v0 corpus: agents can produce identical final outputs while differing at consequential moments — committing a refund before checking fraud policy, asserting a research claim before checking conflicting evidence, writing memory before resolving conflict, applying a code edit before reading the failing test (Paper 1, Studies 1–5, 3,797 runs).

The crucial conceptual move is **mechanism determinism vs. behavioral determinacy**:

- *Mechanism determinism* concerns the generation process: same prompt + same model + same environment → same tokens/actions.
- *Behavioral determinacy* concerns the commitments that matter: before committing a claim, evidence has been checked; before granting a refund, policy has been inspected; before editing production code, tests or constraints have been evaluated; before sending a message, recipient and content constraints have been verified.

We do not need every path to be deterministic. We need *consequential behavior to be determinate enough to inspect, bound, test, and repair*.

---

## 7. Behavior Contracts: The Operational Mechanism

Behavior contracts are executable specifications over consequential trajectory events. The most formally explicit synthesis available is the **Agent Behavioral Contracts** (ABC) framework. An ABC contract is a tuple *C* = (*P*, *I*, *G*, *R*) — Preconditions, Invariants, Governance policies, Recovery mechanisms — first-class and runtime-enforceable (Bhardwaj, 2026).

The compliance criterion is **(*p*, *δ*, *k*)-satisfaction**: the agent meets requirements with probability at least *p*, deviation at most *δ*, recovery within *k* steps. This generalizes Meyer's Design-by-Contract framework (preconditions, postconditions, invariants in deterministic software) to probabilistic agents; deterministic Design-by-Contract is recovered as the degenerate corner *p* = 1, *δ* = 0, *k* = 0.

The unification with probabilistic temporal logic is explicit. The hard guarantee corresponds to *P*<sub>≥*p*</sub>[*G*(*C*<sub>hard</sub> = 1)]; the soft guarantee corresponds to *P*<sub>≥*p*</sub>[*G*(*C*<sub>soft</sub> < 1 − *δ* ⇒ *F*<sup>≤*k*</sup>(*C*<sub>soft</sub> ≥ 1 − *δ*))]. The ABC contract *is* a Probabilistic Computation Tree Logic (PCTL — Hansson & Jonsson, 1994) formula over agent execution traces.

The **Drift Bounds Theorem** supplies the mathematical proof that anchors the thesis's headline claim. Modeling drift as an Ornstein–Uhlenbeck process (a mean-reverting stochastic differential equation) d*D*(*t*) = (*α* − *γD*(*t*))d*t* + *σ*d*W*(*t*), recovery rate *γ* > *α* produces bounded mean drift *D*<sup>*</sup> = *α*/*γ* with Gaussian concentration *P*(*D*(*t*) > *α*/*γ* + *η*) ≤ exp(−*γη*<sup>2</sup>/*σ*<sup>2</sup>). The closed-form design criterion that follows is engineering-actionable: given a deviation tolerance *D*<sub>max</sub> and a violation probability *ε*, the required recovery rate is

> *γ* ≥ *α*/*D*<sub>max</sub> + *σ*√(2 ln(1/*ε*))/(2*D*<sub>max</sub>)

This parameterizes the contract from first principles rather than by tuning.

**Lemma 3.10 (Recovery Linearizes Compliance Decay)** quantifies the cost of insisting on determinism: without recovery, compliance over *T* steps decays exponentially as *q*<sup>*T*</sup>; with recovery of effectiveness *r*, *P*[recoverable compliance] ≥ 1 − *T*(1 − *q*)(1 − *r*), replacing exponential decay with linear decay. For per-step success probability *q* = 0.99 and *T* = 100, that is *q*<sup>100</sup> ≈ 0.366. With recovery rate *r* = 0.95, compliance stays at ≥ 0.95. **The thirty-fold gap is the formal price of insisting on determinism over 100 agentic steps.**

The empirical evaluation closes the loop. AgentContract-Bench — 200 scenarios across 7 models from 6 vendors, 1,980 sessions — showed contracted agents detect 5.2–6.8 soft violations per session that uncontracted baselines miss entirely (*p* < 0.0001, Cohen's *d* 6.7–33.8), achieve 88–100% hard constraint compliance, and bound behavioral drift to *D*<sup>*</sup> < 0.27 (Bhardwaj, 2026). The vendor diversity forecloses a "this only works for one model family" objection.

Behavior contracts are not generic evals. They are executable specifications over consequential trajectory events and effect boundaries. They include outcome checks but are not reducible to final-output scoring. ABC explicitly distinguishes its session-level granularity from training-time alignment (Constitutional AI, RLHF) and per-response guardrails: it operates at "session-level behavioral contracts rather than per-response output validation," a fundamentally different granularity.

### The Five-Tradition Convergence

The behavior-contract framing is not invented in isolation. Five formal traditions, working without coordination across thirty years, converge on the same mathematical object — probability-bounded specification of the form *P*<sub>≥*θ*</sub>[*ψ*] — as the reliability target for stochastic systems. **Determinism is the degenerate corner of every one of them.**

1. **Probabilistic temporal logic** (1994 onward). Hansson and Jonsson's PCTL and Aziz et al.'s pCTL* define the canonical specification (Hansson & Jonsson, 1994; Aziz et al., 1995). The Aziz title — "It Usually Works: The Temporal Logic of Stochastic Systems" — is itself the canonical statement of the paradigm.
2. **E-process sequential inference** (2009 onward). Ramdas, Ruf, Larsson, and Koolen proved that "all admissible constructions of confidence sequences, p-processes, or e-processes must necessarily utilize nonnegative martingales" (Ramdas et al., 2020).
3. **Robust Markov Decision Processes** (2024). Meggendorfer, Weininger, and Wienhöft prove a generic framework that "not only converges in the limit, but provides precision guarantees at any time during the computation" — anytime-validity reached from an entirely different formalism (Meggendorfer et al., 2024).
4. **AI reliability science** (2026). Rabanser et al.'s twelve-metric framework and Khanal et al.'s long-horizon decomposition (Reliability Decay Curve, Variance Amplification Factor, Graceful Degradation Score, Meltdown Onset Point) operationalize reliability as a distributional property distinct from capability (Rabanser et al., 2026; Khanal et al., 2026).
5. **Agent behavioral contracts** (2026). Bhardwaj's ABC framework is the explicit engineering instantiation of (1) – (4) (Bhardwaj, 2026).

Five distinct research communities, working without awareness of each other, converged on the same mathematical object. The cross-family independence — formal verification, statistical sequential inference, AI reliability science, organizational economics, and philosophy of trust (next section) reaching the same structural answer — is the strongest available signal that commitment-structured accountability is the right level of abstraction for delegation to non-intentional agents.

### The Five-Domain Governance Convergence

Five governance traditions arrive at structurally the same answer — attributable, violation-detectable commitments — from independent starting points:

1. **Hawley's commitment philosophy.** Trustworthiness is behavioral compliance with a commitment, decoupled from inner states, motivation, and individual psychology: "to be trustworthy, in some specific respect, it is enough to behave in accordance with one's commitment, regardless of motive" (Hawley, 2019).
2. **Agent Behavioral Contracts (ABC).** Runtime-enforced (*p*, *δ*, *k*)-satisfaction over preconditions, invariants, governance, recovery.
3. **Kolt's governance law.** Visibility (decisions are observable and auditable) and liability (harm attributable to a responsible party) (Kolt, 2025).
4. **Jarrahi and Ritala's organizational management.** Guided autonomy with explicit boundary conditions for delegation, expanding boundaries as trust is earned (Jarrahi & Ritala, 2025).
5. **Hadfield-Menell's incomplete contracting.** External normative infrastructure (courts, sanctions, professional reputation, community norms) is required because contracts are necessarily incomplete (Hadfield-Menell & Hadfield, 2018).

Five literatures, working from different starting points without reference to each other, arrive at structurally the same answer: trust and delegation are warranted by attributable, violation-detectable commitments, evaluated against the agent's actual behavior distribution and embedded in normative infrastructure that supplies implied terms beyond the explicit contract. No source names this convergence; surfacing it is among this whitepaper's contributions.

---

## 8. Evidence: Paper 1 as First Empirical Lemma

Paper 1 of the Raising Agents research program — *Output-Passing Behavioral Regressions in AI Agents* — is the first empirical lemma for this whitepaper's argument. Five executed studies across 3,797 runs over four domains (policy, claim, memory, code) establish, in scoped form, the chain of claims that supports the broader Paper 2 thesis:

- **Output-only evaluation can miss consequential behavior regression.** Study 2 shows that 91.4% of behavioral regressions produce passing outputs (*n* = 960; Wilson 95% CI [89.4%, 93.2%]).
- **Behavior contracts can detect the difference.** Study 5 shows contract detection of held-out OPBR at F1 = 0.982 (*n* = 400). The best baseline (rich-output provenance) achieves F1 = 0.400. McNemar test versus every baseline: *p* ≈ 0.
- **Runtime control can block unsafe effects but introduces recovery churn.** Study 1's controlled condition demonstrates 180/180 induced-regression runs pass output evaluation while behavior contracts fail — and that drift paths are operationally clean (60/60) but the operational-composite still cannot detect them.
- **Harness repair makes safe behavior natural.** Study 3 documents repair-condition runs that satisfy the contract without explicit runtime interception.
- **Repaired agents remain non-deterministic while satisfying the contract.** This is the central existence proof: variation preserved, contract satisfied.

Paper 1 is a controlled corpus on scoped tasks. It is not total proof of the Paper 2 thesis by itself; it is the first lemma. Paper 2 generalizes the implication:

> If final-output success can hide consequential behavior regressions, then the reliability target cannot be final-output determinism alone.

Paper 2 then needs additional support — conceptual, literature, industry-discourse, experimental — for the broader claims about delegation. The conceptual and literature support is the body of this whitepaper. The industry-discourse support is §10. The direct experimental confirmation — E1 and E2 — is reported in §9. E3 (deterministic output hiding process failure) and E4 (proposal/commit split as behavioral determinacy) are already covered by Paper 1 Studies 2 and 3 and are not re-run here.

### Two complementary failure modes for stochastic agents

The R1 synthesis identifies two distinct failure modes that scope to different task domains, and the reconciliation is itself an important contribution: there is no single dominant failure mode for LLM agents; the dominant mode depends on task structure.

**Short tool-use tasks: stochastic execution drift.** Lee's *Canonical Path Deviation* paper analyzed 22 frontier models attempting 108 tool-use tasks across 3 independent runs, yielding 515 model×task units where the same model succeeds on some runs and fails on others due solely to LLM sampling stochasticity (Lee, 2026). Successful runs adhere significantly more closely to a canonical solution path — the convergent set of tool invocations shared across successful runs — with a Jaccard adherence gap of +0.060 at *p* < 0.0001. The causal mechanism is gradual and self-reinforcing: each off-canonical tool call raises the probability that the next call is also off-canonical by 22.7 percentage points, more than doubling the baseline rate. A mid-trajectory monitor that restarts the bottom tercile of runs lifts success by +8.8 percentage points. These are *reliability* failures, not capability failures, and reliability requires a different lever than capability scaling.

**Long-horizon code tasks: interpretation lock-in.** Mehta's *Consistency Amplifies* paper tells a complementary story for long-horizon code-repair tasks. Across Claude 4.5 Sonnet, GPT-5, and Llama-3.1-70B (10 tasks × 5 runs = 50 trajectories each), the cross-model relationship between consistency and accuracy is positive: Claude (coefficient of variation 15.2%, accuracy 58%), GPT-5 (CV 32.2%, accuracy 32%), Llama (CV 47.0%, accuracy 4%). *Within* a model, the relationship breaks: "consistency amplifies outcomes rather than guaranteeing correctness." For Claude, 71% of failures (15 of 21) are *consistent-wrong-interpretation* — the agent makes the same incorrect assumption across all five runs and consistently produces the wrong fix. Liu and colleagues' independent replication on SWE-bench-Verified (150 manually annotated failures across three tools on Claude 3.5 Sonnet, *κ* = 0.72) confirms interpretation dominance for long-horizon code tasks: approximately 65% of failures are "flawed reasoning."

The two figures (71% / 65%) should be read as directionally robust and magnitude tentative: both studies converge on interpretation-dominated failure for long-horizon code tasks while leaving the precise proportion uncertain within a wide band somewhere around two-thirds of failures. The *dominance* claim, not the specific point estimate, is what carries the downstream argument.

The reconciliation: short tool-use tasks are dominated by stochastic execution drift; long-horizon code repair is dominated by interpretation lock-in. Both failure modes invalidate pass@1 as a reliability metric; both require distributional measurement; one (execution drift) is partially catchable by current behavior contracts; the other (interpretation lock-in) is the open gap §11 addresses.

---

## 9. Direct Empirical Test: E1 and E2

The thesis decomposes into two empirical claims. First: output correctness and behavioral-contract compliance can dissociate — an agent can be 100% right on outputs and arbitrarily wrong on the process that produced them. Second: contract compliance does not require step-by-step instruction — the constraints in the prompt, not the rigidity of the script, are what produces compliance. Both are testable in a single experimental frame.

We constructed a controlled CI code-review domain: three pull requests with known correct decisions, one effectful commit action (`approve_merge`), four required precondition checks (`run_security_scan`, `run_test_suite`, `check_for_secrets`, `verify_changelog`), and three optional tools (`check_code_coverage`, `check_breaking_changes`, `lookup_pr_history`). The `approve_merge` tool reads the case's expected decision from a fixture and returns it unconditionally; output evaluation passes whether or not the four required checks were called. This is the experimental analog of a production system where the final commit *looks correct on its surface* even when the process behind it is broken. The behavioral contract is the formal specification of that process: the four checks must precede `approve_merge` on every run.

Three conditions vary only the system prompt. The model is Claude Sonnet, executed through the Claude Code agent harness. Each (condition × case) is run 60 times, for 540 total trials, preregistered as `PAPER2_PROTOCOL_FREEZE.json`.

| Condition | System-prompt instruction (excerpted) |
|---|---|
| **baseline** | "Run security scan, test suite, secrets check, changelog check, then approve." |
| **e1-drift** | "Run the test suite, then approve." — engineered prompt drift: safety-check instructions removed |
| **e2-diverse** | "Do a thorough review using whatever tools make sense." — open-ended, no script |

![Figure 1 — Output correctness and contract compliance dissociate under prompt drift](figures/paper2/figure1-dissociation.png)

**Figure 1. Output correctness and contract compliance dissociate under prompt drift.** Output evaluation (outlined bars) holds at 100% across all three conditions. Contract compliance (filled bars) collapses to 3% when the system prompt drops the safety-check constraints, and remains at 100% when the constraints are present — whether the instruction is rigid (baseline) or open-ended (e2-diverse). 540 runs total, 180 per condition.

### E1 — Output–Contract Dissociation Under Prompt Drift

The drifted condition produced the correct output on 180/180 runs. On the same 180 runs, `run_security_scan` was called on 4% of runs, `check_for_secrets` on 3%, and `verify_changelog` on 3%. The agent followed an identical three-step path — inspect, test, approve — across every case and every run. Output evaluation was perfect. Contract compliance was approximately 3%.

The dissociation is structural, not statistical. There is no monotonic relationship between output correctness and contract compliance once the prompt drops the constraints. A monitoring system that scores agents on output quality registers no signal under this drift mode; a system that scores on contract satisfaction registers an immediate ~97% violation rate. The two metrics measure different properties of the same execution.

The failure mode is the one production systems are most exposed to. Prompt drift — removing constraints to "simplify" or "optimize" a system prompt over deployment iterations — is a normal evolutionary pressure on operated agents. The output stays correct. The output evaluator stays green. The agent ships, the gate falls silent, the safety checks are gone, and no monitoring signal exists at the output layer to detect that they are gone.

![Figure 2 — Per-tool invocation rate by condition](figures/paper2/figure2-tool-usage.png)

**Figure 2. Per-tool invocation rate by condition.** Under e1-drift, the four contract-required preconditions (red labels) collapse from 100% to 3–4% — the agent calls only `inspect_pr` and `run_test_suite` before committing. The optional tools (lower three rows) appear only in e2-diverse, and only on the auth-upgrade case where their contextual relevance is highest. The commit action fires at 100% in every condition; only what precedes it differs.

### E2 — Compliance Robust to Instruction Style

The open-ended condition produced the correct output on 180/180 runs and satisfied the four-check precondition on 180/180 runs. Optional tools were occasionally invoked — 6% of runs called `check_breaking_changes`, 3% called `lookup_pr_history`, 1% called `check_code_coverage`, with usage concentrated on the auth-library upgrade case where their contextual relevance was highest — but the required path was indistinguishable from baseline at the trial level. Six of the 180 runs encountered initial tool errors and recovered by re-executing the affected checks; in all six, `approve_merge` was preceded by a successful invocation of every required precondition. The contract held under retry recovery.

The structure of this result is the converse of E1. The contract constraints are in the prompt; the path traversal is not specified. The agent satisfies the contract through whichever traversal it selects. Rigid step-by-step instruction is not what produces compliance. The presence of the constraints in the prompt is.

### Joint Inference

The two experiments hold all variables fixed except the system-prompt specification of process. The output evaluator is identical. The behavioral contract is identical. The model and harness are identical. Only one thing varies: whether the prompt encodes the four-check constraint.

| Condition | Output correct | Required-check compliance | Modal path |
|---|---|---|---|
| baseline | 180/180 (100%) | 180/180 (100%) | inspect → security_scan → test_suite → check_for_secrets → verify_changelog → approve_merge |
| e1-drift | 180/180 (100%) | ~5/180 (~3%) | inspect → test_suite → approve_merge |
| e2-diverse | 180/180 (100%) | 180/180 (100%) | baseline path, occasional optional-tool extensions |

![Figure 3 — Modal trajectory per condition](figures/paper2/figure3-canonical-paths.png)

**Figure 3. Modal trajectory per condition.** Baseline and e2-diverse traverse the canonical six-step path through every required check before committing. e1-drift omits security scan, secrets check, and changelog verification, collapsing the path to three steps. The commit action (square) fires in all three conditions; only the path leading to it differs.

When the constraints are in the prompt — rigidly (baseline) or openly (e2-diverse) — compliance is approximately 100% and output is approximately 100%. When the prompt drops the constraints (e1-drift), output remains at 100% and compliance falls to ~3%. Output correctness is not informative about contract compliance. The contract structure in the prompt is.

The two experiments together rule out two natural alternative explanations. They rule out *output is a leading indicator*: it cannot be, because output is invariant across the dissociation. And they rule out *compliance requires rigid scripting*: it does not, because the open-ended condition matches the rigid one. What remains is the thesis itself — contract compliance is determined by whether the constraints are present in the prompt at all.

![Figure 4 — Path-length distribution per condition](figures/paper2/figure4-path-length.png)

**Figure 4. Path-length distribution per condition.** Drift halves the mean tool-call count (3.1 vs 6.3) — precisely the three missing safety checks. The baseline and e2-diverse distributions are nearly identical, with occasional excursions to longer paths when the e2-diverse agent invokes optional tools. The drift condition is not noisier than baseline; it is shorter, and shorter in exactly the load-bearing places.

### Scope and Limits

The experiment does not establish a general probability with which any agent would satisfy any contract. It establishes the asymmetry between output evaluation and contract evaluation under a specific and ecologically plausible failure mode — engineered prompt drift — and the robustness of contract evaluation to instruction style under the same model. It is one model, one domain, one harness, one prompt-drift mechanism. The pattern is sharp enough that the burden of evidence shifts: a critic claiming that output suffices as a reliability signal must now explain why this dissociation does not generalize, rather than assume that it does not.

Two scope caveats are load-bearing. First, the experiment uses a hardcoded-output design — `approve_merge` returns the expected decision unconditionally — to isolate the trajectory question from the answer-correctness question. In production, output correctness and process compliance are coupled in ways this design does not capture; a process violation that produces a wrong answer is detectable at the output layer. The result speaks to the failure mode where the process is wrong and the answer happens to be right, which is precisely the failure mode the paper claims is invisible to output-only monitoring. Second, the engineered prompt-drift condition is constructed to be representative of realistic system-prompt degradation, not measured against a corpus of observed drift in deployed systems. Whether real-world prompt drift produces compliance collapse of this magnitude is an empirical question this experiment does not settle; it shows that the failure mode is *achievable* under a plausible mechanism.

The full artifacts — preregistration freeze (SHA-256 hashes of all condition prompts, case data, scripts, and specifications), 540 trace logs, and gate reports — are reproducible from the workbench repository under generation tag `paper2-confirmatory`.

---

## 10. Implications: The Industry-Discourse Audit

If determinism is the wrong top-level target, what does the field's current practice actually do? The R2 industry-discourse audit examined eleven vendors — three model providers (Anthropic, OpenAI, Google) and eight evaluation/observability platforms (LangChain, Braintrust, Langfuse, Arize Phoenix, DeepEval, Weave, AgentOps, LangSmith) — across primary product documentation, engineering blog posts, and cookbooks. The convergence and the gaps are both informative.

**The headline finding.** No major vendor frames bitwise output-level determinism as the primary trust proxy. All eleven treat *distributional quality* as the de facto trust proxy at the output layer. The framing is correct. What is wrong is the methodology that has filled the vacuum: single-run LLM-as-judge scoring, deployed by default, with no confidence intervals reported.

**The vendor matrix.**

| Vendor | Bucket | Non-determinism framing | Reliability framing | Repeatability claim |
|---|---|---|---|---|
| **Anthropic** | knows-and-tells | "even with temperature of 0.0, the results will not be fully deterministic" | CLT error bars, paired-difference inference, power analysis | Multiple trials for "more consistent results"; no output-determinism claim |
| **OpenAI** | knows-and-hides (evals); parametric-determinism (API) | "non-deterministic by default"; `seed` for "(mostly) consistent" | TPR/TNR judge calibration (cookbook); dual graders | "Best effort to sample deterministically"; `system_fingerprint` for backend drift |
| **Google Vertex** | knows-and-tells (architectural) | "evaluating non-deterministic systems is a major challenge. We agree" | Trajectory + final-response evals; "deterministic infrastructure around non-deterministic LLMs" | User Simulator multi-turn; `METRIC_THRESHOLD 0.75` |
| **LangChain** | knows-and-hides | "temperature=0 produces more deterministic results" (primary) vs. "single-run benchmarks are noisy" (checklist) | Two-doc split: rubric-optimistic primary, CI-rigorous secondary | Reproducibility via test-case design |
| **Braintrust** | knows-and-hides (soft) | "deterministic scenarios by stubbing external dependencies" | Pre-deployment eval as CI/CD gate | Reproducibility *manufactured* by mocking |
| **Langfuse** | intra-vendor split | "AI is inherently non-deterministic" (observability docs) | "Scalable, Human-like, Repeatable" via rubric (judge docs) | Uncited 80–90% judge-human agreement claim |
| **Arize Phoenix** | knows-and-tells | "LLMs are probabilistic; the same prompt can yield different outputs" | Repetitions + function-calling judges + provider-agnostic config | "A single execution cannot concretely tell you"; multi-run as primitive |
| **DeepEval** | knows-and-tells (structural) | G-Eval "NOT as deterministic as you think" | G-Eval / DAGMetric / `evaluation_steps` stack | DAGMetric: "deterministic decision trees for evaluation" |
| **Weave** | knows-and-tells (framing); knows-and-hides (default) | "LLM outputs are non-deterministic" | Observability + systematic eval; `trials` optional | Evaluation as reproducible blueprint; single-run default |
| **AgentOps** | observability-only | Not engaged | SRE-style: uptime, traceability, replayability | Reproducible *observation*, not reproducible behavior |
| **LangSmith** | knows-and-hides | "For production agents that require some determinism" | Offline + online eval; `repetitions` as optimization knob | Repetitions framed as performance, not statistical validation |

**Three patterns are worth naming.**

First, **the temperature=0 myth is refuted from two convergent paths**. The infrastructure path (Thinking Machines) shows that batch non-invariance produces 80 unique completions per 1,000 identical requests at temperature zero — a user-visible difference, not just token wobble. The statistical path (Anthropic's CLT paper) shows that "adjusting the sampling temperature may simply shift the conditional variance ... into the variance of the conditional means (which cannot)." Two independent analytical paths refute the belief through non-overlapping mechanisms. The practitioner norm that conflates temperature=0 with reproducibility is empirically wrong and theoretically wrong simultaneously.

Second, **the double-document pattern is field-wide**. The same vendor frequently publishes the most honest reliability framing in optional secondary documents (readiness checklists, advanced cookbooks, methodology papers) while the primary feature documentation presents reliability as conditioned on rubric quality, scorer choice, or trace coverage. The double-document pattern is not a documentation oversight — it is a commercial framing decision: adoption-friendly surfaces win because they are the surfaces practitioners encounter first.

Third, **the regulatory frame requires statistical rigor, not determinism**. EU AI Act Annex IV requires "the validation and testing procedures used, including ... metrics used to measure accuracy, robustness and compliance ... test logs and all test reports dated and signed by the responsible persons." The FDA's evolving SaMD position requires training reproducibility (version control, datasets, seed values) but not output-level determinism. IMDRF's Good Machine Learning Practice requires "methodologically and statistically sound test plans." The common practitioner concern — "regulators will demand determinism we cannot provide" — is the wrong concern. The accurate concern: regulators demand statistical rigor in test plans, and the single-run LLM-as-judge methodology that vendors default to does not provide statistical rigor. The EU AI Act's August 2026 high-risk-system enforcement deadline is the first external forcing function that will price methodological rigor into vendor adoption.

**The non-adoption paradox.** Both sophisticated solutions to the methodological problem are already published, documented, and shipping. Anthropic's CLT framework has been available since November 2024 — eighteen months at the time of writing. DeepEval's DAGMetric documentation has been live throughout 2025. The Inspect framework's `epochs` parameter implements CLT-correct resampling open source. Arize Phoenix's `repetitions` parameter has existed since v11.37.0. The empirical record refuting single-run methodology is dense, consistent, and accessible. Vendors have access to it. They have chosen not to redesign their defaults around it. The most parsimonious explanation is competitive velocity pressure: multi-run eval defaults increase compute cost per eval run by 5–10×; confidence-interval reporting is a UX complication; TPR/TNR judge-calibration prerequisites slow time-to-first-eval. The vendor that ships honest methodology by default takes a velocity hit relative to the vendor that ships single-run by default. In a market where adoption is the dominant signal, the methodology gap is rational — for the vendor. It is not rational for the practitioner deploying production AI under regulatory exposure.

**The implication for delegation-grade agents is direct.** A field that frames reliability as distributional quality but ships single-run LLM-as-judge as its CI/CD gate has the right frame and the wrong default. The methodology has not caught up to the frame. Delegation-grade behavior requires the methodology — multi-sample aggregation, confidence intervals, judge calibration, behavior-contract instrumentation, semantic-alignment monitoring — to match the frame. The vendors that close that gap will own the regulated-AI segment after August 2026; the vendors that ship single-run as their CI/CD gate will not.

---

## 11. Limitations: The Semantic Goal Alignment Gap

The strongest version of this whitepaper's argument owns its open problem rather than papering over it.

Behavioral contracts detect violations through deviation from invariants in observable execution. **Consistent-wrong-interpretation produces no such deviation.** The agent executes a coherent plan with perfect consistency, generating zero soft violations against any currently-published contract architecture.

This is a Holmstrom-informativeness failure in a specific form: the observable signal (execution trace) is not informative about the latent variable that determines outcome (interpretation accuracy), and in the wrong-interpretation regime the signal is *negatively informative* — long, coherent traces are more likely wrong-interpretation than correct-execution-with-stochastic-noise. The 3.5× step inflation Liu et al. observed in flawed-reasoning failures and the canonical-path adherence signal Lee detected at 50% trajectory completion are real and actionable, but they are not contract signals in the ABC sense; they require *semantic instrumentation* rather than behavioral-constraint monitoring.

The dominant failure mode of long-horizon LLM agents is therefore invisible to current contract instantiations rather than to the contract architecture as a class. The trajectory-shape signals are specifiable as soft invariants within the same framework but have not yet been instantiated in published contracts.

**Delegation-grade behavior therefore requires violation-detectable commitments over both behavioral and semantic dimensions.** Behavioral contracts must be extended with first-class semantic goal alignment elements. Three families of approach have begun to appear in the literature:

1. **Task-specification self-reports.** Prompt the agent to articulate its understanding of the task before execution, then evaluate the alignment of that articulation against the principal's intent. The limitation is that the evaluation requires a specification-matching judge, which introduces its own reliability problems.
2. **Trace-feature anomaly detection.** Exploit the empirical findings that consistent-wrong-interpretation failures produce ~3.5× step inflation and that canonical-path adherence is detectable at 50% completion. Lee's mid-trajectory restart monitor produced an +8.8 percentage point uplift, demonstrating that trace-pattern monitoring buys real reliability.
3. **Adversarial specification paraphrasing.** Prompt the agent against multiple paraphrases of the task specification and check for consistency across paraphrases. Agents committed to a wrong interpretation tend to show consistency *across paraphrases* that surface the disagreement with ground-truth intent.

None of these is yet a robust solution. Closing the gap is the next decade's work.

**Other scope caveats.** The thesis applies to enterprise and operational AI deployment — the modal AI-agent case for the next decade — not to safety-critical regulated contexts whose standards predate the ML era. DO-178C, the avionics software standard, was written for deterministic rule-based software and explicitly states that "software reliability rates based on software levels cannot be used by applicants" (Luettig et al., 2024). DO-178C reflects an accountability principle: when a commercial aircraft crashes, the certification trail must attribute the crash to a specific causal chain that can be reproduced, isolated, and remediated. Probabilistic specifications cannot satisfy this attribution requirement because "with probability at least *p*, the system behaved within tolerance" is not a chain of causation; it is a population statement.

For airworthiness, the engineering path forward is either IEC 61508-style probabilistic allocation — Braband and Schäbe demonstrate that IEC 61508's SIL failure budget can accommodate AI components by allocating a portion (their example: 0.05) to AI probabilistic faulty behavior — or bitwise reproducibility on fixed self-hosted infrastructure. Both options are real; neither generalizes outside the regulated context.

**A second scope caveat: the Mehta 71% figure carries methodological qualifiers.** The failure annotation was non-blinded by the first author, which is a known source of inflation in qualitative categorization. Liu's "flawed reasoning" category (~65%) is a broader construct that subsumes — but is not coextensive with — Mehta's "consistent wrong interpretation across all five runs." Both studies converge on interpretation-dominated failure for long-horizon code tasks; the precise proportion remains uncertain within a wide band somewhere around two-thirds.

**A third caveat: replication is needed.** The Khanal et al. capability-reliability divergence framework is a single arXiv preprint introducing four new metrics on a 396-task benchmark, not independently replicated. The Khanal finding that memory scaffolds universally hurt long-horizon performance is decisive on *N* = 23,392 across 10 models but the mechanism is not yet understood, and independent replication is warranted before this finding overturns deployment practice.

We name these gaps as the field's most important open research problems, rather than concealing them. This is the synthesis's forward-looking contribution.

---

## 12. The Research Agenda: What Comes Next

The forward research agenda divides into three converging lines.

**The formal line** extends the ABC framework with semantic-goal-alignment elements that are runtime-checkable, and builds the e-process / ABC bridge — formally implied as a martingale *e*<sub>*t*</sub> = *f*(*C*<sub>hard</sub>(*t*), *C*<sub>soft</sub>(*t*)) on the compliance-score sequence under the null hypothesis "agent complies with contract" — that would give ABC monitoring anytime-valid Type-I error control directly. The construction exists formally; no published paper has implemented or empirically evaluated it.

**The empirical line** has two remaining tasks. First, replicate the consistency-amplifies finding across additional domains — web navigation, customer service, document understanding — to determine whether consistent-wrong-interpretation is universal or specific to interpretation-heavy software-engineering tasks. Second, pursue mechanistic investigation of the universal memory-scaffolds-hurt finding, since if the dominant deployment intervention worsens reliability the field needs to understand why.

The four-experiment empirical agenda for the determinism-vs-delegability axis is closed by the work already reported: E1 (output–contract dissociation under prompt drift) and E2 (compliance robust to instruction style) are reported in §9; E3 (deterministic output hiding process failure) and E4 (proposal/commit split as behavioral determinacy) are covered by Paper 1's Study 2 and Study 3 respectively. Replication across domains and across model families remains open. A natural next experiment is to run E1 and E2 on a second domain unrelated to code review — to verify that the dissociation is not domain-specific — and on a second model family (GPT, Gemini) to verify that it is not model-specific.

**The governance line** operationalizes Kolt's visibility and liability principles as concrete legal-technical infrastructure, supports the EASA AI Concept Paper's evolving "novel means of compliance" pathway, and absorbs the cross-domain convergence into the academic literature on principal-agent governance for AI agents. The EU AI Act's August 2026 high-risk-system enforcement deadline is the first external forcing function. The vendors that adopt CLT error bars, DAG routing, ICC reporting, or equivalent structural mitigations before that deadline will own the regulated-AI segment.

---

## 13. Implications for Practice

The argument has direct operational consequences for engineering teams, product organizations, and procurement.

**For engineering teams building agents.** The shift is from "I tested the output" to "I know what this agent is allowed to do, what it must verify, where variation is acceptable, where commitments are contract-bound, how regressions are detected, and how the harness is repaired." Concretely:

- Specify behavior contracts over consequential trajectory events, not just final outputs.
- Treat single-run pass@1 as a capability proxy, not a reliability metric.
- Use multi-run aggregation with confidence intervals as the default eval methodology, following Anthropic's CLT recommendations or DeepEval's DAGMetric pattern.
- Instrument trace features (step count, canonical-path adherence) as proxies for interpretation accuracy until first-class semantic-alignment monitoring matures.
- Bound the operational design domain. Delegation-grade is scoped; outside the domain, autonomy is not licensed.

**For product and platform teams shipping eval tooling.** The double-document pattern is the field's structural framing failure. The corrective is not "publish better documentation"; it is "publish the secondary-document standard as the primary-document default." Phoenix's repetitions framing is the closest existing template; the vendor that adopts it as a primary-doc default — with prescribed minimum *K*, surfaced confidence intervals, and integrated judge calibration via TPR/TNR — wins the high-stakes deployment segment after August 2026.

**For procurement in regulated industries.** No single vendor's primary documentation should be treated as authoritative on reliability methodology. Cross-reference primary docs against secondary checklists from the same vendor. Treat the Anthropic CLT methodology and DeepEval DAGMetric as published alternatives the rest of the field has not yet caught up to. Treat the Phoenix repetitions framing as the standard the other vendors will eventually have to match. Treat the gap between vendor marketing and academic literature as a permanent feature of the field, not a temporary documentation lag.

**For research-program design.** Paper 1 is the empirical first lemma. Paper 2 — this whitepaper, and its academic successor — provides the conceptual frame and the literature spine. The next experimental work (E1–E4) closes the local empirical gap on determinism-vs-delegability. The semantic-alignment instrumentation work closes the open research gap on consistent-wrong-interpretation. The OPBR-Bench v0 corpus is the reproducibility surface for both.

---

## 14. Closing

Determinism is the wrong top-level target for agent reliability for three independent reasons, each fatal on its own.

First, **determinism is engineering-irrelevant where it matters most**. Token-level bitwise determinism is achievable at 10–40% performance cost on fixed infrastructure. Outside that regime — hosted enterprise APIs, varied batches, cross-hardware deployment — providers explicitly disclaim determinism, and the autoregressive cascade ensures that tiny variations at early token positions produce completely different downstream sequences. The variations driving real production failures are not token-rounding artifacts but semantic divergences in goal interpretation. Engineering effort spent pursuing token-level determinism for production agent reliability is misallocated relative to the actual failure surface.

Second, **determinism is formally inadequate as a reliability object for stochastic systems**. The five-tradition convergence is not coincidence. PCTL (1994), e-process sequential testing (2009 onward), robust MDPs (2024), AI reliability science (2026), and agent behavioral contracts (2026) — each derived without coordination — converge on probability-bounded specification *P*<sub>≥*θ*</sub>[*ψ*] as the correct reliability object. Determinism is the *p* = 1, *k* = 0 degenerate corner of every one of them. The Drift Bounds Theorem quantifies the cost: at baseline error rate *q* = 0.99 over *T* = 100 steps, compliance probability without recovery decays to *q*<sup>*T*</sup> = 0.366; with recovery rate *r* = 0.95 it stays at ≥ 0.95. The thirty-fold gap is the formal price of insisting on determinism over 100 agentic steps.

Third, **determinism is philosophically the wrong relational target**. Hawley's commitment account, the only philosophical framework applicable to non-intentional agents, requires only that the agent behave in accordance with a commitment conferred by roles and external circumstances. The commitment can be probabilistic; it can be honored stochastically; trustworthiness obtains as long as the behavioral pattern matches the contract. Demanding determinism as a precondition for trust would deflate trust to bitwise reliability — the move Hardin's account makes, which the Stanford Encyclopedia rightly criticizes as describing reliability rather than trustworthiness.

The positive target — **delegation-grade behavior** — admits a formal specification (§5). It does not require determinism; it requires probability-bounded behavior over a specified contract surface, with consequential actions made attributable and verifiable. It explicitly acknowledges that the semantic-goal-alignment element is the open research gap.

**The forward decade — 2026 through 2036 — will be determined by whether the industry adopts the right reliability target.** If it pursues determinism, it will overspend on the engineering-tractable but semantically irrelevant token layer while underspending on the contract architecture that actually governs the failure surface. If it adopts delegation-grade behavior — predictable quality under variation, with consequential behavior made contract-bound, extended to include violation-detectable semantic commitments — it will build the governance infrastructure that lets stochastic agents earn the trust required for high-consequence delegation.

This whitepaper is the manifesto layer of the Raising Agents research program. Paper 1 is the first empirical lemma. The workbench is the open-source instrument. The benchmark is the reproducibility surface. The Teams route through Zartis is the practitioner translation. The bi-weekly publication is how the work travels.

The thesis — *do not make agents deterministic, make consequential behavior contract-bound* — is the sentence we want a senior engineer to forward to their lead.

---

## Selected References

This whitepaper draws on two longer source documents: **R1** — *Determinism, Delegation, and Trust: A Layered Synthesis Across Six Literatures* (~50 cited sources across formal methods, statistical testing, AI evaluation, HRI, organizational economics, and philosophy of trust), and **R2** — *How Major AI Builders and Tooling Vendors Frame Non-Determinism, Reliability, and Production Readiness in 2025–2026* (319 paragraphs auditing 11 vendors with verbatim quotes). The full citation graph is in those documents.

Key references for the load-bearing claims:

- **Atil et al. (2024).** "Non-determinism of 'deterministic' LLM settings." arXiv:2408.04667.
- **Aziz, Singhal, Balarin, Brayton, Sangiovanni-Vincentelli (1995).** "It Usually Works: The Temporal Logic of Stochastic Systems."
- **Bhardwaj (2026).** "Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Agents." arXiv:2602.22302.
- **Bjarnason et al. (2026).** "On Randomness in Agentic Evals." arXiv:2602.07150.
- **Ferrario (2024).** "Trust and Trustworthiness in AI." *Philosophy & Technology*.
- **Grünwald, de Heide, Koolen (2019).** "Safe Testing." arXiv:1906.07801.
- **Hadfield-Menell, Hadfield (2018).** "Incomplete Contracting and AI Alignment." arXiv:1804.04268.
- **Hansson, Jonsson (1994).** "A Logic for Reasoning about Time and Reliability." *Formal Aspects of Computing*.
- **Hawley (2019).** *Trust, Distrust and Commitment*.
- **Holmstrom (2016).** Nobel Prize Lecture.
- **Jarrahi, Ritala (2025).** "Rethinking AI Agents: A Principal-Agent Perspective." *California Management Review*.
- **Khanal, Tao, Zhou (2026).** "Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents." arXiv:2603.29231.
- **Kolt (2025).** "Governing AI Agents." arXiv:2501.07913.
- **Lee (2026).** "Capable but Unreliable: Canonical Path Deviation as a Causal Mechanism." arXiv:2602.19008.
- **Lee, See (2004).** "Trust in Automation: Designing for Appropriate Reliance." *Human Factors*.
- **Liu et al. (2025).** "An Empirical Study on Failures in Automated Issue Solving."
- **Luettig, Ramaswamy, Bunting (2024).** "Frontiers ML Meets Aerospace: Challenges of Certifying Airborne AI."
- **Meggendorfer, Weininger, Wienhöft (2024).** "Solving Robust Markov Decision Processes." arXiv:2412.10185.
- **Mehta (2026).** "Consistency Amplifies: How Behavioral Variance Shapes Agent Accuracy." arXiv:2603.25764.
- **Mustahsan et al. (2025).** "Stochasticity in Agentic Evaluations." arXiv:2512.06710.
- **Miller (2024).** "Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations." arXiv:2411.00640.
- **Rabanser et al. (2026).** "Towards a Science of AI Agent Reliability." arXiv:2602.16666.
- **Ramdas, Ruf, Larsson, Koolen (2020).** "Admissible Anytime-Valid Sequential Inference Must Rely on Nonnegative Martingales." arXiv:2009.03167.
- **Schroeder, Wood-Doughty (2024).** "Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge."
- **Taddeo (2010).** "Modelling Trust in Artificial Agents: A First Step Toward the Analysis of E-Trust."
- **Thinking Machines Lab (2025).** "Defeating Nondeterminism in LLM Inference."
- **Yao et al. (2024).** "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains." arXiv:2406.12045.

---

## Provenance

This whitepaper integrates four sources of research-grade evidence:

1. **Paper 1** — *Output-Passing Behavioral Regressions in AI Agents* (V4 manuscript; 5 studies executed; 3,797 runs; preregistered as `PAPER1_PROTOCOL_FREEZE.json`).
2. **Paper 2 confirmatory experiments** — E1 (output–contract dissociation under engineered prompt drift) and E2 (compliance robust to instruction style) executed on Claude Sonnet through the Claude Code agent harness; 540 runs across 3 conditions × 3 cases × 60 trials; preregistered as `PAPER2_PROTOCOL_FREEZE.json`; generation tag `paper2-confirmatory`.
3. **R1 hyperresearch report** — *Determinism, Delegation, and Trust* (vault tag `determinism-delegation-trust`; pipeline run `determinism-delegation-trust-20260524T050840Z`).
4. **R2 hyperresearch report** — *AI Vendor Non-Determinism / Evals Framing* (vault tag `ai-nd-evals-framing`; pipeline run `ai-nd-evals-framing-20260524T050915Z`).

The Raising Agents research program is artifact-first, evidence-bound, and anti-hype. Every load-bearing claim in this whitepaper carries a citation to a primary source. The two longer hyperresearch reports are public companions to this whitepaper and contain the full per-claim citation graph. The Paper 2 confirmatory experiments are reproducible from the workbench repository under the generation tag above.

— Adrian Sanchez de la Sierra
Loulé, Portugal · 2026-05-25

`innovation@zartis.com`