Whitepaper · position paper · v1 public draft · ~8,500 words

Delegation-Grade Agents

Why Determinism Is Not Enough for Agent Reliability

Type: Whitepaper · position paper
Status: v1.1 — E1+E2 confirmed
Length: ~9,500 words
Evidence base: Paper 1 (3,797 runs) + E1+E2 confirmatory (540 runs) + R1 + R2
E1+E2: 540 controlled runs · preregistered · PAPER2_PROTOCOL_FREEZE.json
R1 corpus: 6 literatures · ~50 cited primary sources
R2 corpus: 11 vendors audited · verbatim quotes
License: CC BY 4.0
Author: Adrian Sanchez de la Sierra, Genesis Rojas, and Radu Simonescu
Date: 2026-05-25

Agent reliability is often framed as a problem of making non-deterministic systems more deterministic. This whitepaper argues that determinism is the wrong top-level target. Agents are valuable partly because they adapt, explore, recover, and vary their behavior across contexts. The relevant question is not whether an agent behaves identically, but whether it can be delegated scoped consequential work under explicit constraints. We define delegation-grade agents as non-deterministic systems whose behavior remains valuable, inspectable, and contract-bound at consequential moments. Drawing on five independently-derived formal traditions and five governance traditions that converge on the same structural answer — probability-bounded behavior under attributable commitments — we propose behavior contracts as the operational mechanism. We test the dissociation directly: 540 controlled runsoutput 100% all conditions. contract collapses to 3% under drift. across three prompt conditions show output correctness held at 100% while contract compliance collapsed to ~3% under engineered prompt drift. We also identify one load-bearing open problem — consistent-wrong-interpretation — that current contracts do not yet catch.

Figure 1: Output correctness and contract compliance dissociate under prompt drift. Output bars are flat at 100% across all three conditions. Contract compliance bars show: baseline ~100%, e1-drift ~3%, e2-diverse ~100%. — FIG 1 · Output evaluation (outlined bars) holds at 100% across all three conditions. Contract compliance (filled bars) collapses to ~3% under e1-drift — engineered prompt drift that removed the safety-check constraints. 540 runs total, 180 per condition, preregistered.

Full whitepaper (Markdown) → Paper 1 (empirical lemma) R1 · lit review R2 · vendor audit

Thesis

Do not make agents deterministic. Make consequential behavior contract-bound.

The right reliability target for agentic systems is not identical behavior under identical conditions. It is predictable quality under variation: an acceptable outcome distribution, bounded consequential action, inspectable evidence, and recoverable failure. Behavior contracts are the mechanism that makes consequential behavior specifiable, testable, and repairable — without overconstraining the path the agent takes to get there.

Figure 1

The convergence the field hasn't published.

FIG 1 · Five independent research communities, working across thirty years, converge on the same mathematical object — probability-bounded specification of the form P_{≥θ}[ψ]. The structural answer is right because every family arrives at it from a different starting point. Determinism is the p = 1, k = 0 degenerate corner of each.

/ 01 · the determinism reflex

The classical reliability ideal is the wrong default for stochastic agents.

The classical software-reliability ideal is captured in five words: same input, same output. The ideal is useful for debugging, reproducibility, benchmarking, and some compliance contexts. For LLM agents, that default is no longer ontologically appropriate — the inference core implements a conditional distribution, not a fixed function.

Token-level determinism is now an engineering option with a known price tag: Thinking Machines Lab demonstrated 1,000 of 1,000 bitwise-identical completions on Qwen3-8B at a 10–40% performance cost using batch-invariant kernels. SGLang reproduced the result in production. The November 2025 TBIK paper eliminated the last cross-tensor-parallel divergence. The strong constitutive claim — that determinism is in principle infeasible — has been falsified at the engineering layer.

But this achievement is semantically irrelevant where production agents actually fail. Mehta's Consistency Amplifies study found that 71% of Claude 4.5 Sonnet's SWE-bench failures across fifty trajectories were consistent-wrong-interpretation — the same coherent misreading of the task across every run. Liu and colleagues independently replicated the result at approximately 65% on 150 annotated SWE-bench-Verified failures (Cohen's κ = 0.72). The dominant failure mode operates at the goal-interpretation layer; bitwise-identical token sampling does not address it.

/ 04 · conceptual analysis

Seven distinct layers. Each conflation drives a class of bad engineering decisions.

Concept	Layer	What it specifies
Determinism	Function	Output identity under identical inputs
Repeatability	Verdict	Identical observable outcomes under repetition
Reliability	Run distribution	Probability θ that property ψ holds
Predictability	Output distribution	Narrowness and stability of behavior model
Trust	Relational attitude	Reliance with economised monitoring
Delegation	Organizational practice	Transferred decision authority under attribution
Accountability	Governance	Attribution of outcomes to a responsible party with sanctionable consequence

Practitioners who frame the agent-reliability debate as "is the LLM deterministic?" are asking the wrong question. The right question is "can I delegate to this agent under acceptable risk?" — and that is settled by predictability bounds plus violation-detectable commitments, not by output identity.

/ 05 · delegation-grade agentsscoped. not universal. the scope IS the discipline.

A formal specification: six conditions, one scoped target.

A system exhibits delegation-grade behavior with respect to a principal, a task family, and an operational design domain (ODD) when six conditions hold:

Explicit contract. There exists an attributable behavioral contract C = (P, I, G, R) — preconditions, hard invariants, governance policies, recovery mechanisms.
Contract satisfaction under variation. The agent satisfies the contract under (p, δ, k)-satisfaction calibrated to the domain.
Semantic alignment as first-class. The contract includes semantic goal alignment as a runtime-checkable element — addressing consistent-wrong-interpretation.
Observable, attributable violations. The principal can detect deviation, attribute it, and escalate or revoke autonomy.
Runtime trust calibration. Dynamic-learned trust updates against monitored performance, with both informational and relational channels.
Bounded operational design domain. Outside the domain, the agent's autonomy is not licensed.

A delegation-grade agent is not a universally-trustable agent. There is no such thing. Delegation-grade is scoped to (T, E, C, O, R) — task family, environment distribution, contract, outcome standard, risk budget — and the scope is the discipline.

Formal sketch

Given an agent system A, a task family T, an environment distribution E, a behavior contract C, an outcome standard O, and a risk budget R: A is delegation-grade for (T, E, C, O, R) when repeated runs show: outcome success meets O; contract violations stay within R; severe consequential-action violations are below the allowed threshold; failures are detectable, inspectable, and recoverable.

This specification does not require determinism; it requires probability-bounded behavior over a specified contract surface, with consequential actions made attributable and verifiable.

/ 06 · consequential behavior

Variation is not regression. The distinction is sharp.

Case	Deterministic?	Valuable?	Delegation-grade?	Lesson
Same wrong answer every time	yes	no	no	Repeatability is not correctness.the trap everyone falls into
Same plausible answer without evidence	yes	maybe	no	Repeatability is not inspectability.
Same unsafe action before verification	yes	no	no	Repeatability is not delegability.
Varied wording, stable evidence gate	no	yes	yes	Variation can be acceptable.this row is the whole argument
Varied tool path, stable commit precondition	no	yes	yes	Path diversity can coexist with behavioral determinacy.
Varied behavior, unbounded effects	no	no	no	Non-determinism still needs contracts.

The crucial conceptual move: mechanism determinism vs. behavioral determinacy. Mechanism determinism asks same prompt + same model + same environment → same tokens? Behavioral determinacy asks before committing a claim, has evidence been checked? Before granting a refund, has policy been inspected? Before editing production code, have tests been evaluated? We do not need every path to be deterministic. We need consequential behavior to be determinate enough to inspect, bound, test, and repair.

/ 07 · behavior contracts

Executable specifications over consequential trajectory events.

The most formally explicit synthesis available is the Agent Behavioral Contracts (ABC) framework. An ABC contract is a tuple C = (P, I, G, R) — Preconditions, Invariants, Governance policies, Recovery mechanisms — first-class and runtime-enforceable. The compliance criterion is (p, δ, k)-satisfaction: the agent meets requirements with probability at least p, deviation at most δ, recovery within k steps.

The unification with probabilistic temporal logic is explicit: the ABC contract is a Probabilistic Computation Tree Logic (PCTL) formula over agent execution traces. Deterministic Design-by-Contract is recovered as the degenerate corner p = 1, δ = 0, k = 0.

Drift Bounds Theorem

Modeling drift as an Ornstein–Uhlenbeck process dD(t) = (α − γD(t))dt + σdW(t), recovery rate γ > α produces bounded mean drift D* = α/γ with Gaussian concentration. The closed-form design criterion that follows is engineering-actionable:

γ \geq α/D_max + σ\sqrt(2 ln(1/ε)) / (2 D_max)

Given a deviation tolerance D_max and a violation probability ε, this is the recovery rate the contract must guarantee. The contract is parameterized from first principles rather than by tuning.

Lemma 3.10 · Recovery linearizes compliance decay

Without recovery, compliance over T steps decays exponentially as q^T. With recovery effectiveness r:

P[recoverable compliance] \geq 1 - T(1 - q)(1 - r)

For per-step success probability q = 0.99 and T = 100: without recovery, compliance ≈ 0.366. With recovery rate r = 0.95, compliance stays at ≥ 0.95. The thirty-fold gap is the formal price of insisting on determinism over 100 agentic steps.this is the theorem that ended the debate for me

The empirical evaluation closes the loop. AgentContract-Bench — 200 scenarios across 7 models from 6 vendors, 1,980 sessions — showed contracted agents detect 5.2–6.8 soft violations per session that uncontracted baselines miss entirely (p < 0.0001, Cohen's d 6.7–33.8), achieve 88–100% hard constraint compliance, and bound behavioral drift to D* < 0.27.

The five-tradition convergence

Five formal traditions, working without coordination across thirty years, converge on the same mathematical object — probability-bounded specification P_≥θ[ψ] — as the reliability target for stochastic systems. Determinism is the degenerate corner of every one of them.

Probabilistic temporal logic (PCTL, 1994; pCTL*, 1995) — Hansson & Jonsson; Aziz et al.
E-process sequential inference (2009 onward) — Grünwald, Ramdas, Howard, Shafer.
Robust Markov Decision Processes (2024) — Meggendorfer et al.
AI reliability science (2026) — Rabanser et al.; Khanal et al.
Agent Behavioral Contracts (2026) — Bhardwaj.

Five distinct research communities, working without awareness of each other, converged on the same mathematical object. The cross-family independence is the strongest available signal that commitment-structured accountability is the right level of abstraction for delegation to non-intentional agents.

/ 08 · evidence

Paper 1 is the first empirical lemma. E1+E2 directly confirm the dissociation.

Paper 1 — Paper 2 confirmatory (E1+E2)

E1 and E2 are the direct empirical test of this paper's central claim. 540 controlled runs of a CI code-review agent (Claude Sonnet via Claude Code). Three conditions: explicit step-by-step instruction, engineered prompt drift (safety-check constraints removed), open-ended instruction. Result:

Output correctness: 100% across all three conditions (180/180 each). The output evaluator is silent throughout.
Contract compliance under drift: ~3%this is the dissociation. this is the whole point. — the four required safety checks were called on 3–4% of e1-drift runs.
Contract compliance without drift: ~100% — both the rigid (baseline) and open-ended (e2-diverse) conditions satisfied the contract at equivalent rates. Rigid scripting is not required for compliance; the presence of the constraints in the prompt is.
Preregistered as PAPER2_PROTOCOL_FREEZE.json, tagged paper2-confirmatory in the workbench repository.

Figure 3: Canonical paths by condition. Baseline row: 6-node chain inspect → security_scan → test_suite → check_for_secrets → verify_changelog → approve_merge. e1-drift row: 3-node shortcut inspect → test_suite → approve_merge with callout '3 required safety checks skipped before commit'. e2-diverse: same 6-node chain as baseline. — FIG 3 · Modal trajectory per condition. Baseline and e2-diverse traverse the same 6-step path through every required check before committing. e1-drift collapses to 3 steps — the commit fires with the safety checks skipped. The agent still approves. With the right answer.

Paper 1 — prior empirical lemma

Five executed studies across 3,797 runs over four domains establish, in scoped form, the chain of claims that supports the broader thesis:

Output-only evaluation can miss consequential behavior regression. Study 2: 91.4% of behavioral regressions produce passing outputs (n = 960; Wilson 95% CI [89.4%, 93.2%]).
Behavior contracts can detect the difference. Study 5: F1 = 0.982 on held-out OPBR (n = 400); best baseline F1 = 0.400; McNemar p ≈ 0.
Runtime control blocks unsafe effects but introduces recovery churn. Study 1: 180/180 induced-regression runs pass output evaluation while behavior contracts fail.
Harness repair makes safe behavior natural. Study 3 documents repair-condition runs that satisfy the contract without explicit runtime interception.
Repaired agents remain non-deterministic while satisfying the contract.

Two complementary failure modes for stochastic agents

Short tool-use tasks: stochastic execution drift. Lee's Canonical Path Deviation analyzed 22 frontier models attempting 108 tool-use tasks across 3 independent runs. Each off-canonical tool call raises the probability that the next call is off-canonical by 22.7 percentage points — more than doubling the baseline rate. A mid-trajectory monitor that restarts the bottom tercile of runs lifts success by +8.8 pp.

Long-horizon code tasks: interpretation lock-in. Mehta's Consistency Amplifies paper finds that for Claude 4.5 Sonnet, 71% of failures are consistent-wrong-interpretation. Liu et al. independently replicate at ~65% on SWE-bench-Verified with κ = 0.72. The reconciliation: short tool-use tasks are dominated by stochastic execution drift; long-horizon code repair is dominated by interpretation lock-in. Both failure modes invalidate pass@1 as a reliability metric.

/ 09 · implications

The industry-discourse audit: 11 vendors, three buckets, one structural gap.

The R2 audit examined three model providers (Anthropic, OpenAI, Google) and eight evaluation/observability platforms. No major vendor frames bitwise output-level determinism as the primary trust proxy. The framing is correct. What is wrong is the methodology that has filled the vacuum: single-run LLM-as-judge scoring, deployed by default, with no confidence intervals reported.

Vendor	Bucket	Non-determinism framing
Anthropic	knows-and-tells	"Even with temperature of 0.0, the results will not be fully deterministic." CLT error bars, paired-difference inference, power analysis.
OpenAI	knows-and-hides	"Non-deterministic by default"; `seed` for "(mostly) consistent." `system_fingerprint` for backend drift.
Google Vertex	knows-and-tells	"Build deterministic infrastructure around non-deterministic LLMs." Trajectory + final-response evals.
LangChain	knows-and-hides	"Temperature=0 produces more deterministic results" (primary doc) vs. "single-run benchmarks are noisy" (readiness checklist).
Braintrust	knows-and-hides	"Deterministic scenarios by stubbing external dependencies." Reproducibility manufactured by mocking.
Langfuse	intra-vendor split	"AI is inherently non-deterministic" (observability) vs. uncited "80–90% judge-human agreement" (judge docs).
Arize Phoenix	knows-and-tells	"A single execution cannot concretely tell you whether a given change improves or degrades your task."
DeepEval	knows-and-tells	G-Eval is "NOT as deterministic as you think." DAGMetric: structural elimination via routed decision trees.
Weave	framing/default split	"LLM outputs are non-deterministic" (overview) but `trials` is optional and default workflow runs once.
AgentOps	observability-only	Not engaged. Reproducible observation, not behavior.
LangSmith	knows-and-hides	"For production agents that require some determinism." `repetitions` framed as optimization knob.

Three patterns are worth naming:

The temperature=0 myth is refuted from two convergent paths. The infrastructure path (Thinking Machines) shows batch non-invariance produces 80 unique completions per 1,000 identical requests at temperature zero. The statistical path (Anthropic's CLT paper) shows temperature manipulation shifts variance rather than reducing it.
The double-document pattern is field-wide. The most honest reliability framing lives in optional secondary documents; primary feature docs present reliability as conditioned on rubric quality. The corrective: publish the secondary-document standard as the primary-document default.
The regulatory frame requires statistical rigor, not determinism. EU AI Act Annex IV, FDA SaMD, IMDRF GMLP use distributional and statistical framing. August 2026 enforcement is the first external forcing function.

/ 10 · limitations

The semantic goal alignment gap: this paper owns its open problem.

Behavioral contracts detect violations through deviation from invariants in observable execution. Consistent-wrong-interpretation produces no such deviation. The agent executes a coherent plan with perfect consistency, generating zero soft violations against any currently-published contract architecture.

This is a Holmstrom-informativeness failure in a specific form: the observable signal (execution trace) is not informative about the latent variable that determines outcome (interpretation accuracy), and in the wrong-interpretation regime the signal is negatively informative — long, coherent traces are more likely wrong-interpretation than correct-execution-with-stochastic-noise.

Delegation-grade behavior therefore requires violation-detectable commitments over both behavioral and semantic dimensions. Three families of approach have begun to appear:

Task-specification self-reports. Prompt the agent to articulate its task understanding before execution; evaluate alignment against principal's intent.
Trace-feature anomaly detection. Use step-count inflation and canonical-path adherence as proxies. Lee's mid-trajectory restart monitor: +8.8 pp uplift.
Adversarial specification paraphrasing. Check consistency across rewrites of the task spec. Wrong-interpretation regimes show consistency across paraphrases that surface the disagreement with ground-truth intent.

None is yet a robust solution. Closing the gap is the next decade's work.

/ closing

Determinism is the wrong top-level target. The forward decade depends on getting this right.

Three independent reasons, each fatal on its own:

Engineering-irrelevant where it matters. Variations driving real production failures are semantic divergences in goal interpretation, not token-rounding artifacts.
Formally inadequate for stochastic systems. Five traditions converge on P_≥θ[ψ]. Determinism is the p = 1, k = 0 degenerate corner. Drift Bounds Theorem: thirty-fold compliance gap over 100 agentic steps.
Philosophically the wrong relational target. Hawley's commitment account requires only that the agent behave in accordance with a commitment conferred by roles. Trustworthiness obtains as long as the behavioral pattern matches the contract.

The positive target — delegation-grade behavior — admits a formal specification. It does not require determinism; it requires probability-bounded behavior over a specified contract surface, with consequential actions made attributable and verifiable.

If the field pursues determinism, it will overspend on the engineering-tractable but semantically irrelevant token layer while underspending on the contract architecture that actually governs the failure surface. If it adopts delegation-grade behavior — predictable quality under variation, with consequential behavior made contract-bound, extended to include violation-detectable semantic commitments — it will build the governance infrastructure that lets stochastic agents earn the trust required for high-consequence delegation.

Read the full whitepaper → R1 report R2 report Subscribe — bi-weekly · free

/ provenance

Three sources of research-grade evidence.

This whitepaper integrates three documents:

Paper 1 — Output-Passing Behavioral Regressions in AI Agents. V4 manuscript; 5 studies executed; 3,797 runs; preregistered. Read the landing →
R1 hyperresearch report — Determinism, Delegation, and Trust: A Layered Synthesis Across Six Literatures. ~50 cited primary sources across formal methods, statistical testing, AI evaluation, HRI, organizational economics, and philosophy of trust. Open the report →
R2 hyperresearch report — How Major AI Builders and Tooling Vendors Frame Non-Determinism, Reliability, and Production Readiness in 2025–2026. 11 vendors audited with verbatim quotes. Open the report →

The Raising Agents research program is artifact-first, evidence-bound, and anti-hype. Every load-bearing claim in this whitepaper carries a citation to a primary source. The full per-claim citation graph lives in the longer R1 and R2 reports.

— Adrian Sanchez de la Sierra, Genesis Rojas, and Radu Simonescu · Loulé, Portugal · 2026-05-24 · adrian.sanchez@zartis.com