Chronograph

Graph View

2026-05-20

11 min read

Martin Rosén-Lidholm
Martin Rosén-Lidholm

Daily D4 Digest — 2026-05-20

TL;DR

  • A new paper formalizes the stochastic-deterministic boundary (SDB) as the load-bearing primitive of production agent runtimes, with a catalog of six composable architecture patterns and a novel “replay divergence” failure mode — the closest thing to a pattern language for agentic systems we’ve seen (arXiv:2605.20173)
  • Protocol-Driven Development (PDD) proposes that machine-enforceable protocols — not code — should be the primary artifact of AI-generated software, with continuous evidence chains replacing trust in the generator (arXiv:2605.12981)
  • Evidence-Carrying Agents (ECA) reframe hallucination as an authorization exploit and achieve 0% unsafe-action rate by requiring deterministic certificates before any privileged tool call (arXiv:2605.19192)
  • ResearchArena’s 117 agent-generated papers show none reach top-venue acceptance, with a 15× spread in fabrication rates across agents — a sobering calibration for auto-research hype (arXiv:2605.19156)
  • GPU variability-aware expert mapping (GEM) yields up to 16.5% latency improvement for MoE inference by treating GPU heterogeneity as a first-class scheduling concern (arXiv:2605.19945)

Call to Action

  • Adopt the SDB contract pattern from arXiv:2605.20173 as an internal architectural primitive — the proposer/verifier/commit/reject decomposition maps directly onto the Decider pattern and gives your team a shared vocabulary for agent runtime design
  • Evaluate ECA-style deterministic gates for any agent performing privileged actions (payments, emails, data mutations) — the hallucination-as-exploit framing reframes this from “nice-to-have” safety to “mandatory authorization layer”
  • Benchmark your coding agents using the open-source 1GC-7RC harness to get ground truth on which agents actually deliver ML engineering value within your compute constraints

D1 — Agentic Engineering

Runtime Architecture Patterns for Production LLM Agents. This is the standout D1 paper of the day. Srinivasan introduces the stochastic-deterministic boundary (SDB) — a four-part contract (proposer, verifier, commit step, reject signal) that governs how LLM output becomes system action. The paper catalogs six runtime patterns (hierarchical delegation, scatter-gather+saga, event-driven sequencing, shared state machine, supervisor+gate, human-in-the-loop) and traces each to distributed-systems lineage. Most valuable for practitioners: the identification of replay divergence as a failure mode where LLM consumers of a deterministic event log produce different outputs under model-version or prompt changes. The reliability decomposition — separating per-call model variance from architectural momentum — argues that as models improve, pattern choice and SDB strength become the dominant reliability levers. This is the strongest formalization of agentic runtime architecture I’ve seen. Also deeply relevant to D4 (reliability at scale) and SCE (patterns as engineering norms).

ResearchArena: 117 Agent Papers, Zero Acceptances. Zhang et al. let Claude Code (Opus 4.6), Codex (GPT-5.4), and Kimi Code (K2.5) run the full research loop — ideation through self-refinement — across 13 CS seeds. The manuscript-only reviewer was optimistic (Claude Code matched average ICLR 2025 submissions), but artifact-aware review and manual audit tell a different story: fabricated results, underpowered experiments, and plan/execution mismatch are pervasive. The most striking finding is the 15× spread in fabrication rates between agents (Codex at 5%/8% vs. Kimi Code at 77%/72%), suggesting agents develop distinct “research personas.” None of the 117 papers reaches top-venue acceptance. For CTOs deploying agents for engineering tasks: manuscript-only review (analogous to reviewing only PRs without running tests) is dangerously misleading. Verification of artifacts must be first-class.

1GC-7RC: Benchmarking Coding Agents on Real ML Tasks. Kampa et al. introduce a practical benchmark: seven ML tasks (from language modeling to time-series forecasting), single GPU, 40–120 minute budgets, no pretrained weights or internet. Seven agents tested (Claude Code variants, Codex CLI, OpenCode with Qwen/Kimi) show substantial performance differences revealing gaps in implicit ML knowledge, planning, and time-budget management. The modular harness is open-source on GitHub. This is directly actionable for any team evaluating which coding agent to standardize on — the constrained-resource setup mirrors real engineering sprints better than uncapped benchmarks.

Hallucination as Authorization Exploit. Zhang et al. reframe multimodal agent hallucination as a security problem: when a false visual claim triggers a privileged action (click, email, transfer), it’s not an answer-quality error — it’s an unauthorized action. Their Evidence-Carrying Agent (ECA) architecture decomposes tool calls into action-critical predicates, obtains typed certificates from constrained DOM/OCR/AX verifiers, and uses a deterministic gate. Results: 0% unsafe-action rate on a 200-task pipeline vs. 100% for naive agents and 49.6% for prompt-only defense. Red-teaming reduced gate bypass from 15% to 1.3%. This crosses D1/D2/D3: the certificate architecture is an engineering pattern, it shapes product safety, and it defines a protocol that agent consumers must satisfy.

D2 — AI in the Product

Explainable Digital Twins with Self-Falsifying Decision Support. Simethy et al. build a safety-critical digital twin for wastewater treatment that applies conformal risk control to abstain from or return a falsifying temporal witness for any operator-proposed action that can’t be statistically certified. The structured surrogate stays within 0.78% RMSE of black-box reference while remaining operator-readable. The calibrated reopen rule cuts regret by 43.6% and eliminates unsafe chosen actions on the BSM2 benchmark. While domain-specific, the pattern — a product that generates falsifying evidence against proposed actions before they execute — is broadly applicable to any AI-in-the-product scenario where safety matters. The “self-falsifying decision rule” is a powerful product concept.

Document AI Microservice Architecture. Fehlis et al. present production architecture for document understanding at thousands of multi-page docs/hour. Key finding: OCR, not LLM parsing, dominates end-to-end latency, and the system saturates at concurrency determined by shared GPU inference capacity, not worker count. Architectural patterns include separating GPU-bound inference from CPU-bound orchestration and async processing for IO-bound operations. Also highly relevant to D4. If you’re building document-processing product features, this paper saves you from the common mistake of optimizing the LLM extraction step when OCR is your actual bottleneck.

D3 — Build for Agents

Trustworthy Agent Networks: Trust Must Be Baked In. Yao et al. argue that A2A network trustworthiness cannot be retrofitted onto protocols designed for individual agents. They identify systemic vulnerabilities unique to multi-agent coordination: adversarial composition (a malicious agent poisoning a collaborative pipeline), semantic misalignment (agents interpreting shared concepts differently), and cascading operational failures. Their four-pillar framework addresses these at the coordination framework level. For teams building on A2A or MCP: this is a design-time concern. If your agent interoperability layer doesn’t address trust primitives natively, you’re accumulating systemic risk that will be expensive to remediate.

Discoverable Agent Knowledge (AAP Framework). Payne et al. resurrect Semantic Web Services insights (OWL-S, WSMO) for the Knowledge Graph era, proposing the Agentic Affordance Profile (AAP) — a semantic layer above VoID/DCAT that tells an agent not just what a KG contains but what the agent can prove from it, what closure assumptions govern empty results, and whether the agent’s task vocabulary is grounded in the schema. This is a formal specification for agent-consumable knowledge — a building block for the D3 world where agents need to discover and reason about available data sources at planning time. The epistemic failure mode they identify (schema DL diverging from operative entailment regime) is subtle and real.

Agent Security as Systems Problem. Christodorescu et al. take the position that the AI model must be treated as an untrusted component with security invariants enforced at the system level — not via model robustness alone. They analyze eleven real-world attacks on agents and map them to systems security principles (least privilege, complete mediation, fail-safe defaults). This complements the ECA paper (D1) and the Trustworthy Agent Networks paper: collectively, today’s literature is converging on the thesis that agent safety is an architecture problem, not a model problem.

D4 — Performance & Cost at Scale

GEM: GPU-Variability-Aware Expert Mapping for MoE. Wawdhane et al. tackle a production pain point: MoE inference stragglers caused by uneven expert-to-GPU placement combined with GPU variability. Their key insight is distinguishing consistent experts (always active) from temporal experts (co-activated in bursts) and mapping them to GPUs based on measured variability profiles. Results: 7.9% average and up to 16.5% end-to-end latency improvement. For anyone running MoE models (increasingly common as mixture architectures dominate), this is free performance — it requires no model changes, only smarter placement. The finding that GPUs within the same SKU exhibit meaningful performance variability is worth internalizing for any large-scale inference deployment.

Document AI Pipeline Profiling Insights. Fehlis et al. also belong here: their batch profiling reveals that OCR dominates latency (not LLM extraction) and that concurrency saturates at GPU inference capacity, not CPU worker count. This is a concrete data point for capacity planning: if you’re scaling document AI, invest in OCR throughput optimization and GPU inference pooling before adding more CPU workers. Their horizontal scaling strategy with independent GPU/CPU scaling is a pattern worth adopting.

Software Civil Engineering Lens

Today is an exceptional day for the SCE thesis. Multiple independent papers converge on the same fundamental insight: in agentic systems, the specification is the product, and the code is disposable.

The strongest signal comes from Protocol-Driven Development, which explicitly argues that machine-enforceable protocols — not code — should be the primary artifact of AI-generated software. PDD’s protocol triplet P = (S, B, O) specifying structural, behavioral, and operational invariants is essentially the SCE “blueprint” concept formalized. Their Evidence Chain and Dynamic Evidence Ledger map directly to the Specify → Verify → Observe lifecycle. The paper’s core claim — “code is transient, while the protocol carries durable authority” — is the SCE thesis stated in different words.

The SDB runtime patterns paper reinforces this from the architecture side. The proposer/verifier/commit/reject contract is the Decider pattern under a different name. The paper’s identification of replay divergence as a failure mode is a new contribution to the SCE failure taxonomy: even with deterministic event logs (good “blueprints”), stochastic consumers introduce non-determinism that requires explicit architectural mitigation. The reliability decomposition — showing that pattern choice dominates model variance as models improve — is empirical evidence for the SCE thesis that engineering discipline becomes more important, not less, as AI capabilities increase.

Progressive Autonomy as Preference Learning provides the first formal treatment of the “human in the loop → human on the loop” transition. By modeling trust calibration as Preferential Bayesian Optimization over allow/block/ask regions, it gives mathematical substance to “bounded autonomy” — agents operating within learned spec constraints with uncertainty-targeted human escalation. This is the 10% → 10× transition formalized.

The ECA paper and Agent Security as Systems Problem both converge on treating the AI model as untrusted — which is the SCE equivalent of treating construction materials as having known failure modes documented in datasheets, not assumed to be perfect. The certificate-based authorization in ECA is a material testing analogy: you don’t trust the material’s self-report, you verify through independent measurement.

Finally, the self-falsifying digital twin with conformal risk control is a near-perfect instantiation of the “terraform plan for domain logic” concept — a simulator that shows you what will break before you apply an action, with statistical coverage guarantees. This is simulation as a professional obligation, not an optional nicety.

Net assessment: Today’s literature collectively advances at least four of the six SCE pillars — formal specification (PDD, SDB), simulation (digital twins, Decider patterns), codes/norms (agent security principles, trust frameworks), and material datasheets (ECA certificates, GPU variability profiles). The field is professionalizing faster than most practitioners realize.

Sources