Chronograph

Graph View

2026-04-29

10 min read

Martin Rosén-Lidholm
Martin Rosén-Lidholm

Daily D4 Digest — 2026-04-29

TL;DR

  • Agentic coding tasks consume 1000× more tokens than code chat, with runs on the same task varying up to 30×—and models cannot predict their own costs (token study)
  • ClawTrace introduces per-step USD cost tracing for agent sessions; its “prune patches” transfer across benchmarks and cut median cost by 32% (ClawTrace)
  • A decoupled Human-in-the-Loop architecture proposes treating human oversight as a protocol-level concern independent of agent application logic—a key step toward bounded autonomy (HITL paper)
  • Multiple papers converge on pre-execution governance gates for AI agents: Right-to-Act (non-compensatory halt conditions), RiskGate (viability-theory bounds), and RAM (coverage envelopes)—collectively formalizing what SCE would call “codes and norms” for agent behavior
  • MoE inference gets up to 20× reduction in all-to-all communication via workload-aware expert placement, directly impacting serving costs for frontier models (MoE paper)

Call to Action

  • Instrument your agent loops with per-step cost attribution now. ClawTrace’s TraceCard format (YAML with USD cost, token counts, redundancy flags) is open-source and can be adopted immediately to find your most expensive wasted steps — ClawTrace repo
  • Evaluate decoupled HITL as an architecture pattern for your multi-agent deployments. The four-dimension framework (intervention conditions, role resolution, interaction semantics, communication channel) maps cleanly to progressive autonomy policies — HITL paper
  • If you’re running MoE models multi-node, profile your expert activation patterns per domain (code vs. math vs. chat) and consider workload-aware micro-batch grouping to reduce inter-node overhead — MoE scaling paper

D1 — Agentic Engineering

Cost-aware skill distillation changes how we learn from agent traces. ClawTrace introduces a tracing platform that records every LLM call, tool use, and sub-agent spawn with per-step USD cost, compiling sessions into “TraceCards.” The key insight is the three-way skill patch taxonomy: preserve (keep what worked), prune (remove expensive steps that didn’t matter, backed by counterfactual evidence), and repair (fix failures). The most striking finding: prune rules generalized across benchmarks (32% median cost reduction on unrelated tasks), while preserve rules—trained on benchmark-specific conventions—caused regressions on new task types. This is a direct challenge to the common assumption that learning “what works” is more valuable than learning “what to skip.” Also cross-cuts D4.

Agentic token economics are worse than you think. The first systematic study of token consumption in agentic coding across eight frontier LLMs on SWE-bench reveals alarming patterns: agentic tasks consume 1000× more tokens than code chat, with input tokens (not output) driving costs. Runs on identical tasks can vary by 30×. Token usage doesn’t correlate linearly with accuracy—performance peaks at intermediate cost and saturates. Most concerning: frontier models systematically underestimate their own token costs (max correlation of 0.39 with actual usage). Human expert difficulty ratings are also weakly aligned with actual compute cost. This fundamentally challenges budget-based agent orchestration strategies. Also D4.

Decoupled HITL as an architectural primitive. Cheng & Cheng argue that human-in-the-loop mechanisms are typically embedded in application logic, limiting reuse and consistency across multi-agent environments. Their proposed architecture treats human oversight as an independent system component with explicit interfaces and a structured execution model. The four-dimension formalization (intervention conditions, role resolution, interaction semantics, communication channel) enables selective, context-aware human involvement while maintaining system-level consistency. Critically, they position HITL as a protocol-level concern that aligns with emerging agent communication protocols—suggesting human oversight could be standardized the way authentication is in web services. Also cross-cuts D3.

FormalScience enables domain-expert-driven formal verification. FormalScience presents a human-in-the-loop agentic pipeline that lets a single domain expert—without formal methods experience—produce syntactically correct and semantically aligned Lean4 proofs at low economic cost. Applied to physics, they built FormalPhysics (200 university-level problems in quantum mechanics and electromagnetism). The paper’s characterization of “semantic drift” in autoformalisation—notational collapse and abstraction elevation—is valuable for anyone building agents that translate between informal specifications and formal representations. This is directly relevant to the SCE thesis of spec-driven development.

D2 — AI in the Product

No significant D2-specific developments today. The selected items skew heavily toward infrastructure, governance, and engineering tooling rather than user-facing product features. The closest D2-relevant work is the FormalScience interactive UI for theorem proving, but its primary significance is in D1/SCE.

D3 — Build for Agents

Reconstructive Authority Model redefines execution validity for autonomous systems. RAM makes a subtle but important argument: cryptographic attestation proves integrity of computation and state, but never proves adequacy of what was measured. Under partial observability, an authenticated projection of state is necessary but insufficient for execution validity. RAM introduces a “reconstruction gate” that reasons over a coverage envelope (proven state + declared assumptions + acknowledged unobservable residual) and permits execution only when coverage is adequate for the action class. When insufficient, it dynamically narrows privileges or fails closed. Synthetic experiments (N=100k) show attestation-based systems exhibit 23.3% invalid execution rates even at full coverage, while RAM achieves zero. This has direct implications for agent-to-agent trust in B2A architectures—agents consuming other agents’ outputs need coverage guarantees, not just integrity proofs.

Three governance gate papers form a complementary stack. Beyond RAM, Right-to-Act proposes a deterministic, non-compensatory pre-execution layer (if any required condition is unmet, execution halts—no compensatory overrides allowed), while RiskGate formalizes runtime governance using Aubin’s viability theory with a scalar Viability Index and predictive time-to-boundary estimation. Together, these three papers sketch what a complete governance stack for agent interoperability looks like: RAM handles state coverage, Right-to-Act handles permission boundaries, and RiskGate handles runtime drift detection. Also D1.

D4 — Performance & Cost at Scale

MoE inference gets 20× communication reduction through expert placement. Bambhaniya et al. profile Llama 4 Maverick, DeepSeek V3-671B, and Qwen3-230B-A22B across 100k+ activation traces and discover persistent properties: variable expert load imbalance, domain-specific expert activation (expert popularity shifts between code, math, chat, general tasks), and strong prefill-decode activation correlation. Their workload-aware micro-batch grouping and expert placement strategy reduces all-to-all communication by up to 20×. For anyone serving these models, this is immediately actionable—the domain-specific activation finding means you should be routing different workload types to different expert placement configurations.

CuTile benchmarks on Blackwell: 2.5× FlashAttention-2 for fused attention, but portability gaps remain. The first independent CuTile evaluation across H100, B200, and RTX PRO 6000 shows NVIDIA’s Python-based tile abstraction hitting 1007 TFLOP/s for fused attention on B200 in just 60 lines of code. However, the same kernel achieves only 53% of FlashAttention-2 on RTX PRO 6000 (sm_120). Meanwhile, Triton sustains 62-101% of cuBLAS across all platforms without architecture-specific tuning. The takeaway: CuTile is compelling for Blackwell datacenter deployments but Triton remains the safer bet for cross-architecture portability.

DepthKV: smarter KV cache pruning for long-context inference. DepthKV challenges the uniform-pruning-across-layers assumption by showing that layers differ significantly in pruning sensitivity. Their layer-dependent allocation of a fixed global KV budget consistently outperforms uniform pruning at the same ratio. For long-context agent workloads (which the token consumption study shows are already extremely expensive), this is a practical memory optimization that preserves quality.

Software Civil Engineering Lens

Today’s batch is unusually rich for the SCE thesis, with multiple papers independently converging on professionalization patterns:

Formal specification is becoming accessible. FormalScience demonstrates that agentic pipelines can bridge the gap between informal reasoning and formal Lean4 proofs with a single domain expert and low cost. This directly addresses the SCE “formal spec” pillar—the historical bottleneck was that formal methods required both domain expertise AND formal language expertise. If agentic pipelines can reliably produce “syntactically correct and semantically aligned” formal representations, the barrier to spec-driven development drops dramatically. Their characterization of semantic drift (notational collapse, abstraction elevation) is especially valuable: it tells us what formal language can and cannot verify when full semantic preservation is impossible, which is exactly the kind of “material datasheet” knowledge the SCE framework demands.

Codes and norms are emerging from three directions simultaneously. The Right-to-Act protocol (non-compensatory pre-execution gates), RiskGate (viability-theory bounds with monotonic restriction), and RAM (coverage envelope requirements) are all independently formalizing what amounts to building codes for AI agent behavior. Right-to-Act’s non-compensatory principle is particularly SCE-aligned: just as a building inspector can’t let a great foundation compensate for inadequate fire escapes, high model confidence shouldn’t override unmet structural conditions. These papers collectively move the “codes/norms” pillar forward, even though none of them reference each other.

The simulation gap is real but addressable. The token consumption study reveals that we can’t currently predict agent execution costs (max r=0.39), and runs on identical tasks vary by 30×. This is the equivalent of not knowing how much concrete a building will require until after it’s poured. ClawTrace offers a partial answer: by instrumenting agent traces with per-step cost and building counterfactual prune patches, we get closer to a “terraform plan” for agent behavior—understanding what will happen and what it will cost before committing resources. The finding that prune rules transfer but preserve rules don’t is a profound insight for SCE: the transferable knowledge in agentic systems may be about constraints (what not to do) rather than procedures (what to do), which is exactly how building codes work.

The EU-AI-Act-compliant forecasting package (spotforecast2-safe) embeds regulatory requirements directly into API contracts, persistence formats, and CI gates with a bidirectional traceability matrix from regulation to code. Its four non-negotiable rules (zero dead code, deterministic processing, fail-safe handling, minimal dependencies) and deliberate exclusion of non-deterministic components represent compliance-by-design rather than compliance-by-audit. This is perhaps the most concrete example yet of the SCE “licensure” pillar manifesting in software: not just meeting standards, but making non-compliance structurally impossible.

SemML 2.0 advances LTL synthesis—generating reactive systems from temporal logic specifications that are correct by construction. While niche, this is exactly the kind of tool that the “simulation” pillar of SCE needs: given a formal specification, can we automatically generate a controller that provably satisfies it? The answer is increasingly yes, and the tool now outperforms all competitors on SYNTCOMP benchmarks.

Net assessment: Today represents a high-water mark for SCE evidence. We’re seeing simultaneous progress on formal specification (FormalScience), codes/norms (Right-to-Act, RiskGate, RAM), simulation/planning (ClawTrace, SemML), and compliance-by-design (spotforecast2-safe). The decoupled HITL paper explicitly frames “human on the loop” as a protocol-level architectural concern—which is precisely the SCE model of relocating human judgment to a higher control plane. The transition from craft to engineering discipline is no longer theoretical; individual components of the professionalization stack are being built in parallel by independent research groups.

Sources