2026-05-09

Daily D4 Digest — 2026-05-09

TL;DR

• VibeServe demonstrates agents that synthesize entire bespoke LLM serving stacks, outperforming vLLM in non-standard scenarios — generation-time specialization over runtime generality
• RGAO brings formal budget algebras to multi-agent code generation, proving resource conservation under dynamic topology selection — a concrete step toward provable agent contracts
• Execution lineage (DAG-based replay) eliminates state contamination in evolving agentic workflows, separating “answer quality” from “maintained-state quality”
• TACT identifies “agent drift” as a steerable direction in LLM residual streams, cutting coding-agent steps by 26% via activation steering
• Two inference cost breakthroughs: int4 KV cache outperforms fp16 on Apple Silicon; SPEED cuts 128K-context TTFT by 33% with layer-asymmetric visibility

Call to Action

• Evaluate VibeServe for non-standard serving needs: if you run unusual architectures or workloads, the agentic stack synthesis approach may yield wins that general-purpose systems miss — code is open source
• Adopt execution lineage for long-running agent workflows: if your agents produce evolving artifacts, consider the DAG-based execution model to prevent state contamination across revisions
• Benchmark your agents against Partial Evidence Bench: if you deploy enterprise agents with ACL-scoped retrieval, the 72-task benchmark will expose whether your system silently overclaims completeness

D1 — Agentic Engineering

Agents That Build Their Own Serving Infrastructure. VibeServe is the most provocative D1 result today: a multi-agent loop that generates entire LLM serving stacks end-to-end, competitive with vLLM in standard settings and superior in six non-standard scenarios (unusual architectures, workload-aware optimizations, hardware-specific tuning). The outer loop plans and tracks design search; the inner loop implements, validates correctness, and benchmarks. This is the “agents writing infrastructure” thesis made concrete — and it suggests a future where serving systems are generated per-deployment rather than maintained as monolithic projects. Also strongly D4.

Provable Budget Conservation in Multi-Agent Orchestration. RGAO tackles the fundamental routing problem in multi-agent code generation: which orchestration topology should handle a given task? The system extracts a structural complexity vector from a hierarchical code index, then selects the topology accordingly. The headline result is a formal budget algebra with structural-induction conservation proofs — agents governed by six-dimensional budget vectors can dynamically switch topologies without exceeding resource bounds. Misrouting drops from 30.1% to 8.2%. This is formal contracts for agent orchestration, not just heuristic routing. Also D3/SCE-relevant.

Steering Away from Agent Drift. TACT identifies two failure modes in long-horizon coding agents — overthinking (redundant reasoning) and overacting (tool calls without integrating observations) — and shows these correspond to linearly separable directions in the model’s residual stream (AUC ≈ 0.9). By projecting activations back toward a “calibrated” region at inference time, TACT lifts resolve rates by +5.8pp on SWE-bench Verified while cutting steps-to-resolve by 26%. This is a mechanistic handle on agent reliability that doesn’t require prompt engineering or architectural changes.

Mise en Place: Deliberate Preparation as Context Engineering. This paper formalizes the intuition that “vibe coding” fails because agents lack structured context. The proposed three-phase methodology — contextual grounding, collaborative specification, task decomposition — enabled parallel agent implementation of a full-stack platform after ~2 hours of human preparation during a hackathon. The paper introduces “context fluency” as an emerging developer skill. While less technically novel than other items, it directly addresses the practical question of how engineering teams should restructure their workflows around agentic tools. SCE-relevant.

Auto-Research Loops with Specialist Agents. This work instantiates a closed empirical loop where specialist agents propose hypotheses, edit code, run experiments, and absorb feedback (including crashes and budget overruns) across 1,197 trials — with zero human intervention after setup. The key insight is that lineage feedback (auditable trajectories of proposals, diffs, scores, and failure labels) lets agents turn failures into later program-level edits. Across three benchmark tasks, the loop autonomously improved upon public starting recipes, including non-trivial changes like NanoChat attention-kernel path rewrites.

D2 — AI in the Product

Production-Grade Grounded QA for Banking at 20-50× Lower Cost. FinRAG-12B is deployed at 40+ financial institutions, achieving a 7.1pp improvement in query resolution over prior approaches. The 12B model outperforms GPT-4.1 on citation grounding with a calibrated refusal mechanism (12% “I don’t know” rate vs. GPT-4.1’s over-refusing 20.2%). Built with only 143M tokens of training data via curriculum learning and LLM-as-a-Judge filtering. The cost story is the headline for CTO audiences: 3-5× faster and 20-50× cheaper than GPT-4.1, validating the “fine-tuned small model beats frontier API” thesis for domain-specific production workloads. Also strongly D4.

D3 — Build for Agents

Partial Evidence Bench: When Agents Don’t Know What They Can’t See. This benchmark addresses a governance-critical failure mode: agents operating within ACL-scoped retrieval that produce answers appearing complete while material evidence lies outside the authorization boundary. Across 72 tasks in due diligence, compliance audit, and security incident response, silent filtering proved “catastrophically unsafe” while explicit fail-and-report behavior eliminated unsafe completeness. This is directly relevant to any B2A or agent-to-agent architecture where delegated agents operate with partial information — the benchmark makes a previously unmeasurable failure mode measurable without human judges.

Governed Metaprogramming: Eval as a Governed Effect. This paper argues that when AI agents synthesize executable code at runtime, the transition from representation to execution is an authority amplification that must be mediated. The proposed MashinTalk DSL (compiling to BEAM bytecode) treats materialization as a governed effect — the governance system inspects capability requirements, policy compliance, and resource estimates before permitting execution. Three formal properties are proven: purity of form manipulation, no-bypass, and boundary preservation. Backed by 454 machine-checked Rocq theorems. This is foundational infrastructure for any system where agents generate and execute code.

D4 — Performance & Cost at Scale

Int4 KV Cache Outruns fp16 on Apple Silicon. This result inverts the conventional quality-latency tradeoff: a fused Metal kernel for int4 KV-cache quantization runs faster than fp16 on Apple Silicon’s unified memory (3-8% ms/tok improvement on Gemma-3 1B across 256-4096 token prefixes) with 3× memory compression and near-zero perplexity increase. The kernel’s ~25ns/vec overhead is below bandwidth savings. For teams deploying on-device or on Mac infrastructure, this is essentially free compression. Also reduces Qwen’s 4-bit catastrophe by 12.5×.

SPEED: Layer-Asymmetric KV Visibility for 128K Context. SPEED materializes prompt-token KV states only in lower layers (75% depth cutoff) while keeping decode-phase tokens full-depth. On Llama-3.1-8B at 128K context: TTFT improves 33%, TPOT improves 22%, active KV memory drops 25%, with negligible quality loss (51.2 vs 51.4 average score). This is an architecture-level insight — prompt tokens don’t need full-depth persistence — that’s complementary to quantization approaches and directly applicable to agentic workloads with long context windows.

Software Civil Engineering Lens

Today’s batch is one of the strongest SCE signals in recent memory, with four papers directly addressing formal guarantees, specification-driven workflows, and the professionalization gap.

Formal contracts are entering agent orchestration. RGAO’s budget algebra with structural-induction conservation proofs is exactly the “material datasheet + simulation” pattern from SCE applied to multi-agent systems. Agents operate within formally bounded resource envelopes, and topology changes are provably safe. This is the Decider pattern for agent infrastructure — verify before you apply.

Execution lineage is blueprints for evolving work. The DAG-based execution model directly instantiates the SCE lifecycle: artifacts have explicit dependencies, stable intermediate boundaries, and identity-based replay. The key finding — that “final answer quality” and “maintained-state quality” are distinct — is the software engineering analog of civil engineering’s distinction between a building passing inspection today vs. being maintainable over decades. Loop-based agents can produce a good answer once while silently accumulating state inconsistencies. This is exactly the class of failure that professionalized engineering practices exist to prevent.

Governed metaprogramming fills the “codes and norms” gap. MashinTalk’s treatment of eval-as-governed-effect, backed by 454 machine-checked theorems, is the most formally rigorous governance contribution today. When agents can synthesize and execute code, the boundary between “planning” and “doing” must be enforced structurally, not by convention. The no-bypass theorem is a building code for agent-generated programs.

Mise en Place validates spec-driven development empirically. The MEP methodology — contextual grounding → collaborative specification → task decomposition — maps cleanly onto the SCE lifecycle’s Specify → Plan → Verify → Apply → Observe. The paper’s framing of “context fluency” as a core developer skill is the human-on-the-loop thesis: the developer’s job shifts from writing code to writing specifications that agents can reliably execute within bounded autonomy.

The professionalization thesis is advancing fastest along the “formal specification” and “simulation” pillars today. Budget algebras, execution DAGs, governed effects, and structured preparation methodologies all represent different research groups converging on the same insight: unstructured agent autonomy doesn’t scale, and the solution looks increasingly like engineering discipline.

Sources

• Partial Evidence Bench — Benchmark for measuring unsafe completeness when agents operate under ACL-scoped retrieval
• FinRAG-12B — 12B grounded QA model deployed at 40+ banks, 20-50× cheaper than GPT-4.1
• RGAO — Complexity-conditioned topology routing with provable budget conservation for multi-agent code generation
• TACT — Activation steering to correct overthinking/overacting in coding agents, +5.8pp on SWE-bench
• VibeServe — Multi-agent loop that synthesizes bespoke LLM serving stacks, competitive with vLLM
• SPEED — Layer-asymmetric KV visibility: 33% TTFT improvement at 128K context
• Execution Lineage — DAG-based execution model for reproducible, maintainable AI-native work
• KG for Formal Verification — Knowledge graphs for agentic SystemVerilog assertion synthesis
• Governed Metaprogramming — Reclassifying eval as governed effect with 454 machine-checked theorems
• Mise en Place — Three-phase preparation methodology for agentic coding as context engineering
• Int4 KV Cache on Apple Silicon — Fused Metal kernel achieving faster-than-fp16 with 3× compression
• Auto Research Agents — Specialist agents running 1,197 autonomous experimental trials with lineage feedback