Skip to content

agent-loop-v0.md

Latest commit

 

History

History
186 lines (156 loc) · 9.07 KB

agent-loop-v0.md

File metadata and controls

186 lines (156 loc) · 9.07 KB
Raw

Agent-loop v0 — design (Step 2.11 spike)

Spike contract. This document describes a validation spike, not a production agent runtime. The deliverable of Step 2.11 is the gap-list memo, not a hardened agent. Phase 3.x will replace AgentLoopV0's heuristic controller with an LLM tool-loop driver in a dedicated rag-agent package. Until then AgentLoopV0 lives in rag-retrieval as a deliberately thin composition of the Phase 2 primitives.

Why this spike exists

Phase 2 (steps 2.1 – 2.10) built ten retrieval components — read layer, BM25, vector, graph, hybrid RRF fusion, query understanding, cross-encoder reranker, context packer, GraphRAG, retrieval router — and tested each in isolation against unit + integration fixtures.

Phase 3 will turn this into a gateway + agent runtime in which the same retrieval primitives are driven in a loop by an LLM-as- controller: rewrite query, retrieve, observe results, decide next move, retrieve again, …, up to a hard budget. That access pattern is fundamentally different from one-shot retrieval — cache reuse matters, plan reuse matters, sticky routing matters, budget enforcement matters at every layer.

Spike goal: surface mismatches between the Phase 2 design and agent access patterns before Phase 3 commits to them.

Design constraints

  1. Composability over completeness. The spike orchestrator must re-use existing Phase 2 components without rewriting them. Any gap that requires a Phase 2 refactor becomes a gap-list item, not an in-spike fix.
  2. Cross-package isolation preserved. rag-retrieval deliberately depends only on rag-core (see the package pyproject.toml). Integration with rag-query happens via a structural QueryUnderstander Protocol so neither package needs to know about the other. Same pattern the router uses for _graph_adapter and GraphAdapter.
  3. Heuristic controller only. No LLM-as-controller in v0 — stopping is on budget, max-iter, diminishing returns, or caller callback. Otherwise the spike measures LLM quality instead of architecture.
  4. Noop SPIs only in the harness. Real embedders / vector stores / graph stores would mask architectural gaps with provider quirks. The 50-query harness is exclusively against the in-memory noop SPIs so the numbers are deterministic and point at design problems.
  5. PolicyEngine boundary untouched. The loop is not a PDP consumer — every read_chunk consultation still happens inside HybridRetriever. The tests/policy/coverage.py linter passes naturally.

Components

                  ┌────────────────────────────────────────┐
                  │              AgentLoopV0               │
                  │                                        │
                  │   ┌──────────┐    ┌─────────────────┐  │
   ctx, query ───▶│   │  loop    │───▶│ stop conditions │  │
                  │   │ control  │    └─────────────────┘  │
                  │   └────┬─────┘                         │
                  │        │                               │
                  │  ┌─────▼──────┐  ┌──────────────────┐  │
                  │  │understander│  │ embedding cache  │  │
                  │  └─────┬──────┘  └────────┬─────────┘  │
                  │        │                  │            │
                  │  ┌─────▼──────────────────▼─────────┐  │
                  │  │       RetrievalRouter            │  │
                  │  │            │                     │  │
                  │  │            ▼                     │  │
                  │  │       HybridRetriever            │  │
                  │  │     (PolicyEngine here)          │  │
                  │  └──────────────────────────────────┘  │
                  └────────────────────────────────────────┘

Stop-condition matrix

Condition Checked when Fires StopReason
Budget exhaustion Top of iter budget_tokens / budget_dollars / budget_wall / budget_iter
should_continue returns False End of iter caller_stop
Diminishing returns End of iter, iter > 0 diminishing_returns
refine_query returns None Between iters caller_stop
max_iterations reached Loop completion max_iterations

The order matters: budgets are checked first so an exhausted budget never pays for one more iter's understanding + retrieval work. Diminishing returns runs last so the first iter always counts.

Sticky routing

AgentLoopConfig.allow_route_switch=False (the v0 default) makes iteration 0 pick a RoutingDecision and pins that decision for every subsequent iteration. The loop still calls router.route() internally, so the spike measures sticky behaviour without optimising for it — the gap-list flags the duplicate decide() cost as a candidate Phase 3.1 fix (G-02).

Cache integration

Each iteration's vector-input resolution priority (mirrors the router's own resolution):

  1. UnderstoodQuery.hyde_vector (already paid for inside the understander).
  2. EmbeddingCache.get((tenant, model_id, model_version, text_hash)) — hit.
  3. Embedder.embed(...) + EmbeddingCache.put(...) — miss.
  4. None — the router falls back to its own embed-or-skip behaviour.

text_hash uses the same lowercase + whitespace-collapse normalisation as the rag-packer dedup normalisation.

Plan-reuse semantics

Two adjacent decisions are "the same plan" iff their shape, backend-selection booleans, BM25-fallback flag, and weights match. reason and degraded_backends differences are diagnostic and ignored — they don't change execution semantics. See _decisions_match in agent_loop.py.

Invariants

  1. Budget monotonicity. Every iteration produces a new RequestContext via ctx.model_copy(update={"budget": new}). Each dimension can only decrease. Budget.spend clamps at 0.
  2. No mid-loop ctx mutation. RequestContext is frozen Pydantic. Any update goes through model_copy.
  3. Final-chunk dedup preserves first-seen order. See _dedupe_preserve_order.
  4. AgentLoopResult.stop_reason is always set. An assert enforces this immediately before construction.
  5. Strict failure mode for orchestration bugs, transparent propagation for retrieval errors. RetrievalError subclasses propagate unwrapped so the existing observability dashboards continue to attribute the failure to the retrieval layer. Other unexpected exceptions wrap in AgentLoopError(iter=…).

What the spike learned

Concrete findings live in agent-loop-v0-gaps.md. Headline numbers from the 50-query run on 2026-05-27:

Metric Value
Plan-reuse rate (mean) 58.9%
Embedding cache hit rate (mean per query) 28.2%
Embedding cache hit rate, decomposable bucket 0.0%
Stop reasons hit 5 of 7 possible

The decomposable-bucket 0% hit rate is the single most important finding: agent loops that fan out to genuinely-different sub-queries get zero benefit from the existing exact-text-hash embedding cache. That's a Phase 2 design hole the gap-list memo prioritises as 🔴 must-fix-before-3.1.

Reviewer checklist

  • The loop's _execute_with_decision path goes through RetrievalRouter.route (canonical read_chunk PDP call site), not directly into HybridRetriever.retrieve.
  • No new dependency added between rag-retrieval and rag-query; integration is purely via the Protocol.
  • Every Budget.spend(...) call charges wall_ms via math.ceil (sub-ms iters still bill 1 ms).
  • OTel attributes match the rag.agent_loop.* schema in reference/agent-loop.md.
  • Structured-log event uses the agent_loop.iteration_complete kind.
  • tests/policy/coverage.py passes without an allowlist entry.

Related ADRs and docs