Skip to content

agent-quickstart.md

Latest commit

 

History

History
159 lines (128 loc) · 5.5 KB

agent-quickstart.md

File metadata and controls

159 lines (128 loc) · 5.5 KB
Raw

Agent quickstart (Step 3.6)

A five-minute tour of the agent runtime: drive a goal-directed plan → act → observe → finalize loop over REST SSE, gRPC, and the CLI — no credentials required. The default gateway wires a credential-free HeuristicController (retrieve once, then finalize), so everything below runs against the in-process noop stack.

1. The fastest path — ragctl agent

uv run ragctl agent "What is retrieval-augmented generation?"

This boots an in-process gateway (build_app() with noop wiring) and streams the agent event log:

goal:    'What is retrieval-augmented generation?'
tenant:  ragctl-local
events:
  run_started
  phase_changed     -> planning (step 0)
  tool_started      -> retrieve
  tool_completed    -> ok=True chunks=0
  checkpoint_saved
  phase_changed     -> finalizing (step 1)
  answer_delta      -> 'No supporting passages were found in the knowledge base for: …'
  run_completed:
    status:       final_answer
    steps:        2
    chunks:       0
    final_answer: '…'
    elapsed_ms:   1.2
    run_id:       …
  [DONE]

Useful flags: --tenant/-t, --principal/-p, --corpus/-c (repeatable), --top-k/-k, --max-steps, --budget-tokens, --budget-iter.

2. REST — POST /v1/agent (SSE)

Start a gateway:

uv run uvicorn rag_gateway.app:build_app --factory --port 8000

Post a goal and stream the frames (curl -N disables buffering):

curl -N http://127.0.0.1:8000/v1/agent \
  -H 'content-type: application/json' \
  -H 'x-tenant-id: acme' -H 'x-principal-id: alice' \
  -d '{
    "tenant_id": "acme",
    "principal_id": "alice",
    "goal": "Summarise our incident-response policy.",
    "top_k": 8,
    "max_steps": 6,
    "budget_tokens": 4000
  }'

Each line is one AgentEvent as data: {json}, closed by data: [DONE]:

data: {"kind":"run_started","run_id":"…"}
data: {"kind":"phase_changed","phase":"planning","step_index":0}
data: {"kind":"tool_started","tool_call":{"tool":"retrieve","args":{"query":"…"}}}
data: {"kind":"tool_completed","tool_result":{"ok":true,"chunk_refs":[…]}}
data: {"kind":"checkpoint_saved","step_index":1}
data: {"kind":"phase_changed","phase":"finalizing","step_index":1}
data: {"kind":"answer_delta","text":"Based on 8 retrieved passage(s), …"}
data: {"kind":"run_completed","result":{"status":"final_answer", …}}
data: [DONE]

The SSE framing mirrors /v1/chat/completions, so the same client plumbing consumes both surfaces.

3. gRPC — Converse

import asyncio, grpc
from rag_gateway._grpc_gen import rag_pb2, rag_pb2_grpc

async def main() -> None:
    async with grpc.aio.insecure_channel("127.0.0.1:50051") as channel:
        stub = rag_pb2_grpc.RagServiceStub(channel)
        req = rag_pb2.ConverseRequest(
            tenant_id="acme", principal_id="alice",
            goal="Summarise our incident-response policy.",
            top_k=8, max_steps=6, budget_tokens=4000,
        )
        async for event in stub.Converse(req):
            kind = rag_pb2.AgentEventKind.Name(event.kind)
            if kind == "AGENT_EVENT_KIND_ANSWER_DELTA":
                print("answer:", event.text)
            elif kind == "AGENT_EVENT_KIND_RUN_COMPLETED":
                r = event.result
                print("status:", rag_pb2.StopReason.Name(r.status))
                print("final:", r.final_answer)

asyncio.run(main())

Run a gRPC server with uv run ragctl grpc-serve (or serve() from rag_gateway.grpc_server). Converse is server-streaming and carries the same event shape as REST, frame-for-frame.

4. Budgets — make a run stop early

Budgets are four independent dimensions (budget_tokens, budget_dollars, budget_wall_ms, budget_iter); null/omitted means uncapped. Cap iterations at one and the run stops after a single step:

curl -N http://127.0.0.1:8000/v1/agent \
  -H 'content-type: application/json' \
  -d '{"tenant_id":"acme","principal_id":"alice","goal":"…","budget_iter":1}'

The terminal frame carries "status":"budget_iter" and exactly one step. max_steps is a separate, always-present safety ceiling (status:"max_iterations" when it bites first).

5. Resume a run

Every step is checkpointed (InMemoryCheckpointStore by default). Resume a crashed/paused run by passing its run_id:

curl -N http://127.0.0.1:8000/v1/agent \
  -d '{"tenant_id":"acme","principal_id":"alice","goal":"…","resume_run_id":"<run_id>"}'

Resuming an unknown run_id is reported in-band as a terminal run_failed frame (with error_code / message), not a transport error. Note the in-memory store is process-local and not durable across restarts; a Postgres/Redis-backed store is the production drop-in.

What's happening under the hood

  • The default HeuristicController retrieves once for the goal, then finalizes — no model needed. Production swaps an LLMController sourced from rag.yaml.
  • The retrieve tool runs the same policed retrieval path as /v1/query, so ACL/PII governance applies unchanged.
  • The final answer is checked against PolicyEngine.egress_text before it hits the wire — a deny withholds it, a transform redacts it.

See also