feat(pilot): design-partner pilot program — framework, kit, ragctl pilot, case study (Step 7.4)

[officialCodeWork]

Step 7.4 — Design-partner pilots

Step 7.4 is a GTM/process step. Signing referenceable design partners is an
external, human deliverable (the same way the Step 7.3 external pentest is a
documented process item). What this PR ships in-repo is the machine that makes
every pilot run identically and report objective KPIs pulled from the platform's
own signals — never self-reported. Delivered as one PR, sliced 7.4a–d.

What shipped

Slice	Deliverable
7.4a	Operator runbook (guides/design-partner-pilots.md) + five fill-in templates (docs/pilots/templates/) + ADR-0046
7.4b	Customer-support/KB pilot kit — sample corpus (synthetic-PII handbook + product FAQ + a planted prompt-injection probe), domain-calibrated success criteria
7.4c	ragctl pilot — onboard (per-tenant config + checklist) and report (weekly-KPI dashboard + PASS/FAIL verdict from the feedback / drift / cost signal components behind GET /v1/status/*)
7.4d	Worked case study — the framework run end-to-end on the kit with real ragctl pilot report output

Key decision (ADR-0046)

Every success criterion maps to a platform signal — quality → eval harness +
GET /v1/status/feedback; latency → /v1/status/metrics; security →
/v1/status/compliance. KPIs are pulled, not self-reported — the same
"checkable, not just prose" posture as the Step 6.10 compliance mapping.

Real tooling output (used in the case study)

$ ragctl pilot report --tenant acme-support --week 3
  volume:        60 signals (explicit=43 implicit=17)
  satisfaction:  mean=+0.733  +52/-8  clicks=17   [ok]
  drift:         0/5 monitors drifting   [ok]
  cost:          status=ok ratio=1.00 tokens=820   [ok]
  verdict:       PASS — all online signals green

Maps to the plan's acceptance

• Signed success criteria (quality/latency/integration/security) → the
  success-criteria template, each row
  tied to a signal.
• Weekly status with KPIs → ragctl pilot report + the
  weekly-kpi template.
• Feedback incorporated → the closed intake → triage → incorporate → close loop
  (feedback-log template).
• Reference architecture + customer story → the
  case study.
• 3 live referenceable tenants → tracked as the external GTM acceptance item
  layered on this framework.

Verification

• ruff check + ruff format --check ✅
• mypy --strict (ragctl) ✅
• Full ragctl test suite (+3 new test_pilot.py) ✅
• RAG001 logging gate ✅
• Markdown link-check across all 16 touched docs (441 links) ✅
• No dist/ / SPI / config / core-or-wire-type change → schema-drift +
  policy-coverage gates unaffected (pilot is a pure ragctl reader)

Documentation

• Added: docs/guides/design-partner-pilots.md, docs/pilots/ (index + 5
  templates + the customer-support kit: README, 3-doc corpus, case study),
  docs/adr/ADR-0046-design-partner-pilots.md, docs/reference/pilot.md
• Updated: docs/README.md (guides + reference + a new pilots/ section),
  TRACKER.md (Step 7.4 ✅, 78/84, detailed section, PR-history row)

Deferred / external

Sourcing + signing the real referenceable partners (GTM); the legal +
internal-engineering vertical kits; per-pilot golden-set generation tooling; a
pilot KPI card in the admin console.

🤖 Generated with Claude Code