From 7e5fe2c7e567ee1c68b41094455964555195a05f Mon Sep 17 00:00:00 2001 From: Jonathan Haas Date: Tue, 4 Aug 2026 10:02:57 -0700 Subject: [PATCH] Add architecture, intent, invariants, and ADR documentation suite Introduce the product/system documentation layer alongside module rustdoc: intent, architecture, twenty numbered invariants, data model, spec map, threat model, operator runbooks, and ten ADRs. Wire the suite from docs/README and the project README. --- README.md | 10 + crates/corpus-core/src/lib.rs | 7 + docs/README.md | 21 +++ docs/adrs/0001-two-listeners.md | 34 ++++ docs/adrs/0002-server-owned-writes.md | 27 +++ docs/adrs/0003-immutable-rule-bundles.md | 30 +++ .../0004-typed-edges-not-fuzzy-families.md | 27 +++ docs/adrs/0005-pure-rust-semantic.md | 31 ++++ .../adrs/0006-one-to-one-function-matching.md | 27 +++ docs/adrs/0007-filesystem-cas-trait.md | 28 +++ docs/adrs/0008-observe-only-agent.md | 30 +++ docs/adrs/0009-tiered-scan-isolation.md | 34 ++++ docs/adrs/0010-content-derived-receipts.md | 30 +++ docs/adrs/README.md | 21 +++ docs/architecture.md | 171 ++++++++++++++++++ docs/data-model.md | 141 +++++++++++++++ docs/intent.md | 93 ++++++++++ docs/invariants.md | 52 ++++++ docs/runbooks.md | 93 ++++++++++ docs/spec-map.md | 92 ++++++++++ docs/threat-model.md | 69 +++++++ 21 files changed, 1068 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/adrs/0001-two-listeners.md create mode 100644 docs/adrs/0002-server-owned-writes.md create mode 100644 docs/adrs/0003-immutable-rule-bundles.md create mode 100644 docs/adrs/0004-typed-edges-not-fuzzy-families.md create mode 100644 docs/adrs/0005-pure-rust-semantic.md create mode 100644 docs/adrs/0006-one-to-one-function-matching.md create mode 100644 docs/adrs/0007-filesystem-cas-trait.md create mode 100644 docs/adrs/0008-observe-only-agent.md create mode 100644 docs/adrs/0009-tiered-scan-isolation.md create mode 100644 docs/adrs/0010-content-derived-receipts.md create mode 100644 docs/adrs/README.md create mode 100644 docs/architecture.md create mode 100644 docs/data-model.md create mode 100644 docs/intent.md create mode 100644 docs/invariants.md create mode 100644 docs/runbooks.md create mode 100644 docs/spec-map.md create mode 100644 docs/threat-model.md diff --git a/README.md b/README.md index 49f9363..5d30078 100644 --- a/README.md +++ b/README.md @@ -76,8 +76,18 @@ Details: [docs/deploy.md](docs/deploy.md), [docs/hardening-decisions.md](docs/ha ## Docs +Start at **[docs/README.md](docs/README.md)** (read order: intent → architecture → invariants → data-model → deploy). + | Doc | Contents | |---|---| +| [docs/intent.md](docs/intent.md) | Problem, thesis, users, non-goals | +| [docs/architecture.md](docs/architecture.md) | Processes, trust boundaries, planes, sequences | +| [docs/invariants.md](docs/invariants.md) | Numbered guarantees + enforcing code | +| [docs/data-model.md](docs/data-model.md) | Glossary, tables, relationships | +| [docs/spec-map.md](docs/spec-map.md) | Spec section → implementation | +| [docs/threat-model.md](docs/threat-model.md) | Assets, actors, residual risk | +| [docs/runbooks.md](docs/runbooks.md) | Operator procedures | +| [docs/adrs/](docs/adrs/) | Architecture decision records | | [docs/deploy.md](docs/deploy.md) | Env vars, auth policy, first hunt, gVisor, reverse proxy | | [docs/openapi.json](docs/openapi.json) | HTTP API (`GET /api/v1/openapi.json`) | | [docs/hardening-decisions.md](docs/hardening-decisions.md) | mTLS, spool crypto, sandbox research | diff --git a/crates/corpus-core/src/lib.rs b/crates/corpus-core/src/lib.rs index 67d2724..34d7600 100644 --- a/crates/corpus-core/src/lib.rs +++ b/crates/corpus-core/src/lib.rs @@ -26,6 +26,13 @@ //! //! [`ENGINE_VERSION`] is folded into rule-bundle digests so engine upgrades //! invalidate prior scan caches (spec 14 / 15.4). +//! +//! # Design docs (repo `docs/`) +//! +//! Product intent, system architecture, numbered invariants, data model, +//! ADRs, and operator runbooks live under `docs/` (see `docs/README.md`). +//! Module rustdoc explains local behavior; those pages explain cross-cutting +//! why. pub mod agents; pub mod analyst; diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..2d49397 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,21 @@ +# Corpus documentation + +| Doc | Purpose | +|-----|---------| +| [intent.md](intent.md) | Problem, thesis, users, non-goals | +| [architecture.md](architecture.md) | Processes, trust boundaries, data/control/analysis planes | +| [invariants.md](invariants.md) | Numbered guarantees the code must not violate | +| [data-model.md](data-model.md) | Glossary, tables, relationships | +| [spec-map.md](spec-map.md) | Product-spec section → code location | +| [threat-model.md](threat-model.md) | Assets, attackers, residual risk | +| [runbooks.md](runbooks.md) | Operator procedures for common failures | +| [adrs/](adrs/) | Architecture decision records | +| [deploy.md](deploy.md) | Env vars, auth policy, first hunt | +| [semantic-similarity-design.md](semantic-similarity-design.md) | Function-level matching design | +| [hardening-decisions.md](hardening-decisions.md) | mTLS, spool crypto, scanner tiers (research notes) | +| [detonation-design.md](detonation-design.md) | CAPE adapter design | +| [openapi.json](openapi.json) | HTTP surface (`GET /api/v1/openapi.json`) | + +Read order for a new engineer: **intent → architecture → invariants → data-model → deploy**. + +Decision history: start at [adrs/README.md](adrs/README.md). Milestone research notes (hardening, semantic, detonation) stay as standalone design docs; ADRs capture the cross-cutting choices in short form. diff --git a/docs/adrs/0001-two-listeners.md b/docs/adrs/0001-two-listeners.md new file mode 100644 index 0000000..0712372 --- /dev/null +++ b/docs/adrs/0001-two-listeners.md @@ -0,0 +1,34 @@ +# ADR-0001: Separate admin and agent listeners + +## Status + +Accepted (M6). + +## Context + +Agent authentication requires mTLS with a deployment-only CA. Admin/CLI +traffic uses bearer tokens and often runs on loopback without TLS in dev. +axum-server did not expose peer certificates cleanly for per-route TLS +policy (see hardening notes). + +## Decision + +Run **two listeners**: + +- Plain HTTP(S) admin/CLI on `CORPUS_LISTEN` +- mTLS agent listener on `CORPUS_AGENT_LISTEN` with a hand-built + `tokio_rustls::TlsAcceptor` + +Enrollment (one-time token) remains on the plain listener as the +documented bootstrap path; renew uses mTLS. + +## Consequences + +- Route tables are duplicated for agent ingest/heartbeat/gaps. +- Ops must open/monitor two ports. +- Clear trust split: agent identity ≠ admin token. + +## References + +- [hardening-decisions.md](../hardening-decisions.md) §1 +- `corpus-server` dual bind in `main.rs` diff --git a/docs/adrs/0002-server-owned-writes.md b/docs/adrs/0002-server-owned-writes.md new file mode 100644 index 0000000..70451d8 --- /dev/null +++ b/docs/adrs/0002-server-owned-writes.md @@ -0,0 +1,27 @@ +# ADR-0002: Server owns all durable writes + +## Status + +Accepted (M0). + +## Context + +Endpoints are untrusted. If agents could invent artifact ids or write +shared storage, multi-tenant integrity and rehash guarantees collapse. + +## Decision + +Only `corpus-server` writes Postgres catalog/ledger and CAS objects. +Agents and `corpusctl` are HTTP clients. Agents may write **local** SQLite +and spool only. + +## Consequences + +- All validation (rehash, classify, policy) centralizes on the server. +- Offline agent work is queue-and-forward, not peer-to-peer CAS. +- Offline import still goes through announce/finalize. + +## References + +- [architecture.md](../architecture.md) +- [invariants.md](../invariants.md) §1, §9 diff --git a/docs/adrs/0003-immutable-rule-bundles.md b/docs/adrs/0003-immutable-rule-bundles.md new file mode 100644 index 0000000..5df275d --- /dev/null +++ b/docs/adrs/0003-immutable-rule-bundles.md @@ -0,0 +1,30 @@ +# ADR-0003: Immutable content-addressed rule bundles + +## Status + +Accepted (M0 / M3). + +## Context + +Retro-hunts must name the exact rule set and engine that produced a match +months later. Mutable “current rules” pointers make historical results +unverifiable. + +## Decision + +- Individual rules are compile-validated and stored. +- A **bundle** freezes sorted sources + `COMPILER_CONFIG` + engine version + into a digest. +- Activation is a pointer for forward coverage; digests never rewrite. +- Scan cache keys include bundle digest and engine version. + +## Consequences + +- Rule edits require a new bundle publish. +- Engine upgrades invalidate cache entries by construction. +- Hunts pin digests, not “latest”. + +## References + +- `corpus_core::rules`, `registry` +- [invariants.md](../invariants.md) §6–7 diff --git a/docs/adrs/0004-typed-edges-not-fuzzy-families.md b/docs/adrs/0004-typed-edges-not-fuzzy-families.md new file mode 100644 index 0000000..0dc8716 --- /dev/null +++ b/docs/adrs/0004-typed-edges-not-fuzzy-families.md @@ -0,0 +1,27 @@ +# ADR-0004: Typed edges; weak never merge groups + +## Status + +Accepted (M3a); reaffirmed for semantic weak edges. + +## Context + +ssdeep and weak semantic scores produce useful **leads** but high false +family membership if they auto-cluster. + +## Decision + +- Persist **typed** edges with explicit `edge_type` and `model_version`. +- Only strong types merge variant groups (`merges_groups`). +- `byte_similar`, `shared_provenance`, `semantic_variant_weak` never merge. + +## Consequences + +- Analyst UIs must show weak edges as leads. +- Group membership stays high-precision. +- Spec 28.5 encoded as a unit test. + +## References + +- `similarity::model::merges_groups` +- [invariants.md](../invariants.md) §3 diff --git a/docs/adrs/0005-pure-rust-semantic.md b/docs/adrs/0005-pure-rust-semantic.md new file mode 100644 index 0000000..8a1365e --- /dev/null +++ b/docs/adrs/0005-pure-rust-semantic.md @@ -0,0 +1,31 @@ +# ADR-0005: Pure-Rust semantic matching (no Ghidra) + +## Status + +Accepted (M8). + +## Context + +Spec 16.2/16.5 describes Ghidra BSim-class capability. Embedding a JVM +Ghidra service adds ops weight and Windows CI friction. + +## Decision + +Implement function-level matching in-process with **iced-x86** + goblin: + +- x86-64 only for v1 +- Heuristic + symbol/pdata boundaries +- Mnemonic-family tokens + Jaccard; simhash retained for future indexing + +Document honest limits (no decompiler CFG, no AArch64 yet, uncalibrated τ). + +## Consequences + +- Same process as server (resource bounds required). +- AArch64 and calibration tracked as follow-ups. +- Design doc thresholds locked to `MODEL_V1` via CI test. + +## References + +- [semantic-similarity-design.md](../semantic-similarity-design.md) +- `semantic::*` diff --git a/docs/adrs/0006-one-to-one-function-matching.md b/docs/adrs/0006-one-to-one-function-matching.md new file mode 100644 index 0000000..07ea9fc --- /dev/null +++ b/docs/adrs/0006-one-to-one-function-matching.md @@ -0,0 +1,27 @@ +# ADR-0006: One-to-one greedy function matching + +## Status + +Accepted (similarity investigation foundation). + +## Context + +Many-to-one assignment lets one popular CRT-like function inflate coverage +for many peers (false strong edges). + +## Decision + +- Candidate pairs with Jaccard ≥ τ sorted by score, then offsets. +- Greedy assignment: each function used at most once. +- Strong edges require coverage floors **and** min matched pair count. + +## Consequences + +- Slightly lower scores vs many-to-one inflation (documented in PR notes). +- Contested/unmatched sets available for explainability. +- Deterministic ties for stable receipts. + +## References + +- `semantic::edges::coverage` +- [semantic-similarity-design.md](../semantic-similarity-design.md) diff --git a/docs/adrs/0007-filesystem-cas-trait.md b/docs/adrs/0007-filesystem-cas-trait.md new file mode 100644 index 0000000..457ed69 --- /dev/null +++ b/docs/adrs/0007-filesystem-cas-trait.md @@ -0,0 +1,28 @@ +# ADR-0007: Filesystem CAS + CasBackend trait + +## Status + +Accepted (M0 filesystem; trait in investigation foundation). + +## Context + +Need put-if-absent object storage without requiring S3 for single-node +homelab/dev. Future object stores should not rewrite ingest. + +## Decision + +- Default `FsCas` under `CORPUS_CAS_ROOT` (`objects/`, `staging/`). +- `CasBackend` trait for stage/commit/read/delete. +- `MemoryCas` + `conformance_suite` for tests. +- Digest verification remains in ingest (caller), not inside `commit`. + +## Consequences + +- Ops is directory backup + permissions. +- S3/MinIO can implement the trait later without changing announce flow. +- No built-in CAS GC yet (tracked separately). + +## References + +- `corpus_core::cas` +- [architecture.md](../architecture.md) storage section diff --git a/docs/adrs/0008-observe-only-agent.md b/docs/adrs/0008-observe-only-agent.md new file mode 100644 index 0000000..4bf57e1 --- /dev/null +++ b/docs/adrs/0008-observe-only-agent.md @@ -0,0 +1,30 @@ +# ADR-0008: Observe-only agent + +## Status + +Accepted (M1; product hard rule). + +## Context + +A control plane that can run arbitrary commands on endpoints is an RCE +product. IR value must not require that surface. + +## Decision + +`corpus-agent`: + +- Discovers and uploads code-bearing files +- Reports heartbeats and gaps +- **Never** receives server-side command payloads for execution +- **Never** blocks process start + +## Consequences + +- No remote response actions (kill process, quarantine) in this tree +- Capture failure modes become gap rows, not silent drops +- Security review focuses on steal-credentials and DoS, not command inject + +## References + +- [intent.md](../intent.md) non-goals +- [invariants.md](../invariants.md) §9–10 diff --git a/docs/adrs/0009-tiered-scan-isolation.md b/docs/adrs/0009-tiered-scan-isolation.md new file mode 100644 index 0000000..8582b0b --- /dev/null +++ b/docs/adrs/0009-tiered-scan-isolation.md @@ -0,0 +1,34 @@ +# ADR-0009: Tiered scan isolation + +## Status + +Accepted (M6). + +## Context + +YARA on hostile bytes can attack the scanner. Full microVM isolation is +heavy for default single-node deploy. Need a ladder of controls. + +## Decision + +Tiers via `CORPUS_SCANNER_TIER`: + +| Tier | Mechanism | +|------|-----------| +| `inprocess` | Dev only; same process as API | +| `subprocess` (default) | `corpus-scanner` + seatbelt/landlock where available | +| `gvisor` | `runsc` when configured | + +`CORPUS_MIN_SCANNER_TIER` refuses weaker tiers at startup. + +## Consequences + +- Default is stronger than in-process, weaker than Kata. +- Docs must not claim microVM isolation for default installs. +- Operator can raise the floor on multi-tenant hosts. + +## References + +- [hardening-decisions.md](../hardening-decisions.md) §3 +- [invariants.md](../invariants.md) §14 +- [threat-model.md](../threat-model.md) diff --git a/docs/adrs/0010-content-derived-receipts.md b/docs/adrs/0010-content-derived-receipts.md new file mode 100644 index 0000000..f0c09be --- /dev/null +++ b/docs/adrs/0010-content-derived-receipts.md @@ -0,0 +1,30 @@ +# ADR-0010: Content-derived analysis receipts + +## Status + +Accepted (similarity investigation foundation). + +## Context + +Analysts need “what analyzer/model saw this artifact?” without re-reading +samples. Random UUID receipts make concurrent re-analysis noisy and hard +to upsert. + +## Decision + +- `AnalysisReceipt` JSON without sample bytes +- `receipt_id` = truncated SHA-256 over tenant, artifact, analyzer, + versions, config digest, input sha256, status, function_count +- Upsert on id for idempotent concurrent runs + +## Consequences + +- Same inputs → same receipt row +- Different function counts → different ids (history preserved) +- Edge evidence can embed `receipt_id` for join + +## References + +- `similarity::receipts` +- [invariants.md](../invariants.md) §19 +- migration `0010_receipts_and_cleanup.sql` diff --git a/docs/adrs/README.md b/docs/adrs/README.md new file mode 100644 index 0000000..406c2ee --- /dev/null +++ b/docs/adrs/README.md @@ -0,0 +1,21 @@ +# Architecture decision records + +Short, durable decisions. Milestone research writeups stay in parent docs; +ADRs capture the choice and consequences. + +| ADR | Title | Status | +|-----|-------|--------| +| [0001](0001-two-listeners.md) | Separate admin and agent listeners | Accepted | +| [0002](0002-server-owned-writes.md) | Server owns all durable writes | Accepted | +| [0003](0003-immutable-rule-bundles.md) | Immutable content-addressed rule bundles | Accepted | +| [0004](0004-typed-edges-not-fuzzy-families.md) | Typed edges; weak never merge groups | Accepted | +| [0005](0005-pure-rust-semantic.md) | Pure-Rust semantic matching (no Ghidra) | Accepted | +| [0006](0006-one-to-one-function-matching.md) | One-to-one greedy function matching | Accepted | +| [0007](0007-filesystem-cas-trait.md) | Filesystem CAS + CasBackend trait | Accepted | +| [0008](0008-observe-only-agent.md) | Observe-only agent | Accepted | +| [0009](0009-tiered-scan-isolation.md) | Tiered scan isolation | Accepted | +| [0010](0010-content-derived-receipts.md) | Content-derived analysis receipts | Accepted | + +## Format + +Each ADR: **Context → Decision → Consequences → References**. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..7ed872a --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,171 @@ +# Architecture + +Cross-cutting map of the Corpus control plane. Local module behavior is in +rustdoc; product *why* is in [intent.md](intent.md); guarantees are in +[invariants.md](invariants.md). + +## Processes + +| Binary | Role | Durable writes | +|--------|------|----------------| +| `corpus-server` | HTTP control plane (axum) | Postgres + filesystem CAS | +| `corpus-agent` | Endpoint observer | Local SQLite WAL + encrypted spool only | +| `corpusctl` | Operator CLI | None (HTTP client) | +| `corpus-scanner` | Out-of-process YARA-X helper | None (stdin/stdout job) | + +`corpus-core` is a library: domain logic shared by server and (pure pieces) +by agent/CLI. The server is the only process that commits artifacts. + +## Deployment sketch + +```text +┌─────────────────────────────┐ mTLS :8443 ┌──────────────────────────────┐ +│ corpus-agent (endpoint) │────────────────────▶│ corpus-server │ +│ sensors → capture → spool │ enroll on :8080 │ admin/CLI :8080 │ +│ SQLite queue │ │ agent listener :8443 │ +└─────────────────────────────┘ │ │ │ │ + │ PostgreSQL 16 CAS fs │ +┌─────────────────────────────┐ bearer :8080 │ │ │ │ +│ corpusctl / automation │────────────────────▶│ spawn corpus-scanner │ +└─────────────────────────────┘ │ optional CAPE / Merlin │ + └──────────────────────────────┘ +``` + +Listeners (see [ADR-0001](adrs/0001-two-listeners.md)): + +- **Admin / CLI** — `CORPUS_LISTEN` (default `127.0.0.1:8080`) +- **Agents (mTLS)** — `CORPUS_AGENT_LISTEN` (default `127.0.0.1:8443`) + +## Trust boundaries + +| Boundary | Mechanism | +|----------|-----------| +| Admin API | Loopback free for demos; non-loopback requires `CORPUS_ADMIN_TOKEN` | +| Agent API | mTLS with deployment CA; enrollment token is one-time bootstrap | +| Tenant isolation | `tenant_id` on durable rows; `X-Corpus-Tenant` resolves scope (not auth) | +| Sample execution | YARA in `corpus-scanner` under seatbelt/landlock or gVisor tier | +| Sample egress | CAPE detonation off unless `CORPUS_DETONATION_ENABLED=1` | +| Agent host | Observe-only; spool encrypted at rest; no server-pushed commands | + +Tenant header selects a tenant; it does **not** authenticate the caller. +Put the admin listener behind a gateway in production ([deploy.md](deploy.md)). + +## Planes + +### Data plane (ingest) + +```text +announce(sha256, size, occurrence?) + → disposition: already present | need upload +stage(upload_id, bytes) +finalize(upload_id, sha256, …) + → server rehash (invariant #1) + → CAS commit objects/{tenant}/{sha256} + → artifact + occurrence_event + capture_attempt + → hooks: forward_scan, similarity analyze, optional continuous work +``` + +Code: `corpus_core::ingest`, `corpus_core::cas`. Protocol: spec 11.1 / 11.2. + +### Control plane (rules & hunts) + +```text +rule source → create_rule (compile-validate) + → publish_bundle (immutable digest = sources + compiler config + engine) + → activate_bundle (pointer for forward coverage) + → create_hunt / enqueue / execute + scan_cache key: (artifact, bundle_digest, engine_version) +``` + +Code: `corpus_core::rules`, `registry`, `scan`, `sandbox`, `hunts`. + +Hunt states: `DRAFT → QUEUED → PLANNED → RUNNING → COMPLETED|COMPLETED_PARTIAL|FAILED`. + +### Analysis plane (similarity) + +Post-ingest (and backfill): + +1. **Byte path** — ssdeep, structural hashes, LSH candidates, typed edges + (`similarity::extract`, `fuzzy`, `lsh`, `edges`) +2. **Semantic path** — triage → functions → suppress → 1:1 coverage → strong/weak + (`semantic::*`, model thresholds in `similarity::model::MODEL_V1`) +3. **Receipts** — content-derived analysis_receipt rows (no sample bytes) + +Analyst APIs: neighborhood, export, evidence, cleanup, analyzers. + +### Analyst / automation plane + +- Prevalence, rarity search, opinions (`analyst`, `opinions`) +- Investigation report assembly (`investigate`) +- Detection events from forward/retro/intel (`detect`, `continuous`) +- Webhook triggers on hunt_match / malicious_verdict / detection_event + (`triggers`) +- Optional MCP read-only endpoint (`/mcp`) + +### Integration plane + +| Integration | Direction | Note | +|-------------|-----------|------| +| Merlin | inbound segments/observations | Separate from occurrence ledger | +| OCI | pull layers → ingest blobs | Provenance on artifact | +| CAPE | submit sample → poll report | Findings typed `DYNAMIC_BEHAVIOR` | +| TAXII / hash intel | indicators → exact hash hunt | Continuous when enabled | + +## Multi-tenancy + +- Default tenant uuid `00000000-0000-0000-0000-000000000001` (slug `default`), seeded by migration +- Missing `X-Corpus-Tenant` → default tenant +- CAS object keys: `objects/{tenant_id}/{sha256_hex}` +- Similarity indexes and edges never query without `tenant_id` + +## Storage + +| Store | Contents | +|-------|----------| +| PostgreSQL 16 | Catalog, ledger, rules, hunts, similarity, agents, audit | +| Filesystem CAS | Immutable sample bytes under `CORPUS_CAS_ROOT` | +| Agent SQLite | Local capture queue, sensors cursors, sequence | +| Agent spool | Encrypted staged files pending upload | + +`CasBackend` trait ([ADR-0007](adrs/0007-filesystem-cas-trait.md)): `FsCas` production, `MemoryCas` tests. + +## Crate map + +```text +corpus-server ──uses──▶ corpus-core ◀── corpusctl (DTOs, hash, classify) + │ ▲ + │ spawns │ + ▼ │ +corpus-scanner corpus-agent (dto + hash + classify only for pure bits) +``` + +Module index for `corpus-core` is in the crate rustdoc (`lib.rs`). + +## Sequence: new file on endpoint + +```text +sensor event → agent state enqueue + → stable_read (retry on mutation) + → spool encrypt + → announce / upload / finalize (mTLS) + → server: artifact committed + → forward_scan(active bundles) + → similarity analyze_artifact (+ semantic analyze_and_link) + → optional detection_event / triggers +``` + +## Sequence: rule activation + +```text +publish_bundle → activate + → persistent forward hunt for post-commit scans + → if CORPUS_AUTO_RETRO_ON_ACTIVATE: enqueue full retro-hunt + → continuous_reanalysis row tracks progress +``` + +## Related + +- [invariants.md](invariants.md) +- [data-model.md](data-model.md) +- [adrs/](adrs/) +- [deploy.md](deploy.md) diff --git a/docs/data-model.md b/docs/data-model.md new file mode 100644 index 0000000..f277ef6 --- /dev/null +++ b/docs/data-model.md @@ -0,0 +1,141 @@ +# Data model + +PostgreSQL holds the catalog and ledger. Sample bytes live only in the CAS. +Migrations are ordered under `migrations/`; two files share prefix `0010_` +(Merlin vs receipts) and both apply via `sqlx::migrate!`. + +## Glossary + +| Term | Meaning | +|------|---------| +| **Tenant** | Isolation unit; every business row carries `tenant_id` | +| **Artifact** | One unique sha256 within a tenant; points at a CAS object key | +| **Occurrence** | Observation of an artifact (or intended path) on a host at a time | +| **Capture attempt** | Agent/server record of a capture outcome (including gaps) | +| **Upload session** | Staging handle between announce and finalize | +| **Rule** | Single YARA source, compile-validated, stable id | +| **Bundle** | Immutable set of rules + compiler/engine config, content-digested | +| **Hunt** | Retro or forward scan job over a planned artifact set | +| **Hunt match** | Rule hit on an artifact under a hunt | +| **Scan cache** | Memo of scan outcome keyed by artifact × bundle × engine | +| **Detection event** | First-class “something lit up” without requiring external SIEM | +| **Opinion** | Human verdict on an artifact (separate from analyzer scores) | +| **Similarity feature** | Versioned extracted attribute (ssdeep, import hash, triage, …) | +| **Similarity edge** | Typed link between two artifacts under a model version | +| **Variant group** | Partition of artifacts merged by strong edges | +| **Function row** | Per-function semantic signature for one artifact | +| **Analysis receipt** | Deterministic audit of an analysis pass (no sample bytes) | +| **Finding** | Analyzer output row (e.g. CAPE dynamic behavior) | +| **Merlin observation** | Telemetry event; not a verified file hash | + +## Core tables (by migration) + +### 0001 init + +| Table | Role | +|-------|------| +| `tenant` | id, slug, name, status | +| `artifact` | sha256, size, class, storage_state, object_key, provenance | +| `upload_session` | announce staging | +| `occurrence_event` | ledger: host, agent, boot, sequence, path, times | +| `capture_attempt` | terminal outcomes including gaps | +| `rule` | YARA sources | +| `rule_bundle` / `rule_bundle_rule` | immutable publication | +| `hunt` / `hunt_match` | retro/forward jobs and hits | +| `scan_cache` | (artifact, bundle, engine) → outcome | + +### 0002 agents + +| Table | Role | +|-------|------| +| `enrollment_token` | one-time bootstrap (hashed) | +| `agent` | identity, cert serial, heartbeat fields | + +### 0003–0006 / 0008 / 0010–0011 similarity + +| Table | Role | +|-------|------| +| `similarity_feature` | family/name/version → JSON value | +| `similarity_edge` | src, dst, edge_type, model_version, score, evidence | +| `variant_group` / `variant_group_member` | strong-edge partitions | +| `similarity_function` | func_offset, sig (packed token hashes), version | +| `similarity_lsh_band` | byte fuzzy candidate index | +| `similarity_function_band` | function-level candidate index | +| `analysis_receipt` | content-derived id + body JSON | +| `similarity_cleanup_log` | destructive cleanup audit | + +### 0004 bootstrap / 0005 analyst + +| Table | Role | +|-------|------| +| `intel_indicator` | hash/string IOCs + provenance | +| `artifact_opinion` | human verdicts | +| `trigger_rule` / `trigger_outbox` | webhook automation | +| `audit_event` | control-plane audit | + +### 0007 detonation + +| Table | Role | +|-------|------| +| `analysis_run` | external/internal analyzer job | +| `finding` | typed findings (`DYNAMIC_BEHAVIOR`, …) | + +### 0008–0009 hunts continuous + +| Table | Role | +|-------|------| +| `hunt_job` | worker queue | +| `detection_event` | autonomous detections | +| `continuous_reanalysis` | progress for always-on re-hunt | + +### 0010 Merlin + +| Table | Role | +|-------|------| +| `merlin_segment` | accepted JSONL segment identity | +| `merlin_observation` | events within a segment | + +## Relationships (conceptual) + +```text +tenant + ├── artifact ──┬── occurrence_event + │ ├── similarity_feature / similarity_function + │ ├── similarity_edge (src|dst) + │ ├── variant_group_member + │ ├── analysis_receipt / analysis_run / finding + │ └── detection_event + ├── rule ── rule_bundle ── hunt ── hunt_match + ├── agent ── enrollment_token + ├── intel_indicator + └── merlin_segment ── merlin_observation +``` + +CAS: `object_key = objects/{tenant_id}/{sha256_hex}` → file bytes. The DB +row is not the sample. + +## Identity rules + +| Entity | Identity | +|--------|----------| +| Artifact within tenant | `sha256` (unique per tenant) | +| Occurrence | `(tenant_id, agent_id, boot_id, agent_sequence)` | +| Edge | `(tenant, src, dst, edge_type, model_version)` with `src < dst` | +| Bundle | content digest of sorted sources + compiler config | +| Receipt | SHA-256 truncated over analysis identity fields | +| Function signature row | `(tenant, artifact, func_offset, version)` | + +## JSON evidence conventions + +Edge `evidence` and receipt `body` are JSON. Conventions: + +- Never embed raw sample bytes or full disassembly +- Prefer digests, counts, offsets, version strings +- Supersession flags: `superseded`, `superseded_by`, `superseded_at` +- Semantic edges include `tau`, `matched_pairs`, `receipt_id`, `matching` + +## Related + +- Migrations under `migrations/` +- [architecture.md](architecture.md) +- [invariants.md](invariants.md) diff --git a/docs/intent.md b/docs/intent.md new file mode 100644 index 0000000..e947d48 --- /dev/null +++ b/docs/intent.md @@ -0,0 +1,93 @@ +# Intent: why Corpus exists + +## Problem + +Code-bearing files appear on endpoints, get deleted or overwritten, and +disappear from investigator reach. Threat intel (YARA rules, hash IOCs) +usually arrives **after** the first observation window. Without retained +bytes and a durable observation ledger, retro-hunting is limited to +whatever still sits on disk or in a backup. + +## Thesis + +For each tenant, Corpus keeps: + +1. **One content-addressed copy** of every unique code-bearing artifact + (sha256 of the uploaded bytes is identity). +2. **An append-only occurrence ledger** of where and when those bytes were + observed (host, path, agent, boot id, sequence). +3. **Versioned intelligence** (immutable YARA-X rule bundles, hash + indicators) that can be re-run over retained history. + +When new intelligence lands, matches join back to occurrences for +blast-radius and investigation without re-collecting the fleet. + +## Users + +| Role | Primary interface | Job | +|------|-------------------|-----| +| Endpoint agent | `corpus-agent` | Observe, capture, upload; never block or execute server commands | +| Operator / IR | `corpusctl`, admin HTTP API | Rules, hunts, reports, opinions, triggers | +| Automation | webhooks, MCP (read-only), continuous re-analysis | Fan out on hunt match / verdict / detection | + +## What ships today (capability map) + +Documented in the root [README](../README.md) table. In short: + +- Multi-tenant CAS + announce-before-upload (server rehash) +- Linux/Windows agents (fanotify / RDCW+USN, mTLS, encrypted spool) +- Immutable YARA-X bundles, retro-hunts, forward coverage, scan cache +- Continuous re-analysis on bundle activate / hash intel +- Byte + semantic similarity (typed edges, variant groups, neighborhood) +- Analyst surface: prevalence, rarity, opinions, investigation report +- Optional CAPE detonation (off by default; sample egress explicit) +- Optional Merlin telemetry bridge (observations ≠ verified file hashes) + +## Non-goals + +These are deliberate exclusions, not missing tickets: + +| Non-goal | Rationale | +|----------|-----------| +| EDR / real-time block | Agent is observe-only (spec 10); no kill-switch surface | +| Server-commanded agent execution | Prevents the control plane from becoming RCE | +| Full decompiler / Ghidra JVM | Semantic path is pure Rust x86-64; see [semantic-similarity-design.md](semantic-similarity-design.md) | +| Public malware sharing / multi-tenant sample exchange | CAS keys are tenant-scoped; digests do not cross tenants | +| Treating path/process telemetry as file identity | Merlin observations stay separate from the occurrence ledger | +| Building an in-house sandbox | Detonation is an external adapter ([detonation-design.md](detonation-design.md)) | +| Fuzzy hash as automatic family membership | Spec 28.5; weak edges never merge groups ([invariants.md](invariants.md) §3) | + +## Product loop + +```text +retain bytes + occurrences + │ + ▼ +new intel (bundle activate, IOC, model) + │ + ▼ +re-evaluate history (forward + retro + continuous) + │ + ▼ +detection / hunt match + │ + ▼ +blast radius + investigation + optional trigger +``` + +## Success criteria (operational) + +A deployment is doing its job when: + +- Agents heartbeats are current and coverage gaps are visible, not silent +- Newly committed artifacts receive forward scan under the active bundle +- Activating a bundle enqueues retro coverage over retained history + (`CORPUS_AUTO_RETRO_ON_ACTIVATE`, default on) +- Investigation for a sha256 returns detections, occurrences, and + similarity context without returning sample bytes to the client + +## Related docs + +- [architecture.md](architecture.md) — how processes and planes fit +- [invariants.md](invariants.md) — guarantees that encode this intent in code +- [threat-model.md](threat-model.md) — what we assume about attackers diff --git a/docs/invariants.md b/docs/invariants.md new file mode 100644 index 0000000..cbc9ad2 --- /dev/null +++ b/docs/invariants.md @@ -0,0 +1,52 @@ +# Core invariants + +These are product guarantees. Breaking one is a bug even if tests are green +for an unrelated path. Each maps to code and usually a unit or integration +test. + +| # | Invariant | Primary code | How it fails closed | +|---|-----------|--------------|---------------------| +| **1** | **Server recomputes SHA-256.** Client digest is a hint. Mismatch rejects commit. | `hash::verify_upload`, `ingest::finalize` | `Error::HashMismatch` | +| **2** | **Magic bytes classify; extensions do not.** | `classify::classify` | Unknown class; no “trust .exe” path | +| **3** | **Weak edges never merge variant groups.** Only `exact_copy`, `normalized_equivalent`, `semantic_variant_strong` call union. | `similarity::model::merges_groups` | Test `fuzzy_never_merges_groups` | +| **4** | **Analyst graph APIs never return sample bytes.** Neighborhood, export, evidence, receipts: digests + metadata only. | `neighborhood`, `export`, `semantic::edges::function_pair_evidence`, `receipts` | Evidence strips bulky fields; size caps | +| **5** | **Every durable query is tenant-scoped.** | All `corpus_core` SQL | Missing tenant → wrong default or NotFound, not cross-read | +| **6** | **Rule bundles are immutable.** Digest covers sources + `COMPILER_CONFIG` + engine version. | `rules`, `registry::publish_bundle` | New engine/config → new digest; old cache entries unused | +| **7** | **Scan cache identity includes engine version.** | `scan::ScanCacheKey`, `ENGINE_VERSION` | Engine bump invalidates cache semantics | +| **8** | **Hunt matches insert idempotently.** | `hunts` match insert | `ON CONFLICT DO NOTHING` / unique key | +| **9** | **Agents are observe-only.** No server-command channel; no process kill API. | `corpus-agent` | Architecture; no route exists | +| **10** | **Coverage gaps are data.** Failed capture is recorded, not dropped. | `capture_attempt`, agent gap batching | Spec 2.2 taxonomy | +| **11** | **Occurrence identity is (tenant, agent_id, boot_id, agent_sequence).** | `ingest` occurrence insert | Idempotent on conflict | +| **12** | **Packed/virtualized binaries do not get confident semantic edges.** | `semantic::triage`, `edges::extract_and_store` | `block_semantic` → limitation receipt, empty functions | +| **13** | **Similarity model thresholds are single-sourced.** Design doc matches `MODEL_V1`. | `similarity::model` | Test `design_doc_matches_model_config` | +| **14** | **Hostile-sample isolation is tiered and explicit.** Default subprocess+OS sandbox; gVisor optional floor. MicroVM-class is documented future, not claimed today. | `sandbox`, `CORPUS_SCANNER_TIER` | Weaker than min tier → refuse start when configured | +| **15** | **Sample egress is opt-in.** Detonation requires `CORPUS_DETONATION_ENABLED`. | `detonate` | Default off | +| **16** | **Merlin telemetry is not file identity.** Observations/segments do not invent artifact rows. | `merlin` | Separate tables; join is best-effort | +| **17** | **Supersession does not delete history.** Old model edges get evidence flags; new version writes new rows. | `similarity::invalidation` | Auditability | +| **18** | **Legal hold blocks destructive similarity cleanup.** | `similarity::lifecycle` | `Error::Conflict` unless dry-run | +| **19** | **Analysis receipts store no sample bytes.** Content-derived id; digests and counts only. | `similarity::receipts` | Schema + serializers | +| **20** | **Enrollment tokens are one-time and hashed at rest.** | `agents::create_enrollment_token` | Plaintext returned once | + +## Edge type reference (invariant #3) + +| Edge type | Merges groups? | +|-----------|----------------| +| `exact_copy` | yes | +| `normalized_equivalent` | yes | +| `semantic_variant_strong` | yes | +| `semantic_variant_weak` | no | +| `byte_similar` | no | +| `shared_provenance` | no | + +Thresholds for semantic classification: [semantic-similarity-design.md](semantic-similarity-design.md) and `MODEL_V1`. + +## Changing an invariant + +1. Update this file and the enforcing code in the same PR. +2. Add or adjust a test that fails if the invariant regresses. +3. If the change is intentional product movement, add an ADR under [adrs/](adrs/). + +## Related + +- [architecture.md](architecture.md) +- [spec-map.md](spec-map.md) diff --git a/docs/runbooks.md b/docs/runbooks.md new file mode 100644 index 0000000..6f14566 --- /dev/null +++ b/docs/runbooks.md @@ -0,0 +1,93 @@ +# Operator runbooks + +Assumes `corpusctl` pointed at the server (`CORPUS_SERVER_URL`, +`CORPUS_ADMIN_TOKEN`, optional `CORPUS_TENANT`). + +## Hunt stuck in RUNNING + +1. `corpusctl hunts` (or `GET /api/v1/hunts/{id}`) — note `scanned`, + `timed_out`, `failed`, `error`. +2. Check server logs for scanner timeouts (`SCAN_TIMEOUT` default 10s per + artifact). +3. If using async worker: confirm `hunt_job` claim loop is running (single + node: server process must be up). +4. Poison artifact: identify last sha256 from logs; optional + `CORPUS_HUNT_SYNC=1` for in-request runs during debug. +5. COMPLETED_PARTIAL is valid when timeouts/failures occurred — not a + deadlock. + +## Agent coverage gaps spiking + +1. `GET /api/v1/coverage/gaps` or corpusctl fleet/gaps view. +2. Classify `capture_attempt.terminal_outcome` / detail codes: + - mutation during read → writer churn; expected under installs + - spool full / too large → raise limits or free disk + - hash mismatch → client/server skew or corrupt stage +3. Confirm heartbeat: `GET /api/v1/agents/{id}` last_seen. +4. Sensor path: Linux fanotify privileges; Windows USN journal id change + forces reconcile (expected after some volume ops). + +## Missing similarity edges + +1. Confirm artifact class is `pe`/`elf`/`macho` and storage committed. +2. Receipts: `GET /api/v1/artifacts/{id}/receipts` — status `limitation` with + packed/virtualized summary means semantic was blocked by design + ([invariants.md](invariants.md) §12). +3. Byte path: run `corpusctl similarity backfill` for the tenant. +4. Analyzers: `GET /api/v1/similarity/analyzers` — versions active. +5. Neighborhood: `corpusctl similarity neighborhood ` with + `min_score=0` to see weak leads. + +## Packed binary / no semantic match + +Expected when triage `block_semantic` is true (high entropy, RWX+entropy, +packer markers + corroboration). Use byte_similar / intel / YARA instead. +Do not lower `packed_entropy_limit` without a model version bump +([invariants.md](invariants.md) §13). + +## Legal hold blocked cleanup + +`POST .../similarity-cleanup` returns conflict when +`artifact.provenance.legal_hold` is true. Dry-run still returns counts. +Clear hold only with dual-control process outside Corpus, then re-run +with `dry_run=false`. + +## Rotate admin token + +1. Generate new secret; set `CORPUS_ADMIN_TOKEN` on server; restart. +2. Update corpusctl/automation secrets. +3. Old token stops working immediately (no dual-token window in tree). + +## Rotate deployment CA / agent certs + +1. `corpusctl ca init` paths under `CORPUS_CA_DIR` (see deploy). +2. Re-enroll agents or use `POST /api/v1/agents/renew` on mTLS listener. +3. Agents with expired certs fail mTLS — expect gap until renewed. + +## Enable detonation safely + +1. Read [detonation-design.md](detonation-design.md) and threat model §sample egress. +2. Set `CORPUS_CAPE_URL` + `CORPUS_CAPE_TOKEN`. +3. Set `CORPUS_DETONATION_ENABLED=1` only when CAPE is authorized to + receive org samples. +4. Keep `CORPUS_DETONATION_AUTO` off until opinion policy is reviewed. +5. Confirm `audit_event` rows on submit. + +## Database / migrations + +1. Server applies `sqlx` migrations at boot (`corpus_core::db::migrate`). +2. Take Postgres backup before upgrading builds that add migrations. +3. Dual `0010_*.sql` files are intentional (Merlin + receipts); both must apply. + +## Postgres down + +1. Server will fail health/DB operations; agents spool locally until upload + succeeds (queue + encrypted spool). +2. Restore DB; restart server; agents drain backlog. +3. CAS files are independent of DB — do not delete `CORPUS_CAS_ROOT` when + restoring DB or digests will 404 on read. + +## Related + +- [deploy.md](deploy.md) +- [architecture.md](architecture.md) diff --git a/docs/spec-map.md b/docs/spec-map.md new file mode 100644 index 0000000..7db95ec --- /dev/null +++ b/docs/spec-map.md @@ -0,0 +1,92 @@ +# Spec map + +Product requirements are referenced in code as “spec §N” / “spec N.M”. +The full product specification is maintained outside this repository +(EvalOps internal). This map is the in-repo index: **section → intent → +implementation**. + +If you change behavior that a row claims, update the row in the same PR. + +## Identity & corpus (2–3, 11–12) + +| Spec | Topic | Code / docs | +|------|-------|-------------| +| 2.2 | Coverage gaps are first-class data | `capture_attempt`, `agents::record_gaps`, agent gap batching | +| 2.3 | Code-bearing classification; extensions not authority | `classify`, corpusctl import filters | +| 3 | SHA-256 artifact identity | `hash`, `ingest::finalize` | +| 5.5 | Human opinions separate from scores | `opinions`, `artifact_opinion` | +| 11.1–11.2 | Announce-before-upload, two-phase commit | `ingest` | +| 12.2 | Analysis run records | `analysis_run` (detonation migration) | +| 12.4 | Boot id + per-boot sequence | agent `state`, occurrence columns | + +## Agents (10) + +| Spec | Topic | Code / docs | +|------|-------|-------------| +| 10 | Observe-only endpoint agent | `corpus-agent` | +| 10.1 | Enrollment, gap reporting | `agents` | +| 10.4 | Capture state machine | agent `state` | +| 10.5 | Stable read / no symlink follow | agent `stable_read` | +| 10.6 | Classification at capture | shared `classify` | +| 10.7 | Baseline walk | agent `baseline` | +| 10.8 | Capture priorities | agent `state` priorities | +| 10.9 | Size / spool policy defaults | agent `config` | +| 10.10 | Sensor fallbacks (poll / RDCW) | agent `sensors` | +| 10.11 | Heartbeat / fleet health | `agents::heartbeat`, corpusctl | + +## Rules & hunts (14–15) + +| Spec | Topic | Code / docs | +|------|-------|-------------| +| 14.3–14.5 | Rule validate, immutable bundles | `rules`, `registry` | +| 14.6 | Hash intel → exact hunt | `intel` | +| 15 | Retro-hunt engine | `hunts` | +| 15.1–15.2 | Plan/execute; COMPLETED_PARTIAL on timeout | `hunts::execute_hunt` | +| 15.4 | Scan cache key | `scan` | +| 15.9 | Forward coverage on commit / bundle | `ingest` hooks, `registry` activate | + +## Similarity (16, 28.5) + +| Spec | Topic | Code / docs | +|------|-------|-------------| +| 16 | Feature families, edges, groups | `similarity::*` | +| 16.2 | Function-level features | `semantic::extract`, `features` | +| 16.4 | Typed edges | `similarity::model::edge_type` | +| 16.5 | Coverage aggregation, suppression | `semantic::edges`, `suppress` | +| 16.6 | Variant groups | `similarity::edges::union_groups` | +| 16.7 | Packed binaries: no false confidence | `semantic::triage` | +| 28.5 | Fuzzy alone ≠ family membership | `merges_groups`, tests | + +## Investigation & evidence (17, 20, 24) + +| Spec | Topic | Code / docs | +|------|-------|-------------| +| 17.1 | Blast radius | `report` | +| 17.2 | Verification tasks (later) | noted in DTOs; not full product yet | +| 17.4 | Evidence typing (e.g. DYNAMIC_BEHAVIOR) | `finding`, detonation | +| 20.6 | External analysis declares sample egress | `detonate`, env flags | +| 24.3 | Audit events | `audit_event` | + +## Hardening notes + +Isolation class for hostile samples (spec invariant #14 in product language) +is implemented as **tiered** scanner isolation, not microVM-by-default. +See [hardening-decisions.md](hardening-decisions.md) and +[invariants.md](invariants.md) §14. + +## Open product gaps (tracked as GitHub issues historically) + +Examples still called out in code/design: + +- Semantic calibration fixtures (#16) +- AArch64 semantic (#18) +- CFG / unwind features (#19–#21) +- CAS GC (#41) +- BinExport (#42) + +Prefer linking issues from ADRs or design docs when closing a gap. + +## Related + +- [intent.md](intent.md) +- [invariants.md](invariants.md) diff --git a/docs/threat-model.md b/docs/threat-model.md new file mode 100644 index 0000000..3e5fefa --- /dev/null +++ b/docs/threat-model.md @@ -0,0 +1,69 @@ +# Threat model + +Scope: a self-hosted Corpus deployment (server + Postgres + CAS + agents). +This is an engineering threat model, not a compliance artifact. + +## Assets + +| Asset | Sensitivity | Store | +|-------|-------------|--------| +| Sample bytes (malware / dual-use code) | High | CAS filesystem | +| Occurrence ledger (host paths, agents) | High (privacy + IR value) | Postgres | +| Admin bearer token | Critical | Env / secret manager | +| Deployment CA + agent client certs | Critical | `CORPUS_CA_DIR`, agent state | +| Spool encryption keys | High | OS key wrap / 0600 file | +| Rule sources / intel indicators | Medium | Postgres | +| MCP token | High if exposed | Env | + +## Actors + +| Actor | Goal | +|-------|------| +| Compromised endpoint | Exfil agent credentials; flood server; read other hosts’ data (should fail) | +| Malicious sample author | Crash scanner/server; escape analysis sandbox; RCE via rule engine | +| Network attacker (path to admin port) | Call admin API without token; MITM agent traffic | +| Malicious tenant principal | Read other tenants’ artifacts (should fail) | +| Operator error | Bind admin without token; enable detonation toward untrusted CAPE | + +## Controls (mapped) + +| Risk | Control | Residual | +|------|---------|----------| +| Cross-tenant sample read | `tenant_id` on keys + SQL; CAS path namespaced | App bug could still join wrong; review SQL carefully | +| Admin API on network | Non-loopback refuses start without `CORPUS_ADMIN_TOKEN` | Token theft = full admin; use short-lived secrets + gateway | +| Agent impersonation | mTLS deployment CA; enrollment one-time | Stolen agent cert = that agent’s ingest rights | +| Malicious PE crashes host process | Out-of-process `corpus-scanner`; seatbelt/landlock or gVisor | **Not** a malware sandbox; assume escape is possible under determined exploit | +| Sample leaves org | Detonation default off; explicit enable | Misconfiguration enables CAPE egress | +| Spool theft on disk | XChaCha20-Poly1305 + OS key wrap | Linux file-key mode is weaker than Keychain/TPM | +| Path confusion / symlink | Stable read opens nofollow | Platform-specific edge cases | +| Tenant header spoofing | Header is **not** auth; auth is token/mTLS | Shared admin token sees all tenants it can address | + +## Explicit non-claims + +- Subprocess + landlock/seatbelt is **not** equivalent to a detonation VM. +- gVisor tier reduces risk; it is still not Kata/Firecracker-class isolation + (documented as future in hardening notes). +- Similarity and YARA matches are leads, not ground truth about host compromise. +- CAPE findings mean “observed in that sandbox,” not “executed on the endpoint.” + +## Logging & forensics + +- `audit_event` for control actions (including detonation requests) +- `capture_attempt` for gaps +- `similarity_cleanup_log` for destructive derived-row cleanup +- Analysis receipts for “what model saw this artifact” + +## Review triggers + +Revisit this document when: + +- Adding a new network listener or auth mode +- Enabling any sample egress path +- Changing scanner isolation tiers +- Introducing multi-tenant admin delegation + +## Related + +- [hardening-decisions.md](hardening-decisions.md) +- [deploy.md](deploy.md) +- [invariants.md](invariants.md) §14–15