feat(automations): add conformance plane with vectors, runner, SLO gates, and diagnostics - #23
Open
CompleteDotTech wants to merge 20 commits into
Open
Conversation
…854) (OpenCoven#861) Adds one dated status/decision record for issue OpenCoven#854 in docs/superpowers/plans/. Independently verified against main @ 1364cec (2026-08-30): the coven#816 foundation series (PR OpenCoven#846, PR OpenCoven#847, parts 6-8 commits) landed the durable local scheduler core - definitions, SQLite definition/occurrence/run records with unique occurrence fencing, RRULE planning, bounded leases with expiry recovery, latest-only misfire / overlap refusal, the 60s daemon tick with scheduled dispatch, shared launch path, health and run history, non-destructive paused legacy import, and coven.automations.* control actions. Records what is still absent for v1 (versioned protocol schemas, authority/receipt binding, deterministic-time contract, SDK surface, conformance/chaos/SLO gates, live Beads mutation), the gate-by-gate verdict, and the critical path. The program issue remains open; this record closes no issue. Refs OpenCoven#854. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
…oven#815) (OpenCoven#871) * docs: plan cryptographic fresh-user and biometric assurance proofs (coven#815) Specify COVEN-ASSURANCE/1, an independently verifiable step-up proof so request authentication can satisfy FreshUserVerification/FreshBiometric grant requirements without trusting client-asserted assurance levels. The plan defines: possession/step-up key separation; binding the optional step-up authorization key and its declared assurance class into the pairing-v2 transcript; a separate authorization-key store; canonical proof bytes over device, grant, revocation epoch, exact request or DeviceActionIntent digest, server-issued single-use challenge, window, and requested assurance; server-side effective assurance computed as min(claimed, class ceiling) and passed to DeviceGrant::authorize; fail-closed possession-only degradation; rotation/revocation semantics; iOS/Android platform mapping; a self-verifying synthetic golden vector; and TypeScript/JSON Schema for implementers. Every claim cites the current mobile_memory code path it builds on. No code changes: this is the design artifact the issue asks for. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * docs(mobile): specify COVEN-ASSURANCE/1 step-up assurance proofs Add docs/design/mobile-assurance-step-up-v1.md, the plan and implementation contract for issue OpenCoven#815: a separately enrolled, platform-policy-protected step-up authorization key, bound into the pairing-v2 transcript before grant issuance, whose fresh signatures over canonical COVEN-ASSURANCE/1 bytes (device, grant, revocation epoch, exact request or DeviceActionIntent digest, single-use server challenge, validity window, requested assurance) are the only way to raise assurance above possession. The server verifies the signature, computes effective assurance itself, and passes it to DeviceGrant::authorize; absent/invalid/expired/replayed proofs fail closed. Covers storage separate from the possession key, replay protection independent of request nonces, rotation/revocation semantics, iOS/Android key-policy mapping, a security-invariant mapping, TypeScript types + JSON Schema, and a portable golden vector. Design artifact only; no behavior change. Refs OpenCoven#815 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
…heck (coven#859) (OpenCoven#862) Establish the GitHub-side half of the OpenCoven#859 operational delivery graph: - docs/roadmaps/coven-automations-v1.md: program roadmap with canonical tracker roles, sync metadata, P0/P1/P2 policy, generated outcome-mapping table, dependency graph, release gates, active blockers, and drift detection usage. The mapping table block is generated by docs/roadmaps/drift-check.mjs --render and verified on every run. - docs/roadmaps/coven-automations-v1.mapping.json: machine-readable one-to-one Bead <-> GitHub mapping for OpenCoven#854, OpenCoven#816, OpenCoven#855, OpenCoven#856, OpenCoven#857, and OpenCoven#858, with explicit dependencies, acceptance gates, dispositions, and evidence links. Bead IDs stay null (warn-level W010) until provisioning lands through OpenCoven/coven-cave#5220 per the operational correction on OpenCoven#859; no competing Beads store is created in this repository. - docs/roadmaps/drift-check.mjs: credential-free, offline drift check covering state, priority, parent/dependency, evidence, generated-mirror, duplicate/missing mapping, cycle, and sensitive-payload drift classes, with --beads-export cross-checking and --selftest proving detection. - docs/superpowers/plans/2026-08-30-issue-859-coven-automations-v1-...md: dated status/decision record: what exists on main (automations series landed 2026-08-28 via PRs OpenCoven#846/OpenCoven#847), pre-change integrity report, decisions D1-D7, verdict against the OpenCoven#859 acceptance criteria, remaining work, critical path, and the initial evidence packet. Refs OpenCoven#859 fix: keep automation tracker docs scanner-safe Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
… claims * docs: consolidate Coven security policy, threat boundary, and support claims Replace the duplicated SECURITY.md (repository policy plus a copied organization-wide disclosure addendum) with one Coven-specific operational security contract that visibly separates: - supported release family and security-supported surfaces, with experimental/disabled surfaces (AFS mount backend, OpenClaw plugin, remote transports) marked explicitly; - enforced properties today, each tied to its normative contract and verification family (Rust authority checks, capability advertisement, pre-effect validation, owner-protected local transport, redaction and artifact defaults, secret/privacy guards, explicit mutation replay); - residual risk and safe configuration (same-user trust vs sandboxing, raw-artifact opt-in, retention limits, untrusted harnesses, AgentFS mount posture synced with OpenCoven#779); - private reporting via GitHub Security Advisories as the primary path, with no personal-channel dependency and no unsupported response-time commitment, plus scoped third-party dependency/provider handling; - design goals (session/memory/agent isolation) moved out of guarantees, cross-referencing OpenCoven#803/OpenCoven#804 instead of claiming unimplemented A2A isolation. Removes the stale OpenTrust-scope addendum, the personal Discord DM reporting path, and the 48-hour/14-day response promises that no accountable process currently backs. Refs OpenCoven#808 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * fix: document tokenized invite URL privacy rule Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
…enCoven#785) (OpenCoven#867) * docs(pairing): plan TUI QR bootstrap and E2EE mobile pairing (refs OpenCoven#785) Add the implementation plan for issue OpenCoven#785: the coven device command family, canonical CBOR pairing offer with Universal Link encoding, Noise_XK handshake with QR-pinned host authentication, rendezvous relay MVP on coven-relay, six-word human verification derived from the handshake hash, scope selection and capability binding, the adversarial test matrix, and the maintainer decision points. Extends the accepted mobile-device-trust architecture and elaborates PR 3 + the rendezvous slice of PR 4 of the pairing delivery train. Docs-only change; cross-links the delivery plan to the new plan. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * chore(pairing): retrigger CI for the plan PR (no content change) The pull_request opened event raced the fork's lazy workflow registration, so no run was created. This empty commit re-emits the synchronize event; it changes no content. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
* docs: point install guidance at canonical coverage The canonical install, platform, deployment, uninstall, and install-debugging guides are merged in OpenCoven/coven-docs, so the duplicate local pages under docs/install/ become stable pointers per the Wave A cleanup plan. docs/install/development-channels.md keeps its local content because no canonical page covers pre-release channels yet. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * docs: point onboarding pages at canonical guidance Duplicate local onboarding pages (quickstart, first session, onboarding, showcase, doctor) become pointers to the canonical getting-started and doctor routes. coven-tui.md keeps its legacy-TUI contract and its root-absolute links are fixed to the retained local session-lifecycle contract and the canonical CLI reference. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * docs: shrink the README to a landing page and codify the public-docs boundary The README stops duplicating the canonical manual and keeps the landing-page facts: what Coven is, one install and quick-start path, canonical links, the source-adjacent ownership table, contributor rules, and security. DOCS-MAINTENANCE.md gains the enforceable public-doc directory boundary (canonical pointers or source-adjacent exceptions with stated ownership reasons), and docs/index.md links to it. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * ci: retrigger pull request checks after enabling fork Actions No file changes. The pull request was opened before Actions was enabled on this fork, so no workflow run was created for its head SHA. This empty commit fires a pull_request synchronize event so the CI matrix can report. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
…Coven#863) Investigate issue OpenCoven#807 against main at 1364cec (2026-08-30) and record a dated status/decision document in the repo's record location (docs/superpowers/plans/). Facts recorded: the existing non-gating benchmark/chaos corpus, health and recovery surfaces, release certification (certify-release.sh, 2026-08-29), packaged-artifact journey E2E, release-stress workflow, and AFS gating posture, each with evidence links and dates; a verdict against all six acceptance criteria (not satisfied on main; partial on separation-by-convention, structured receipts, and privacy-by-design); and a dependency-ordered critical path. No measurements, targets, or SLOs are invented. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
* docs: record issue 670 docs program status on main Add a dated status/decision record for OpenCoven#670 at docs/superpowers/plans/2026-08-30-issue-670-docs-program-status.md. It records, with evidence links and dates: the merged progressive-help implementation (OpenCoven#834), the merged packaged first-session E2E (OpenCoven#835, closing OpenCoven#777), the canonical journey and docs CI live in OpenCoven/coven-docs (OpenCoven#775/OpenCoven#778 state), the still-open single-source cleanup of this repository (OpenCoven#776: README shrink, residual local pages, missing CI ownership enforcement), and the open OpenCoven#779 certification matrix. It gives a per-criterion verdict against OpenCoven#670's nine acceptance criteria, what remains, and the critical path. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * chore: retrigger vehicle CI after a dropped pull_request event The fork vehicle's Actions went live at 2026-08-30T15:54Z, after PR #10 opened at 15:46Z, so the opened/ready_for_review events never started a run (0 check-runs on the head SHA ~20 minutes after creation). Push an empty signed commit to fire the synchronize event; no file changes. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * docs: keep issue 670 record clear of the privacy phone detector The Coven privacy guard's phone_number rule (E.164-like plus-digit tokens) flags the diff-stat notation (+1828), (+464), (+279), (+323), and (+2686) in the OpenCoven#670 status record. Restate those as insertion and deletion counts; no content changes. Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
* test(agents): characterize input-guardrail ingress at handoff boundaries Characterizes the coven-agents runner's input-guardrail behavior so the handoff bypass in OpenCoven#803 is visible from test output before the enforcement change: - direct-entry rejection vs handoff entry to the same target (parity regression: currently the handoff target's input guardrails never run) - handoff target GuardrailChecked event ordering (after Handoff, before the target's first ModelRequested) and the exact evaluated input - multi-hop A->B->C per-boundary checks and a target rejection at hop C - input-guardrail implementation error vs policy rejection, direct and handoff (GuardrailFailed keeps its own message and source) - handoff/tool action exclusivity stays pinned Tests encode the corrected expectation and are paired with the runner change in the next commit; compile proof and green runs defer to CI (this workspace has no Rust toolchain). Refs OpenCoven#803 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * fix(agents): enforce target input-guardrail parity across handoffs Closes the ingress bypass in Runner::run_loop (OpenCoven#803): a handoff target's input guardrails were never evaluated, so an agent reached through a model handoff could consume input that direct entry to it would reject. - extract the input-guardrail evaluation into one shared path, Runner::check_input_guardrails, used by direct starts and by every handoff boundary - evaluate the handoff target's input guardrails against the original user input — the same bounded string a direct start would check, never a serialized transcript — after the Handoff item/event is recorded and before the target's first model turn or tool execution - a target rejection or guardrail implementation error fails the run (RunFailureKind::InputGuardrail) before the target's model call, so the target receives no model request and executes no tool; the GuardrailFailed/GuardrailRejected distinction is preserved - emit GuardrailChecked with the target identity and Input stage; exactly one terminal RunCompleted/RunFailed pairing is preserved - multi-hop A->B->C now applies each hop's ingress policy at its own boundary Documented compatibility contract: target ingress is evaluated over the original/root user input only. The structured task/context manifest for delegated invocations is the successor contract in OpenCoven#804; this patch is not complete A2A security. Handoff/tool exclusivity, fail-fast topology validation, handoff limits, session semantics, and output-guardrail behavior are unchanged. cargo fmt/clippy/test verification defers to CI (no Rust toolchain in this workspace); formatting verified with a standalone rustfmt binary. Refs OpenCoven#803 Refs OpenCoven#804 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * fix: keep handoff checks warning-free Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
Issue OpenCoven#774 asks Coven to keep default help concise, preserve every command through the grouped `coven help --all` view, and expose a deterministic JSON help contract. The executable side of that work landed in OpenCoven#834; the contract itself remained undocumented for consumers. Document the contract in the developer core-functionality guide (schemaVersion 1 payload shape, determinism and leak guarantees, drift-fails-loudly rule) and surface it to script authors in the automation JSON guide. Extend scripts/cli-docs-test.mjs so the docs guard fails when the contract documentation regresses. Refs OpenCoven#774 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
…es, and changefeed Publish the coven.automations.v1 contract as spec artifacts plus a normative design document, so Cave, the SDK, Psyche adapters, runtimes, and future implementations can consume automations without importing Coven internals. spec/coven-automations/v1/ (draft 2020-12 JSON Schemas and machine-readable artifacts, following the spec/device-pairing/v1 precedent): - five object schemas (definition, occurrence, run, attempt, receipt) with monotonic revisions, integrity digests, exact-revision pinning, and fail-closed unknown-field behavior through an explicit extension bag - command/response envelope: every mutating command carries a stable adoption key (first-commit-wins replay, ADOPTION_REPLAY_MISMATCH on conflict) and expectedRevision where applicable; responses are committed/replayed/rejected only, so a domain failure can never be wrapped as accepted - typed error envelope with a frozen HTTP/control-action status mapping - versioned changefeed envelope: per-stream gapless sequences, globally unique event ids, causation, cursor expiry, compaction snapshots - state-machines.json: authoritative definition/occurrence/run/attempt machines plus the ten normative invariants - capabilities.json: positive and explicit negative variant negotiation - compatibility-matrix.json: change classes and profile-refusal rules - test-vectors.json: golden valid/invalid/unknown-field/downgrade/ unknown-variant vectors, adoption replay and revision-conflict cases, duplicate and out-of-order event replay, with pinned RFC 8785 digests - coven.automations.v1.d.ts: pinned TypeScript projection for SDK/Cave canaries docs/architecture/coven-automations-v1.md: the design document, citing the current OpenCoven#816 code paths for every gap, ratifying the lifecycle semantics, the OpenCoven#816 non-destructive migration, the Rust type mapping, and recording alternatives considered for the genuinely maintainer-owned decisions. No runtime code changes: schemas, vectors, and docs only. Refs OpenCoven#855 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
* refactor: extract route/version authority gate from coven-cli api Move ApiRoute, normalize_api_route, split_path_query, and the route-version constants out of the 25k-line api.rs into a dedicated api_routes.rs gate module (issue OpenCoven#806 slice 1: extract pure request/route parsing first, behind stable contracts). - the gate keeps its exact contract: non-/api/ paths pass through borrowed, /api/v1/<route> is rewritten to /<route>, unsupported versions answer 404 invalid_request with the apiVersion and supportedApiVersions payload, other malformed /api/... shapes answer 404 not_found - characterize the pure classifier (positive, malformed, unsupported, passthrough, query-splitting, version-prefix-never-retained property) and pin the full unsupported-version rejection envelope plus the uniform 404 envelope for malformed API prefixes through handle_request - document the authority-direction invariant on the gate and publish the concentration inventory, extraction order, and where-new-behavior-belongs map in docs/authority-module-inventory.md No behavior change: handlers, routes, status codes, error envelopes, payloads, and persistence are untouched. Refs OpenCoven#806 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> * chore: format authority gate test Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --------- Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>
…tes, and diagnostics Certification plane for Coven Automations v1 (coven#858): an executable, implementation-independent plane that proves schedule semantics, state transitions, authority binding, crash recovery, duplicate prevention, privacy controls, and client compatibility at an exact revision. - conformance/automations/schemas/: versioned artifact schemas (definition, occurrence, run, event, receipt, authority) plus vector/report/doctor envelope schemas - conformance/automations/vectors/ + scenarios/: 97 certification vectors including all 22 golden end-to-end scenarios and cross-repo canary placeholders with pinned-artifact prerequisites - conformance/automations/runner/: dependency-free runner with a deterministic reference oracle (virtual clock, IANA/DST schedule math, misfire-latest fencing, lease recovery, fail-closed authority, retry quarantine, receipts over canonical JSON, idempotent changefeed), seeded randomized invariant fuzzing, and node --test coverage - conformance/automations/slo/: release SLO gates, provisional until a measured baseline lands on the exact release artifact - scripts/agent-bootstrap, scripts/agent-check: canonical agent interface (fast / full / automations-conformance) with machine-readable receipts; unsupported platform capabilities reported separately from failures - ci.yml: automations-conformance job wired into the PR gate The gate enforces zero duplicate dispatches per fence, zero silent eligible-occurrence loss, zero false success under injected failures, and bounded ledger growth - checked continuously, not only at the end. Reports are machine-readable, pin source revision and artifact digests, and are redacted before writing. Refs OpenCoven#858 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
… report path Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
…port Finding 7 of the OpenCoven#882 review: the vector envelope advertised a 'first-event' crash boundary the runner did not implement, negative BYHOUR entries were accepted by the RRULE vocabulary gate, and reports were published without checking them against conformance.report.v1. - parseRrule refuses negative BYHOUR entries (schema and runner agree). - 'first-event' is a real boundary: the pass dies right after the first changefeed event is durably published; the rest of the buffer is lost with the crash, and a restart reconciles without re-dispatch. - runConformance validates the report (in-memory and the exact redacted artifact it writes) against conformance.report.v1 and aborts the run on any violation instead of certifying from a malformed report. Verified: node --test conformance/automations/runner/conformance.test.mjs (21 tests) and a full-profile runner run, both green. Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
…nt oracle Finding 6 of the OpenCoven#882 review: the misfire-latest planner walked forward one slot at a time under a 4096-step cap, silently selecting a stale slot after an outage longer than the cap, and the no-silent-eligible-occurrence-loss invariant used the same implementation so it could never catch its own staleness. - latestDueSlot now computes the latest slot in (cursor, now] directly from the calendar: candidate local dates descend from now (eight days reaches every weekday plus a DST transition day), hours descend, gaps do not exist, folds take the earliest pass. No cap, no walk, nothing to exhaust. - New latestDueSlotBrute is a structurally independent O(days) oracle; the invariant now derives loss from it and adds a new occurrence-planner-agreement violation whenever the planner's answer differs from the oracle's. Verified: node --test ... (23 tests), full-profile runner run with --fuzz 300, all green. Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
…ver all forms Finding 5 of the OpenCoven#882 review: redaction leaked prompts shorter than four characters, multiline and quoted prompts, credentials, Windows paths, and /private/... macOS paths, and it only scrubbed text after serialization. - redactDeep replaces sensitive structured values (prompt/intent keys, path keys holding absolute paths, credential keys) BEFORE JSON.stringify; redactText then scrubs the serialized text so secrets embedded in already-serialized strings are covered too. - Exact prompt replacement no longer has a minimum length; quoted occurrences (single, double, backtick) are replaced as well. - Credential patterns: AWS access keys, GitHub/Slack/GitLab tokens, Google/OpenAI-style API keys, bearer tokens, key=value secrets, PEM private-key blocks, and userinfo URLs (postgres://user:pass@host). - Path patterns: /Users, /home, /root, /private/..., Windows drive-letter homes, %USERPROFILE%-style env paths, and UNC shares — in raw and JSON-escaped backslash forms. - The report write path and the vector redaction scan both go through the deep+text pipeline (redactPublishedText). Verified: node --test ... (27 tests, including 4 new redaction suites with negative assertions) and a full-profile runner run, all green. Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
…for certification Finding 2 of the OpenCoven#882 review: empty and all-skipped profiles were reported as passed, a selected vector could skip and exit 0, and a structural-only run could report the full profile passed. - Vector outcomes are passed | failed | not-applicable. Nothing is 'skipped': a vector the target cannot run is not-applicable with an explicit reason, and a target that advertises the capability but has no evaluator for a selected vector is a hard failure. - Profile results are passed | failed | incomplete | not-applicable. A required vector that does not execute forces its profile incomplete; target-dependent vectors (cross-repo canaries pinning artifacts outside this plane, now marked execution: target-dependent in the envelope and in the six affected vectors) never upgrade a profile. - The full profile only exists on a full-plane run: scoped runs report it not-applicable instead of counting their subset as full passed. - The gate now fails when: any failure, any required profile not passed on a full-plane run, or any selected vector not executed on a scoped run. - Report schema renamed skipped -> notApplicable (with a required flag), added fullProfileStatus and the incomplete/not-applicable profile statuses; reports are still schema-validated before publication. Verified: node --test ... (30 tests) and full-profile runner run, green. Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
… to environment and artifact Finding 4 of the OpenCoven#882 review: the SLO gate accepted a report containing a single recognized measure and treated absent, provisional, or not-run evidence as non-failing. - New slo/slo.measured.v1.schema.json: evidence must carry measuredVersion, the environment profile (cpu, memory, disk, os, store), artifact binding (exact source commit and packed artifact digest), and one value per measure. - evaluateSloGate now fails closed: 'invalid' for malformed or unbound evidence (including an environment that does not match the SLO profile's supportedLocalProfile), 'incomplete' when any declared measure is missing, 'failed' on gate violations, 'passed' only when everything validates. 'provisional' acceptance is gone; slo.v1.json status moves provisional -> enforced and its measurement contract names the binding. - The conformance run gate fails whenever SLO evidence is provided and not passed; no evidence is reported as not-run and explicitly noted as 'vector conformance only, no SLO certification'. - The report schema carries the new statuses plus the gate detail string. Verified: node --test ... (31 tests, incl. the new SLO negative suites) and a full-profile runner run, green. Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
…t adapters Finding 3 of the OpenCoven#882 review: CI exercised only the JavaScript reference oracle while the job language implied product certification; no daemon, in-process, or packaged artifact was ever exercised. - The CI job is now 'Automations conformance vector self-tests (reference oracle)' with a step that says 'not product certification'; scripts/check-ci-workflow-test.py pins the honest labeling. - New runner/lib/adapters: daemon (HTTP endpoint via --endpoint or COVEN_CONFORMANCE_ENDPOINT), in-process (linked module exporting probe()/evaluate() via COVEN_CONFORMANCE_INPROCESS_MODULE), and packaged-release (binary via COVEN_CONFORMANCE_PACKAGE_BIN). Targets must advertise coven.automations.conformance.v1 through probe(); until then every vector is not-applicable, the gate fails, and the report says the adapter 'certifies nothing' — scaffolding cannot pass. - runConformance routes all target kinds through createTarget and validates --target against the known kinds. - Manifest target entries and the runner header document the adapters and the self-test vs certification distinction. Verified: node --test ... (32 tests) green; a structural run against an unwired daemon target exits 1 with the explicit unwired-adapter note. Signed-off-by: CompleteDotTech <5861166+CompleteDotTech@users.noreply.github.com>
CompleteDotTech
force-pushed
the
agent/issue-858-p0-build-automations-conformance-chaos-slo
branch
from
August 31, 2026 09:06
11734ab to
2032e8f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
conformance/automations/**(schemas, vectors, scenarios, runner, SLO, README),scripts/agent-bootstrap,scripts/agent-check,.github/workflows/ci.yml(+PR-gate wiring),scripts/check-ci-workflow-test.py(+assertions).Implementation
conformance.vector.v1envelope schema, then executes it against a target. The default target is the plane's own deterministic reference oracle; daemon and packaged-release targets are supported through the same vectors once they advertisecoven.automations.conformance.v1(unmet capabilities are reported as skips, never silent passes).failed, bounded 64 KiB logs, run terminal monotonicity, legacy-import stays PAUSED) and pins the contracts P0: Specify coven.automations.v1 schemas, state machines, idempotency, and changefeed OpenCoven/coven#855/P0: Harden automation time, retries, cancellation, fencing, and crash recovery OpenCoven/coven#856/P0: Bind automation runs to principal authority, familiar revisions, capabilities, approvals, and receipts OpenCoven/coven#857 specify (IANA time zones, retry/backoff/quarantine, adoption-key idempotency, definition revisions, receipts over canonical JSON, fail-closed authority with approval nonces, delivery-before-settlement, erasure/tombstone with retained audit evidence). Two deliberate contract corrections the vectors codify: the planner walk must reach the latest due slot no matter how long the outage (a capped walk silently drops slots), and changefeed cursors are durable with gaps after crashes (never reused).diagnostics.doctor.v1defines 19 stable finding codes, each with subject ids, a redacted observation, and exact safe next steps (guarded retries,--dry-runreconcile; never row deletion). Diagnostics vectors prove every finding code end to end../scripts/agent-bootstrap,./scripts/agent-check fast|full|automations-conformance(machine-readable receipts, clean-worktree guarantee, unsupported-platform reporting distinct from failures), and a CI job that gates PRs on the conformance plane.plan/claim/adoption/dispatch/session-create/terminal-observation/delivery/settlement/receipt/event-publication) are vector-injectable at every durable or external boundary.Verification
node --test conformance/automations/runner/conformance.test.mjs— 19/19 pass (schema validator, DST gap/fold + IANA + 30-minute-DST zones, planner walk across a year-long gap, canonical JSON, monotonicity/quarantine guards, receipt tamper detection, invariant fuzz across 4 seeds, all-vectors green, report redaction, SLO gate, manifest pinning)node conformance/automations/runner/conformance.mjs --profile all --fuzz 500— gate passed: 91 vectors passed / 0 failed / 6 skipped (5 cross-repo canaries + ACL vector, prerequisite-gated, reported separately); re-verified with seeds 42, 7scripts/agent-check fast/full/automations-conformance— passed; receipts written toconformance/automations/reports/(gitignored); worktree left cleancargo fmt --check,cargo clippy,cargo test— not run locally (no Rust toolchain on this host); deferred to CI. This PR touches no Rust code; the Rust matrix is exercised by the routed CI jobs.python3 scripts/check-secrets.py,privacy guard— deferred to CI (no python3 locally); new files were hand-checked against the guard's patterns (no absolute home paths, no phone numbers, no session keys, no credentials)passed/failed/provisionalstates), single-vector reproduction command verified, vector-list inventory verified (97 vectors, 7 profiles)Risk and Rollback
Evidence packet (per OpenCoven#858)
coven.automations.{definition,occurrence,run,event,receipt,authority}.v1,diagnostics.doctor.v1,conformance.{vector,report}.v1; shipped Rust behavior incrates/coven-cli/src/automations/used as the baseline contract.slo/slo.v1.jsonwith hard gates (zero duplicate dispatch, zero silent loss, zero false success, bounded recovery/growth) enforced by the conformance gate; numeric gates provisional until measured on the release artifact.git rev-parse HEAD), runner version, and SHA-256 digests of every plane artifact; receipts are redacted (prompts, absolute paths) before writing.coven.automations.acl.v1), and the Rust-side target adapter.Agent Handoff
agent/issue-858-p0-build-automations-conformance-chaos-slo.coven.automations.conformance.v1in the Rust daemon/CLI so the same vectors certify the real implementation and packed artifacts; wire the daemon/packaged-release target adapters; execute cross-repo canaries against pinned producer artifacts; run the load harness and ratify the numeric SLO gates; implement the ACL layer for the unauthorized-read vectors; land the P0: Specify coven.automations.v1 schemas, state machines, idempotency, and changefeed OpenCoven/coven#855/P0: Harden automation time, retries, cancellation, fencing, and crash recovery OpenCoven/coven#856/P0: Bind automation runs to principal authority, familiar revisions, capabilities, approvals, and receipts OpenCoven/coven#857 contracts the vectors specify (several vectors intentionally pin contracts ahead of the implementation — e.g. IANA time zones, adoption keys, receipts, quarantine — and will drive those issues).Test plan
node --test conformance/automations/runner/conformance.test.mjs(ran locally, 19/19)node conformance/automations/runner/conformance.mjs --profile all --fuzz 500(ran locally, gate passed; seeds 858/42/7)scripts/agent-check fast/full/automations-conformance(ran locally)cargo fmt --check/cargo clippy/cargo test --workspace --locked(deferred to CI — no Rust toolchain on this host; no Rust code touched)python3 scripts/check-secrets.py+ privacy guard (deferred to CI — no python3 locally; patterns hand-checked)