From 7bcedd7d9a0ae889038dd9d81ce103a43a1585cb Mon Sep 17 00:00:00 2001 From: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> Date: Sun, 30 Aug 2026 16:00:14 +0000 Subject: [PATCH 1/2] fix: scope retained G2 evidence validity to the attested contract surface The passed-evidence relationship verifier required the historic tested source commit to be the head's merge-base and the entire tested-source- to-head diff to contain exactly one modified file, docs/G2-EVIDENCE.md. That protected the original evidence-only attestation, but applied forever it froze every later non-evidence change out of the attested branch: the G2 evidence relationships step fails on PR #15's onboarding/readiness-only diff (run 32956215259). Introduce scripts/g2-scope-manifest.json, a versioned (schema_version 1) fail-closed attestation-scope manifest, and make both relationship verifiers compare the tested commit to the terminal commit by scope: - protected patterns enumerate the G2-attested contract surface derived from what this checker validates (crates/**, the architecture/schema/ testing documents and golden fixtures it requires), what the reviewed workflow exercises (.github/workflows/ci.yml, packages/psyche-npm/**, Cargo.toml, Cargo.lock, rust-toolchain.toml, clippy.toml, deny.toml), and the trust root itself (the checker, its tests, the test manifest, the scope manifest, and the evidence document); - explicitly unscoped operational/readiness paths (onboarding, governance, roadmap, agent metadata, non-attested docs) may change without refreshing G2 evidence; - unmatched or new paths default to protected until explicitly classified; - the checker enforces a REQUIRED_PROTECTED_PATHS floor independently of manifest edits, so the manifest itself cannot shrink the trust root; - renames, copies, deletes, typechanges, submodule changes, unsafe paths, and truncated (>=300-file) compare responses fail closed even when the paths classify unscoped; - changing only docs/G2-EVIDENCE.md remains subject to the original exact evidence-only relationship rules; - local and GitHub Actions verification share one classification logic; - diagnostics name the protected paths that invalidated the attestation without quoting their contents. Because this change touches protected trust-root paths, the retained attestation is invalidated by design: docs/G2-EVIDENCE.md transitions from Status: passed (tested source f4f242f88f41dcb8286c867ea1601dac2c14a80a, run https://github.com/OpenCoven/psyche/actions/runs/31565714638) to the exact candidate placeholders until a new attestation replaces it. The historic record remains in git history. Every file of PR #15 is explicitly classified unscoped, so it passes this gate without claiming its new files were covered by G2. Refs OpenCoven/psyche#16 Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> --- docs/G2-EVIDENCE.md | 76 ++++---- scripts/check-g2-evidence-test.py | 300 +++++++++++++++++++++++++++--- scripts/check-g2-evidence.py | 237 +++++++++++++++++++++-- scripts/g2-scope-manifest.json | 36 ++++ 4 files changed, 566 insertions(+), 83 deletions(-) create mode 100644 scripts/g2-scope-manifest.json diff --git a/docs/G2-EVIDENCE.md b/docs/G2-EVIDENCE.md index 2ae215b..81bf23a 100644 --- a/docs/G2-EVIDENCE.md +++ b/docs/G2-EVIDENCE.md @@ -1,11 +1,11 @@ # G2 Contract Foundation Evidence -**Status:** passed -**Tested source commit:** `f4f242f88f41dcb8286c867ea1601dac2c14a80a` -**CI attestation:** `https://github.com/OpenCoven/psyche/actions/runs/31565714638` -**Coven plan source commit:** `5f22ebef1e23d045a10f2ec0a3c87be029446cf6` -**Coven plan URL:** `https://github.com/OpenCoven/coven/blob/5f22ebef1e23d045a10f2ec0a3c87be029446cf6/docs/superpowers/plans/2026-08-05-psyche-w2-g2-foundation.md` -**Coven plan SHA-256:** `sha256:4fba002ad9f969cd01866ea08f270654f82b53c7d90b73d28643a9abb12cba68` +**Status:** candidate +**Tested source commit:** not recorded before remote review +**CI attestation:** not recorded before remote review +**Coven plan source commit:** not recorded before plan approval +**Coven plan URL:** not recorded before plan approval +**Coven plan SHA-256:** not recorded before plan approval **Coven specification source commit:** `42dcbc43-34cb48ec-af63efb5-50345e3e-ea2fb7ad` | Coven source | Immutable URL | SHA-256 | @@ -18,35 +18,35 @@ | Criterion | Command | Result | Artifact | |---|---|---|---| -| Canonical ID prefixes and execution-binding identity | `cargo test -p psyche-core --test contracts -- --exact delivery_keeps_the_canonical_del_prefix && cargo test -p psyche-core --test contracts -- --exact delegation_uses_the_distinct_dlg_prefix && cargo test -p psyche-core --test contracts -- --exact execution_binding_uses_attempt_as_its_only_record_kind` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Complete canonical error enum | `cargo test -p psyche-core --test contracts -- --exact all_canonical_error_codes_decode` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Canonical delivery v1 shape | `cargo test -p psyche-core --test contracts -- --exact delivery_v1_fixture_round_trips_canonically && cargo test -p psyche-store --test records -- --exact delivery_direct_insert_round_trips_canonically` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Surface and quarantine owned types | `cargo test -p psyche-core --test contracts -- --exact surface_event_and_effect_fixtures_round_trip && cargo test -p psyche-store --test retention -- --exact quarantine_id_constructor_parser_and_serde_round_trip` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Package-local nullable-binding fixtures | `cargo test -p psyche-core --test contracts -- --exact graph_and_node_accept_only_the_two_frozen_nullable_bindings` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Exhaustive registered decode | `cargo test -p psyche-core --test decode -- --exact recognized_error_envelope_decodes_exhaustively` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Unknown kind/version/enum denial and quarantine | `cargo test -p psyche-core --test decode -- --exact unknown_typed_enum_is_a_quarantinable_decode_failure && cargo test -p psyche-store --test retention -- --exact unknown_enum_is_quarantined_without_dispatchable_record` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Quarantine resolution | `cargo test -p psyche-store --test retention -- --exact quarantine_resolution_is_durable_and_idempotent && cargo test -p psyche-store --test retention -- --exact concurrent_quarantine_resolution_has_one_durable_winner` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Direct typed insert validation | `cargo test -p psyche-store --test records -- --exact direct_insert_rejects_wrong_field_id_kind_without_writing && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledged_cancellation_without_evidence && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledged_state_without_termination_correlation && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_mismatched_cancellation_evidence && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_wrong_termination_request_id && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_termination_before_execution_request && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledgement_outside_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledgement_before_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_unresolved_outside_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_unresolved_before_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_acknowledgement_at_termination_window_boundaries && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_unresolved_at_termination_window_boundaries && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_termination_window_after_execution_deadline && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_termination_at_execution_creation_boundary` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Append-only execution-binding revisions | `cargo test -p psyche-store --test records -- --exact execution_binding_revision_appends_termination_outcomes_without_record_conflict && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_forks_gaps_and_changed_correlation && cargo test -p psyche-store --test records -- --exact execution_binding_revision_replay_is_idempotent && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_same_revision_changed_bytes && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_changed_reason_replay && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_every_frozen_execution_field_change && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_session_and_termination_rebinding && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_termination_correlation_removal && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_timestamp_regression && cargo test -p psyche-store --test retention -- --exact pruning_preserves_unresolved_quarantine_binding_revisions_and_transitions` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Transition contract and append-only rules | `cargo test -p psyche-store --test records -- --exact transition_versions_are_monotonic_and_append_only` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Checkpoint-failure shutdown | `cargo test -p psyche-runtime --lib -- --exact tests::checkpoint_failure_stops_and_releases_every_shutdown_waiter` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Migrations | `cargo test -p psyche-store --test migrations -- --exact fresh_store_applies_v1_once_and_reopens` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| State-machine/property | `cargo test -p psyche-test-support --test state_machine -- --exact model_and_store_agree_after_any_foundation_operation_sequence` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Crash/restart | `cargo test -p psyche-store --features test-fault-injection --test crash -- --exact killed_writer_exposes_only_committed_state_after_reopen` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Fake boundaries and durable termination ordering | `cargo test -p psyche-test-support --test fakes -- --exact advertised_adoption_requires_a_scripted_adoption_step && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_requires_durable_session_bound_revision && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_persists_acknowledged_outcome_before_success && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_persists_unresolved_outcome_before_success && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_exact_replay_is_idempotent && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_crash_after_response_leaves_recoverable_request && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_restart_recovers_missing_outcome && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_conflicting_replay_response && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_invalid_outcome_evidence && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_unresolved_outside_termination_window && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_reports_indeterminate_outcome_persistence && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_accepts_concurrent_exact_outcome_replay && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_concurrent_divergent_outcome && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_outcome_byte_attestation_mismatch` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Execution request RFC3339 golden bytes | `cargo test -p psyche-coven --test request_digest -- --exact execution_request_launch_matches_golden_bytes_and_digest && cargo test -p psyche-coven --test request_digest -- --exact execution_request_input_matches_golden_bytes_and_digest` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Validated termination dispatch | `cargo test -p psyche-coven --test bindings -- --exact termination_dispatch_rejects_invalid_request_before_persistence` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| G2 cancellation-state vocabulary | `cargo test -p psyche-core --test contracts -- --exact cancellation_state_vocabulary_requires_matching_o5_evidence` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| Full execution-request digest binding | `cargo test -p psyche-test-support --test state_machine -- --exact request_digest_binds_every_typed_field` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S1 scripted contract negotiation | `cargo test -p psyche-test-support --test conformance -- --exact c_s1_contract_negotiation` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S2 scripted session lifecycle | `cargo test -p psyche-test-support --test conformance -- --exact c_s2_session_lifecycle` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S3 scripted snapshot/attempt binding | `cargo test -p psyche-test-support --test conformance -- --exact c_s3_snapshot_attempt_binding` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S4 scripted stable adoption | `cargo test -p psyche-test-support --test conformance -- --exact c_s4_stable_adoption` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S5 scripted non-adoption proof | `cargo test -p psyche-test-support --test conformance -- --exact c_s5_non_adoption_proof` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S6 scripted ambiguity reconciliation/fence | `cargo test -p psyche-test-support --test state_machine -- --exact c_s6_model_never_redispatches_without_fence && cargo test -p psyche-test-support --test conformance -- --exact c_s6_ambiguity_fence` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S7 scripted ordered cursor | `cargo test -p psyche-test-support --test conformance -- --exact c_s7_ordered_cursor` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S8 scripted terminal authority | `cargo test -p psyche-test-support --test conformance -- --exact c_s8_terminal_authority` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S9 scripted O5 cancellation acknowledgement | `cargo test -p psyche-test-support --test conformance -- --exact c_s9_cancellation_acknowledgement` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S10 scripted result/artifact binding | `cargo test -p psyche-coven --test bindings -- --exact result_bundle_fixture_round_trips_complete_content_references && cargo test -p psyche-coven --test bindings -- --exact result_bundle_fixture_uses_launch_request_correlation && cargo test -p psyche-coven --test bindings -- --exact content_reference_rejects_digest_size_media_type_and_lifetime_mismatch && cargo test -p psyche-test-support --test conformance -- --exact c_s10_result_artifact_binding` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S11 scripted restart persistence | `cargo test -p psyche-test-support --test conformance -- --exact c_s11_restart_persistence` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | -| C-S12 scripted structured denial | `cargo test -p psyche-test-support --test conformance -- --exact c_s12_structured_denial` | passed | https://github.com/OpenCoven/psyche/actions/runs/31565714638 | +| Canonical ID prefixes and execution-binding identity | `cargo test -p psyche-core --test contracts -- --exact delivery_keeps_the_canonical_del_prefix && cargo test -p psyche-core --test contracts -- --exact delegation_uses_the_distinct_dlg_prefix && cargo test -p psyche-core --test contracts -- --exact execution_binding_uses_attempt_as_its_only_record_kind` | not run remotely | none | +| Complete canonical error enum | `cargo test -p psyche-core --test contracts -- --exact all_canonical_error_codes_decode` | not run remotely | none | +| Canonical delivery v1 shape | `cargo test -p psyche-core --test contracts -- --exact delivery_v1_fixture_round_trips_canonically && cargo test -p psyche-store --test records -- --exact delivery_direct_insert_round_trips_canonically` | not run remotely | none | +| Surface and quarantine owned types | `cargo test -p psyche-core --test contracts -- --exact surface_event_and_effect_fixtures_round_trip && cargo test -p psyche-store --test retention -- --exact quarantine_id_constructor_parser_and_serde_round_trip` | not run remotely | none | +| Package-local nullable-binding fixtures | `cargo test -p psyche-core --test contracts -- --exact graph_and_node_accept_only_the_two_frozen_nullable_bindings` | not run remotely | none | +| Exhaustive registered decode | `cargo test -p psyche-core --test decode -- --exact recognized_error_envelope_decodes_exhaustively` | not run remotely | none | +| Unknown kind/version/enum denial and quarantine | `cargo test -p psyche-core --test decode -- --exact unknown_typed_enum_is_a_quarantinable_decode_failure && cargo test -p psyche-store --test retention -- --exact unknown_enum_is_quarantined_without_dispatchable_record` | not run remotely | none | +| Quarantine resolution | `cargo test -p psyche-store --test retention -- --exact quarantine_resolution_is_durable_and_idempotent && cargo test -p psyche-store --test retention -- --exact concurrent_quarantine_resolution_has_one_durable_winner` | not run remotely | none | +| Direct typed insert validation | `cargo test -p psyche-store --test records -- --exact direct_insert_rejects_wrong_field_id_kind_without_writing && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledged_cancellation_without_evidence && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledged_state_without_termination_correlation && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_mismatched_cancellation_evidence && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_wrong_termination_request_id && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_termination_before_execution_request && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledgement_outside_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_acknowledgement_before_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_unresolved_outside_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_rejects_unresolved_before_termination_window && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_acknowledgement_at_termination_window_boundaries && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_unresolved_at_termination_window_boundaries && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_termination_window_after_execution_deadline && cargo test -p psyche-store --test records -- --exact direct_insert_accepts_termination_at_execution_creation_boundary` | not run remotely | none | +| Append-only execution-binding revisions | `cargo test -p psyche-store --test records -- --exact execution_binding_revision_appends_termination_outcomes_without_record_conflict && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_forks_gaps_and_changed_correlation && cargo test -p psyche-store --test records -- --exact execution_binding_revision_replay_is_idempotent && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_same_revision_changed_bytes && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_changed_reason_replay && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_every_frozen_execution_field_change && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_session_and_termination_rebinding && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_termination_correlation_removal && cargo test -p psyche-store --test records -- --exact execution_binding_revision_rejects_timestamp_regression && cargo test -p psyche-store --test retention -- --exact pruning_preserves_unresolved_quarantine_binding_revisions_and_transitions` | not run remotely | none | +| Transition contract and append-only rules | `cargo test -p psyche-store --test records -- --exact transition_versions_are_monotonic_and_append_only` | not run remotely | none | +| Checkpoint-failure shutdown | `cargo test -p psyche-runtime --lib -- --exact tests::checkpoint_failure_stops_and_releases_every_shutdown_waiter` | not run remotely | none | +| Migrations | `cargo test -p psyche-store --test migrations -- --exact fresh_store_applies_v1_once_and_reopens` | not run remotely | none | +| State-machine/property | `cargo test -p psyche-test-support --test state_machine -- --exact model_and_store_agree_after_any_foundation_operation_sequence` | not run remotely | none | +| Crash/restart | `cargo test -p psyche-store --features test-fault-injection --test crash -- --exact killed_writer_exposes_only_committed_state_after_reopen` | not run remotely | none | +| Fake boundaries and durable termination ordering | `cargo test -p psyche-test-support --test fakes -- --exact advertised_adoption_requires_a_scripted_adoption_step && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_requires_durable_session_bound_revision && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_persists_acknowledged_outcome_before_success && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_persists_unresolved_outcome_before_success && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_exact_replay_is_idempotent && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_crash_after_response_leaves_recoverable_request && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_restart_recovers_missing_outcome && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_conflicting_replay_response && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_invalid_outcome_evidence && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_unresolved_outside_termination_window && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_reports_indeterminate_outcome_persistence && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_accepts_concurrent_exact_outcome_replay && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_concurrent_divergent_outcome && cargo test -p psyche-test-support --test fakes -- --exact termination_dispatch_rejects_outcome_byte_attestation_mismatch` | not run remotely | none | +| Execution request RFC3339 golden bytes | `cargo test -p psyche-coven --test request_digest -- --exact execution_request_launch_matches_golden_bytes_and_digest && cargo test -p psyche-coven --test request_digest -- --exact execution_request_input_matches_golden_bytes_and_digest` | not run remotely | none | +| Validated termination dispatch | `cargo test -p psyche-coven --test bindings -- --exact termination_dispatch_rejects_invalid_request_before_persistence` | not run remotely | none | +| G2 cancellation-state vocabulary | `cargo test -p psyche-core --test contracts -- --exact cancellation_state_vocabulary_requires_matching_o5_evidence` | not run remotely | none | +| Full execution-request digest binding | `cargo test -p psyche-test-support --test state_machine -- --exact request_digest_binds_every_typed_field` | not run remotely | none | +| C-S1 scripted contract negotiation | `cargo test -p psyche-test-support --test conformance -- --exact c_s1_contract_negotiation` | not run remotely | none | +| C-S2 scripted session lifecycle | `cargo test -p psyche-test-support --test conformance -- --exact c_s2_session_lifecycle` | not run remotely | none | +| C-S3 scripted snapshot/attempt binding | `cargo test -p psyche-test-support --test conformance -- --exact c_s3_snapshot_attempt_binding` | not run remotely | none | +| C-S4 scripted stable adoption | `cargo test -p psyche-test-support --test conformance -- --exact c_s4_stable_adoption` | not run remotely | none | +| C-S5 scripted non-adoption proof | `cargo test -p psyche-test-support --test conformance -- --exact c_s5_non_adoption_proof` | not run remotely | none | +| C-S6 scripted ambiguity reconciliation/fence | `cargo test -p psyche-test-support --test state_machine -- --exact c_s6_model_never_redispatches_without_fence && cargo test -p psyche-test-support --test conformance -- --exact c_s6_ambiguity_fence` | not run remotely | none | +| C-S7 scripted ordered cursor | `cargo test -p psyche-test-support --test conformance -- --exact c_s7_ordered_cursor` | not run remotely | none | +| C-S8 scripted terminal authority | `cargo test -p psyche-test-support --test conformance -- --exact c_s8_terminal_authority` | not run remotely | none | +| C-S9 scripted O5 cancellation acknowledgement | `cargo test -p psyche-test-support --test conformance -- --exact c_s9_cancellation_acknowledgement` | not run remotely | none | +| C-S10 scripted result/artifact binding | `cargo test -p psyche-coven --test bindings -- --exact result_bundle_fixture_round_trips_complete_content_references && cargo test -p psyche-coven --test bindings -- --exact result_bundle_fixture_uses_launch_request_correlation && cargo test -p psyche-coven --test bindings -- --exact content_reference_rejects_digest_size_media_type_and_lifetime_mismatch && cargo test -p psyche-test-support --test conformance -- --exact c_s10_result_artifact_binding` | not run remotely | none | +| C-S11 scripted restart persistence | `cargo test -p psyche-test-support --test conformance -- --exact c_s11_restart_persistence` | not run remotely | none | +| C-S12 scripted structured denial | `cargo test -p psyche-test-support --test conformance -- --exact c_s12_structured_denial` | not run remotely | none | diff --git a/scripts/check-g2-evidence-test.py b/scripts/check-g2-evidence-test.py index b4081f9..4b5799c 100644 --- a/scripts/check-g2-evidence-test.py +++ b/scripts/check-g2-evidence-test.py @@ -35,11 +35,28 @@ def setUpClass(cls) -> None: cls.checker = load_checker() cls.manifest = json.loads((ROOT / "scripts/g2-test-manifest.json").read_text()) cls.evidence = (ROOT / "docs/G2-EVIDENCE.md").read_text() + cls.scope = cls.checker.validate_scope_manifest( + (ROOT / "scripts/g2-scope-manifest.json").read_text() + ) cls.listed = { target: "".join(f"{name}: test\n" for name in definition["tests"]) for target, definition in cls.manifest["targets"].items() } + @staticmethod + def full_entry(filename: str, status: str) -> dict[str, str]: + return { + "filename": filename, + "status": status, + "raw_url": f"https://github.com/OpenCoven/psyche/raw/head/{filename}", + "blob_url": f"https://github.com/OpenCoven/psyche/blob/head/{filename}", + } + + def scope_manifest_text(self, extra_unscoped: tuple[str, ...] = ()) -> str: + manifest = json.loads((ROOT / "scripts/g2-scope-manifest.json").read_text()) + manifest["unscoped"] = sorted({*manifest["unscoped"], *extra_unscoped}) + return json.dumps(manifest) + def assert_valid( self, *, @@ -503,7 +520,7 @@ def test_passed_remote_verifier_accepts_exact_ci_attestation(self) -> None: passed = self.passed_evidence() completed = ( subprocess.CompletedProcess([], 0, "", ""), - subprocess.CompletedProcess([], 0, "docs/G2-EVIDENCE.md\n", ""), + subprocess.CompletedProcess([], 0, "M\tdocs/G2-EVIDENCE.md\n", ""), ) run = { "conclusion": "success", @@ -535,7 +552,7 @@ def test_passed_remote_verifier_accepts_exact_ci_attestation(self) -> None: ), mock.patch.object( self.checker, "run_json", side_effect=(run, rest, workflow) ) as run_json, mock.patch.object(self.checker, "verify_coven_blob") as verify_blob: - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) self.assertEqual(run_json.call_count, 3) self.assertEqual(verify_blob.call_count, 6) @@ -566,7 +583,7 @@ def test_shallow_actions_verifier_uses_event_and_compare_without_local_git(self) ), mock.patch.object( self.checker, "run_json", side_effect=(self.compare_response(), run, rest, workflow) ) as run_json, mock.patch.object(self.checker, "verify_coven_blob"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) self.assertEqual( run_json.call_args_list[0].args[0], [ @@ -601,7 +618,7 @@ def test_shallow_actions_verifier_accepts_reviewed_squash_merge_push(self) -> No "run_json", side_effect=([self.merged_pull_request()], merge_commit, terminal_commit, self.compare_response()), ) as run_json: - self.checker.verify_actions_source_relationship(ROOT, tested) + self.checker.verify_actions_source_relationship(ROOT, tested, self.scope) self.assertEqual( [call.args[0] for call in run_json.call_args_list], [ @@ -627,7 +644,7 @@ def test_shallow_actions_verifier_rejects_malformed_event_payload(self) -> None: self.checker.pathlib.Path, "read_text", return_value="{not-json" ), mock.patch.object(self.checker, "run_json"): with self.assertRaisesRegex(self.checker.EvidenceError, "event payload"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_shallow_actions_verifier_rejects_wrong_head_repository(self) -> None: passed = self.passed_evidence() @@ -637,38 +654,38 @@ def test_shallow_actions_verifier_rejects_wrong_head_repository(self) -> None: self.checker.pathlib.Path, "read_text", return_value=event ), mock.patch.object(self.checker, "run_json"): with self.assertRaisesRegex(self.checker.EvidenceError, "repository"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_shallow_actions_verifier_rejects_wrong_compare_files(self) -> None: passed = self.passed_evidence() event = json.dumps(self.pull_request_event()) - compare = self.compare_response(files=[{"filename": "src/main.rs", "status": "modified"}]) + compare = self.compare_response(files=[self.full_entry("src/main.rs", "modified")]) environment = {"GITHUB_ACTIONS": "true", "GITHUB_EVENT_PATH": "/tmp/event.json"} with mock.patch.dict(os.environ, environment, clear=False), mock.patch.object( self.checker.pathlib.Path, "read_text", return_value=event ), mock.patch.object(self.checker, "run_json", return_value=compare): - with self.assertRaisesRegex(self.checker.EvidenceError, "modified evidence"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + with self.assertRaisesRegex(self.checker.EvidenceError, "protected G2 attestation paths"): + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_shallow_actions_verifier_rejects_duplicate_missing_or_renamed_file_status(self) -> None: passed = self.passed_evidence() event = json.dumps(self.pull_request_event()) - evidence = {"filename": "docs/G2-EVIDENCE.md", "status": "modified"} + evidence = self.full_entry("docs/G2-EVIDENCE.md", "modified") mutations = ( - [evidence, evidence], - [{"filename": "docs/G2-EVIDENCE.md"}], - [{"filename": "docs/G2-EVIDENCE.md", "status": "renamed"}], + ([evidence, evidence], "protected G2 attestation paths"), + ([{"filename": "docs/G2-EVIDENCE.md", "raw_url": "https://x", "blob_url": "https://y"}], "no comparable status"), + ([{**evidence, "status": "renamed"}], "fail closed by status"), ) environment = {"GITHUB_ACTIONS": "true", "GITHUB_EVENT_PATH": "/tmp/event.json"} - for files in mutations: + for files, expected in mutations: compare = self.compare_response(files=files) with self.subTest(files=files), mock.patch.dict( os.environ, environment, clear=False ), mock.patch.object( self.checker.pathlib.Path, "read_text", return_value=event ), mock.patch.object(self.checker, "run_json", return_value=compare): - with self.assertRaisesRegex(self.checker.EvidenceError, "modified evidence"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + with self.assertRaisesRegex(self.checker.EvidenceError, expected): + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_shallow_actions_verifier_rejects_compare_commit_count_mismatch(self) -> None: passed = self.passed_evidence() @@ -680,7 +697,7 @@ def test_shallow_actions_verifier_rejects_compare_commit_count_mismatch(self) -> self.checker.pathlib.Path, "read_text", return_value=event ), mock.patch.object(self.checker, "run_json", return_value=compare): with self.assertRaisesRegex(self.checker.EvidenceError, "commit counts"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_shallow_actions_verifier_rejects_missing_empty_or_wrong_terminal_commit(self) -> None: passed = self.passed_evidence() @@ -698,7 +715,7 @@ def test_shallow_actions_verifier_rejects_missing_empty_or_wrong_terminal_commit self.checker.pathlib.Path, "read_text", return_value=event ), mock.patch.object(self.checker, "run_json", return_value=compare): with self.assertRaisesRegex(self.checker.EvidenceError, "terminal commit"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_shallow_actions_verifier_rejects_non_ancestor_compare(self) -> None: passed = self.passed_evidence() @@ -709,7 +726,240 @@ def test_shallow_actions_verifier_rejects_non_ancestor_compare(self) -> None: self.checker.pathlib.Path, "read_text", return_value=event ), mock.patch.object(self.checker, "run_json", return_value=compare): with self.assertRaisesRegex(self.checker.EvidenceError, "ancestor"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) + + def test_unrelated_readiness_file_reproduces_the_former_evidence_only_freeze(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + compare = self.compare_response(files=[self.full_entry("docs/ROADMAP.md", "modified")]) + # The former rule froze every non-evidence change, including unrelated + # readiness documentation, out of the repository forever. + with self.assertRaisesRegex(self.checker.EvidenceError, "modified evidence"): + self.checker.validate_evidence_only_compare(compare, tested, terminal) + # The scope-aware relationship rule lifts exactly that freeze for + # explicitly unscoped readiness paths. + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + + def test_unscoped_readiness_delta_preserves_the_retained_attestation(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + files = [ + self.full_entry("README.md", "added"), + self.full_entry("docs/ROADMAP.md", "added"), + self.full_entry("docs/CLI.md", "modified"), + self.full_entry("scripts/agent-check", "added"), + ] + self.checker.validate_scoped_compare(self.compare_response(files=files), tested, terminal, self.scope) + + def test_original_evidence_only_relationship_remains_valid_under_scoping(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + compare = self.compare_response() + self.checker.validate_evidence_only_compare(compare, tested, terminal) + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + + def test_evidence_document_cannot_bless_other_changes(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + files = [ + self.full_entry("docs/G2-EVIDENCE.md", "modified"), + self.full_entry("README.md", "added"), + ] + with self.assertRaisesRegex(self.checker.EvidenceError, "protected G2 attestation paths"): + self.checker.validate_scoped_compare(self.compare_response(files=files), tested, terminal, self.scope) + + def test_local_relationship_verifier_agrees_with_the_actions_verifier(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + ready = (("README.md", "added"), ("docs/CLI.md", "modified")) + compare = self.compare_response(files=[self.full_entry(path, status) for path, status in ready]) + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + raw = ( + ":000000 100644 0000000000000000000000000000000000000000 " + "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 A\tREADME.md\n" + ":100644 100644 1111111111111111111111111111111111111111 " + "2222222222222222222222222222222222222222 M\tdocs/CLI.md\n" + ) + completed = ( + subprocess.CompletedProcess([], 0, "", ""), + subprocess.CompletedProcess([], 0, "A\tREADME.md\nM\tdocs/CLI.md\n", ""), + subprocess.CompletedProcess([], 0, raw, ""), + ) + with mock.patch.object(self.checker.subprocess, "run", side_effect=completed) as runs: + self.checker.verify_local_source_relationship(ROOT, tested, self.scope) + self.assertEqual(runs.call_count, 3) + protected = (("crates/psyche-core/src/lib.rs", "modified"),) + compare = self.compare_response(files=[self.full_entry(path, status) for path, status in protected]) + with self.assertRaisesRegex(self.checker.EvidenceError, "protected G2 attestation paths"): + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + local = ( + subprocess.CompletedProcess([], 0, "", ""), + subprocess.CompletedProcess([], 0, "M\tcrates/psyche-core/src/lib.rs\n", ""), + subprocess.CompletedProcess( + [], 0, + ":100644 100644 3333333333333333333333333333333333333333 " + "4444444444444444444444444444444444444444 M\tcrates/psyche-core/src/lib.rs\n", + "", + ), + ) + with mock.patch.object(self.checker.subprocess, "run", side_effect=local): + with self.assertRaisesRegex(self.checker.EvidenceError, "protected G2 attestation paths"): + self.checker.verify_local_source_relationship(ROOT, tested, self.scope) + + def test_every_protected_surface_mutation_invalidates_the_attestation(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + paths = ( + "crates/psyche-core/src/contracts/mod.rs", + "crates/psyche-store/migrations/001_foundation.sql", + "crates/psyche-coven/tests/bindings.rs", + "crates/psyche-coven/tests/fixtures/result-bundle.json", + "crates/psyche-test-support/src/suites/coven.rs", + "crates/psyche-test-support/tests/conformance.rs", + "packages/psyche-npm/package.json", + ".github/workflows/ci.yml", + "Cargo.toml", + "Cargo.lock", + "rust-toolchain.toml", + "clippy.toml", + "deny.toml", + "scripts/check-g2-evidence.py", + "scripts/check-g2-evidence-test.py", + "scripts/g2-test-manifest.json", + "scripts/g2-scope-manifest.json", + "docs/ARCHITECTURE.md", + "docs/SCHEMAS.md", + "docs/TESTING.md", + ) + for path in paths: + for kind in ("modified", "added"): + compare = self.compare_response(files=[self.full_entry(path, kind)]) + with self.subTest(path=path, kind=kind): + with self.assertRaisesRegex(self.checker.EvidenceError, "protected G2 attestation paths"): + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + + def test_unknown_paths_default_to_protected_until_explicitly_classified(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + compare = self.compare_response(files=[self.full_entry("docs/BRAND-NEW.md", "added")]) + with self.assertRaisesRegex(self.checker.EvidenceError, "protected G2 attestation paths"): + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + scope = self.checker.validate_scope_manifest(self.scope_manifest_text(("docs/BRAND-NEW.md",))) + self.checker.validate_scoped_compare(compare, tested, terminal, scope) + + def test_renames_deletes_copies_and_typechanges_fail_closed(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + for status in ("removed", "renamed", "copied", "changed"): + compare = self.compare_response(files=[self.full_entry("docs/ROADMAP.md", status)]) + with self.subTest(status=status): + with self.assertRaisesRegex(self.checker.EvidenceError, "fail closed by status"): + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + renamed = self.compare_response(files=[{ + **self.full_entry("docs/ROADMAP.md", "renamed"), + "previous_filename": "docs/OLD-ROADMAP.md", + }]) + with self.assertRaisesRegex(self.checker.EvidenceError, "rename or copy"): + self.checker.validate_scoped_compare(renamed, tested, terminal, self.scope) + + def test_local_renames_deletes_and_typechanges_fail_closed(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + mutations = ( + ( + "R100\tdocs/ROADMAP.md\tdocs/ROADMAP-v2.md", + ":100644 100644 3333333333333333333333333333333333333333 " + "4444444444444444444444444444444444444444 R100\tdocs/ROADMAP.md\tdocs/ROADMAP-v2.md\n", + ), + ( + "D\tREADME.md", + ":100644 000000 3333333333333333333333333333333333333333 " + "0000000000000000000000000000000000000000 D\tREADME.md\n", + ), + ( + "T\tREADME.md", + ":100644 120000 3333333333333333333333333333333333333333 " + "4444444444444444444444444444444444444444 T\tREADME.md\n", + ), + ) + for name_status, raw in mutations: + completed = ( + subprocess.CompletedProcess([], 0, "", ""), + subprocess.CompletedProcess([], 0, f"{name_status}\n", ""), + subprocess.CompletedProcess([], 0, raw, ""), + ) + with self.subTest(name_status=name_status), mock.patch.object( + self.checker.subprocess, "run", side_effect=completed + ): + with self.assertRaisesRegex(self.checker.EvidenceError, "fail closed by status"): + self.checker.verify_local_source_relationship(ROOT, tested, self.scope) + + def test_submodule_changes_fail_closed(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + compare = self.compare_response(files=[{"filename": "vendor/lib", "status": "modified"}]) + with self.assertRaisesRegex(self.checker.EvidenceError, "blob provenance"): + self.checker.validate_scoped_compare(compare, tested, terminal, self.scope) + scope = self.checker.validate_scope_manifest(self.scope_manifest_text(("vendor/**",))) + completed = ( + subprocess.CompletedProcess([], 0, "", ""), + subprocess.CompletedProcess([], 0, "M\tvendor/lib\n", ""), + subprocess.CompletedProcess( + [], 0, + ":160000 160000 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa " + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb M\tvendor/lib\n", + "", + ), + ) + with mock.patch.object(self.checker.subprocess, "run", side_effect=completed): + with self.assertRaisesRegex(self.checker.EvidenceError, "submodule change fails closed"): + self.checker.verify_local_source_relationship(ROOT, tested, scope) + + def test_truncated_compare_responses_fail_closed(self) -> None: + tested = "0123456789abcdef0123456789abcdef01234567" + terminal = "fedcba9876543210fedcba9876543210fedcba98" + scope = self.checker.validate_scope_manifest(self.scope_manifest_text(("docs/readiness-*.md",))) + at_cap = [self.full_entry(f"docs/readiness-{index}.md", "added") for index in range(300)] + with self.assertRaisesRegex(self.checker.EvidenceError, "truncated"): + self.checker.validate_scoped_compare(self.compare_response(files=at_cap), tested, terminal, scope) + below_cap = [self.full_entry(f"docs/readiness-{index}.md", "added") for index in range(299)] + self.checker.validate_scoped_compare(self.compare_response(files=below_cap), tested, terminal, scope) + + def test_scope_manifest_is_fail_closed(self) -> None: + base = json.loads((ROOT / "scripts/g2-scope-manifest.json").read_text()) + mutations = { + "invalid json": ("{not-json", "invalid JSON"), + "extra key": (json.dumps({**base, "note": "extra"}), "exactly the keys"), + "missing key": (json.dumps({"schema_version": 1, "protected": base["protected"]}), "exactly the keys"), + "wrong schema version": (json.dumps({**base, "schema_version": 2}), "schema_version must be exactly"), + "boolean schema version": (json.dumps({**base, "schema_version": True}), "schema_version must be exactly"), + "protected not a list": (json.dumps({**base, "protected": "crates/**"}), "non-empty list"), + "empty pattern": (json.dumps({**base, "unscoped": [""]}), "safe repository path"), + "absolute pattern": (json.dumps({**base, "unscoped": ["/etc/README.md"]}), "safe repository path"), + "traversal pattern": (json.dumps({**base, "unscoped": ["docs/../secrets.md"]}), "safe repository path"), + "backslash pattern": (json.dumps({**base, "unscoped": ["docs\\README.md"]}), "safe repository path"), + "repeated pattern": (json.dumps({**base, "unscoped": ["README.md", "README.md"]}), "repeats a pattern"), + "ambiguous classification": ( + json.dumps({**base, "protected": [*base["protected"], "README.md"]}), + "both protected and unscoped", + ), + "dropped required protection": ( + json.dumps({**base, "protected": [p for p in base["protected"] if p != "crates/**"]}), + "drops required paths", + ), + } + for name, (text, expected) in mutations.items(): + with self.subTest(mutation=name): + with self.assertRaisesRegex(self.checker.EvidenceError, expected): + self.checker.validate_scope_manifest(text) + + def test_scope_manifest_floor_covers_the_attested_trust_root(self) -> None: + self.assertLessEqual(set(self.checker.REQUIRED_PROTECTED_PATHS), set(self.scope.protected)) + for path in ("crates/**", "packages/psyche-npm/**", ".github/workflows/ci.yml"): + self.assertEqual(self.scope.classify(path), "protected") + for path in ("README.md", "docs/ROADMAP.md", "agent/manifest.yaml"): + self.assertEqual(self.scope.classify(path), "unscoped") + self.assertEqual(self.scope.classify("scripts/check-g2-workspace.sh"), "protected") + self.assertEqual(self.scope.classify("docs/UNCLASSIFIED.md"), "protected") def test_remote_verifier_rejects_wrong_workflow_or_event(self) -> None: passed = self.passed_evidence() @@ -723,7 +973,7 @@ def test_remote_verifier_rejects_wrong_workflow_or_event(self) -> None: for field, value in (("workflowName", "Other"), ("event", "push")): completed = ( subprocess.CompletedProcess([], 0, "", ""), - subprocess.CompletedProcess([], 0, "docs/G2-EVIDENCE.md\n", ""), + subprocess.CompletedProcess([], 0, "M\tdocs/G2-EVIDENCE.md\n", ""), ) with self.subTest(field=field), mock.patch.dict( os.environ, {"GITHUB_ACTIONS": ""}, clear=False @@ -733,7 +983,7 @@ def test_remote_verifier_rejects_wrong_workflow_or_event(self) -> None: self.checker, "verify_coven_blob" ): with self.assertRaisesRegex(self.checker.EvidenceError, "CI attestation"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_remote_verifier_rejects_wrong_rest_workflow_path_or_repository(self) -> None: passed = self.passed_evidence() @@ -764,7 +1014,7 @@ def test_remote_verifier_rejects_wrong_rest_workflow_path_or_repository(self) -> for index, mutated in enumerate(mutations): completed = ( subprocess.CompletedProcess([], 0, "", ""), - subprocess.CompletedProcess([], 0, "docs/G2-EVIDENCE.md\n", ""), + subprocess.CompletedProcess([], 0, "M\tdocs/G2-EVIDENCE.md\n", ""), ) with self.subTest(index=index), mock.patch.dict( os.environ, {"GITHUB_ACTIONS": ""}, clear=False @@ -774,7 +1024,7 @@ def test_remote_verifier_rejects_wrong_rest_workflow_path_or_repository(self) -> self.checker, "verify_coven_blob" ): with self.assertRaisesRegex(self.checker.EvidenceError, "REST attestation"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_remote_verifier_rejects_inactive_workflow_metadata(self) -> None: passed = self.passed_evidence() @@ -794,7 +1044,7 @@ def test_remote_verifier_rejects_inactive_workflow_metadata(self) -> None: workflow = {"id": 326408880, "name": "CI", "path": ".github/workflows/ci.yml", "state": "disabled_manually"} completed = ( subprocess.CompletedProcess([], 0, "", ""), - subprocess.CompletedProcess([], 0, "docs/G2-EVIDENCE.md\n", ""), + subprocess.CompletedProcess([], 0, "M\tdocs/G2-EVIDENCE.md\n", ""), ) with mock.patch.dict(os.environ, {"GITHUB_ACTIONS": ""}, clear=False), mock.patch.object( self.checker.subprocess, "run", side_effect=completed @@ -802,7 +1052,7 @@ def test_remote_verifier_rejects_inactive_workflow_metadata(self) -> None: self.checker, "run_json", side_effect=(view, rest, workflow) ), mock.patch.object(self.checker, "verify_coven_blob"): with self.assertRaisesRegex(self.checker.EvidenceError, "workflow metadata"): - self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1]) + self.checker.verify_passed(ROOT, passed, self.checker.validate_evidence(passed)[1], self.scope) def test_architecture_lists_cli_direct_dependencies(self) -> None: architecture = (ROOT / "docs/ARCHITECTURE.md").read_text() diff --git a/scripts/check-g2-evidence.py b/scripts/check-g2-evidence.py index f5d8ce9..605791b 100644 --- a/scripts/check-g2-evidence.py +++ b/scripts/check-g2-evidence.py @@ -13,6 +13,8 @@ import sys import urllib.parse from collections.abc import Mapping +from fnmatch import fnmatchcase +from typing import NamedTuple class EvidenceError(RuntimeError): @@ -107,6 +109,45 @@ class EvidenceError(RuntimeError): REVIEWED_WORKFLOW_SHA256 = "1f908303c1a8940ce5ec8c81182ddaf5d82e5c6bddd5ece7fb33e1baf9a087f1" CI_WORKFLOW_ID = 326408880 +# The retained G2 attestation covers a scoped contract surface, not the whole +# future repository tree. `scripts/g2-scope-manifest.json` is the versioned, +# fail-closed classification of that surface: protected patterns enumerate the +# attested contract, unscoped patterns name the operational/readiness paths that +# may coexist with the retained attestation, and every unmatched path defaults +# to protected until it is explicitly classified. +SCOPE_MANIFEST_PATH = "scripts/g2-scope-manifest.json" +SCOPE_SCHEMA_VERSION = 1 +SCOPE_MANIFEST_KEYS = {"schema_version", "protected", "unscoped"} +EVIDENCE_PATH = "docs/G2-EVIDENCE.md" +# The manifest is itself protected, but the checker cannot trust a manifest edit +# to keep protecting the trust root, so this floor is enforced independently of +# the manifest's own content. It is the attested surface derived from what this +# checker validates, what the reviewed workflow exercises, and the trust root. +REQUIRED_PROTECTED_PATHS = ( + ".github/workflows/ci.yml", + "Cargo.lock", + "Cargo.toml", + "clippy.toml", + "crates/**", + "deny.toml", + "docs/ARCHITECTURE.md", + "docs/G2-EVIDENCE.md", + "docs/SCHEMAS.md", + "docs/TESTING.md", + "packages/psyche-npm/**", + "rust-toolchain.toml", + "scripts/check-g2-evidence-test.py", + "scripts/check-g2-evidence.py", + "scripts/g2-scope-manifest.json", + "scripts/g2-test-manifest.json", +) +# GitHub compare responses cap `files` at 300 entries and truncate silently, so +# a response at or above the cap cannot prove which paths changed. +COMPARE_FILE_LIMIT = 300 +# A retained attestation only survives added or modified unscoped paths. Every +# other change kind (rename, copy, delete, typechange, submodule) fails closed. +ALLOWED_UNSCOPED_STATUSES = frozenset({"added", "modified"}) + def fail(message: str) -> None: raise EvidenceError(message) @@ -791,16 +832,129 @@ def verify_coven_blob(root: pathlib.Path, url: str, expected_digest: str) -> Non fail(f"Coven content SHA-256 disagrees with evidence: {path}") -def verify_local_source_relationship(root: pathlib.Path, tested: str) -> None: +class AttestationScope(NamedTuple): + """Fail-closed classification of the retained G2 attestation surface.""" + + protected: tuple[str, ...] + unscoped: tuple[str, ...] + + def classify(self, path: str) -> str: + if any(fnmatchcase(path, pattern) for pattern in self.protected): + return "protected" + if any(fnmatchcase(path, pattern) for pattern in self.unscoped): + return "unscoped" + return "protected" + + +def require_safe_scope_pattern(pattern: str) -> None: + if ( + not pattern + or pattern.strip() != pattern + or pattern.startswith("/") + or pattern.endswith("/") + or "\\" in pattern + or ".." in pathlib.PurePosixPath(pattern).parts + ): + fail(f"scope manifest pattern is not a safe repository path: {pattern!r}") + + +def require_safe_diff_path(path: str) -> None: + if not path or path.startswith("/") or "\\" in path or "\0" in path or ".." in pathlib.PurePosixPath(path).parts: + fail(f"attestation comparison encountered an unsafe path: {path!r}") + + +def validate_scope_manifest(text: str) -> AttestationScope: + try: + data = json.loads(text) + except json.JSONDecodeError as error: + fail(f"scope manifest is invalid JSON: {error}") + if not isinstance(data, dict) or set(data) != SCOPE_MANIFEST_KEYS: + fail(f"scope manifest must contain exactly the keys {sorted(SCOPE_MANIFEST_KEYS)}") + version = data["schema_version"] + if isinstance(version, bool) or not isinstance(version, int) or version != SCOPE_SCHEMA_VERSION: + fail(f"scope manifest schema_version must be exactly {SCOPE_SCHEMA_VERSION}") + patterns: dict[str, tuple[str, ...]] = {} + for key in ("protected", "unscoped"): + entries = data[key] + if not isinstance(entries, list) or not entries or any(not isinstance(entry, str) for entry in entries): + fail(f"scope manifest {key} must be a non-empty list of path patterns") + for pattern in entries: + require_safe_scope_pattern(pattern) + if len(set(entries)) != len(entries): + fail(f"scope manifest {key} repeats a pattern") + patterns[key] = tuple(entries) + overlap = sorted(set(patterns["protected"]) & set(patterns["unscoped"])) + if overlap: + fail(f"scope manifest classifies paths as both protected and unscoped: {overlap}") + missing = sorted(set(REQUIRED_PROTECTED_PATHS) - set(patterns["protected"])) + if missing: + fail(f"scope manifest protected surface drops required paths: {missing}") + return AttestationScope(protected=patterns["protected"], unscoped=patterns["unscoped"]) + + +def validate_scoped_entries(entries: list[tuple[str, str]], scope: AttestationScope) -> None: + """Classify a tested-source-to-terminal diff, failing closed on ambiguity. + + Changing only the evidence document stays subject to the original exact + evidence-only relationship rules, which the callers enforce before this + classification. Diagnostics name the offending protected paths and never + quote their contents. + """ + if len(entries) == 1 and entries[0] == (EVIDENCE_PATH, "modified"): + return + unsupported = [(path, status) for path, status in entries if status not in ALLOWED_UNSCOPED_STATUSES] + if unsupported: + fail(f"changed paths fail closed by status (rename, copy, delete, typechange, submodule): {unsupported}") + protected = sorted({path for path, _ in entries if scope.classify(path) == "protected"}) + if protected: + fail(f"protected G2 attestation paths changed since the tested source: {protected}") + + +LOCAL_DIFF_STATUS = {"A": "added", "M": "modified", "D": "removed", "T": "typechange"} + + +def local_changed_entries(root: pathlib.Path, tested: str) -> list[tuple[str, str]]: + name_status = subprocess.run( + ["git", "diff", "--name-status", "-M", "-C", f"{tested}..HEAD"], + cwd=root, text=True, capture_output=True, check=True, + ).stdout + entries: list[tuple[str, str]] = [] + for line in name_status.splitlines(): + if not line.strip(): + continue + status, _, path = line.partition("\t") + letter = status[:1] + if letter in {"R", "C"}: + entries.append((path, "renamed" if letter == "R" else "copied")) + continue + mapped = LOCAL_DIFF_STATUS.get(letter) + if mapped is None or not path: + fail(f"local tested-source-to-HEAD diff contains an unsupported entry: {line!r}") + require_safe_diff_path(path) + entries.append((path, mapped)) + if not entries: + fail("local tested-source-to-HEAD diff is empty or unreadable") + if len(entries) == 1 and entries[0] == (EVIDENCE_PATH, "modified"): + return entries + raw = subprocess.run( + ["git", "diff", "--raw", f"{tested}..HEAD"], + cwd=root, text=True, capture_output=True, check=True, + ).stdout + for line in raw.splitlines(): + if not line.strip(): + continue + meta, _, path = line.partition("\t") + if "160000" in meta.split()[1:3]: + fail(f"submodule change fails closed: {path!r}") + return entries + + +def verify_local_source_relationship(root: pathlib.Path, tested: str, scope: AttestationScope) -> None: subprocess.run(["git", "merge-base", "--is-ancestor", tested, "HEAD"], cwd=root, check=True) - changed = subprocess.run( - ["git", "diff", "--name-only", f"{tested}..HEAD"], cwd=root, text=True, capture_output=True, check=True - ).stdout.splitlines() - if changed != ["docs/G2-EVIDENCE.md"]: - fail(f"passed source-to-HEAD diff is not evidence-only: {changed}") + validate_scoped_entries(local_changed_entries(root, tested), scope) -def validate_evidence_only_compare(compare: object, tested: str, terminal: str) -> None: +def validate_compare_provenance(compare: object, tested: str, terminal: str) -> None: if not isinstance(compare, dict): fail("GitHub compare response is invalid") base = compare.get("base_commit") @@ -823,17 +977,59 @@ def validate_evidence_only_compare(compare: object, tested: str, terminal: str) or merge_base.get("sha") != tested ): fail("tested source is not the pull-request head's merge-base ancestor") + + +def is_evidence_only_files(files: object) -> bool: + return ( + isinstance(files, list) + and len(files) == 1 + and isinstance(files[0], dict) + and files[0].get("filename") == EVIDENCE_PATH + and files[0].get("status") == "modified" + ) + + +def validate_evidence_only_compare(compare: object, tested: str, terminal: str) -> None: + """Original exact evidence-only relationship rules for a recorded attestation.""" + validate_compare_provenance(compare, tested, terminal) files = compare.get("files") - if ( - not isinstance(files, list) - or len(files) != 1 - or not isinstance(files[0], dict) - or files[0].get("filename") != "docs/G2-EVIDENCE.md" - or files[0].get("status") != "modified" - ): + if not is_evidence_only_files(files): fail(f"passed source-to-pull-request diff is not one modified evidence file: {files}") +def compare_file_entries(files: list[object]) -> list[tuple[str, str]]: + if len(files) >= COMPARE_FILE_LIMIT: + fail(f"GitHub compare response has {len(files)} files and may be truncated") + entries: list[tuple[str, str]] = [] + for entry in files: + if not isinstance(entry, dict): + fail("GitHub compare response contains a non-object file entry") + filename = entry.get("filename") + if not isinstance(filename, str): + fail(f"GitHub compare response file entry has no filename: {entry}") + require_safe_diff_path(filename) + status = entry.get("status") + if not isinstance(status, str) or not status: + fail(f"changed path has no comparable status: {filename}") + if "previous_filename" in entry: + fail(f"changed path fails closed as a rename or copy: {filename}") + if not isinstance(entry.get("blob_url"), str) or not isinstance(entry.get("raw_url"), str): + fail(f"changed path lacks blob provenance and fails closed (possible submodule): {filename}") + entries.append((filename, status)) + return entries + + +def validate_scoped_compare(compare: object, tested: str, terminal: str, scope: AttestationScope) -> None: + """Compare the tested source to the terminal commit by attestation scope.""" + validate_compare_provenance(compare, tested, terminal) + files = compare.get("files") + if is_evidence_only_files(files): + return + if not isinstance(files, list) or not files: + fail("GitHub compare response contains no file entries") + validate_scoped_entries(compare_file_entries(files), scope) + + def squash_merge_terminal(root: pathlib.Path, event: dict[str, object]) -> str: before = event.get("before") after = event.get("after") @@ -904,7 +1100,7 @@ def squash_merge_terminal(root: pathlib.Path, event: dict[str, object]) -> str: return terminal -def verify_actions_source_relationship(root: pathlib.Path, tested: str) -> None: +def verify_actions_source_relationship(root: pathlib.Path, tested: str, scope: AttestationScope) -> None: event_path = os.environ.get("GITHUB_EVENT_PATH") if not event_path: fail("GitHub Actions event payload path is absent") @@ -938,16 +1134,16 @@ def verify_actions_source_relationship(root: pathlib.Path, tested: str) -> None: ["gh", "api", f"repos/OpenCoven/psyche/compare/{tested}...{terminal}"], root, ) - validate_evidence_only_compare(compare, tested, terminal) + validate_scoped_compare(compare, tested, terminal, scope) -def verify_passed(root: pathlib.Path, markdown: str, source_rows: list[list[str]]) -> None: +def verify_passed(root: pathlib.Path, markdown: str, source_rows: list[list[str]], scope: AttestationScope) -> None: tested = field(markdown, "Tested source commit") run_url = field(markdown, "CI attestation") if os.environ.get("GITHUB_ACTIONS") == "true": - verify_actions_source_relationship(root, tested) + verify_actions_source_relationship(root, tested, scope) else: - verify_local_source_relationship(root, tested) + verify_local_source_relationship(root, tested, scope) match = re.fullmatch(r"https://github\.com/(OpenCoven)/(psyche)/actions/runs/([0-9]+)", run_url) if not match: fail("CI attestation URL is malformed") @@ -1025,13 +1221,14 @@ def validate_repository( manifest_data = json.loads(read_text(root, "scripts/g2-test-manifest.json", overrides)) except json.JSONDecodeError as error: fail(f"G2 manifest is invalid JSON: {error}") + scope = validate_scope_manifest(read_text(root, SCOPE_MANIFEST_PATH, overrides)) evidence_text = evidence if evidence is not None else read_text(root, "docs/G2-EVIDENCE.md", overrides) status, source_rows, matrix = validate_evidence(evidence_text) validate_manifest(root, manifest_data, matrix, listed_tests) validate_sources(root, overrides) validate_docs(root, overrides) if status == "passed" and verify_remote: - verify_passed(root, evidence_text, source_rows) + verify_passed(root, evidence_text, source_rows, scope) def main() -> int: diff --git a/scripts/g2-scope-manifest.json b/scripts/g2-scope-manifest.json new file mode 100644 index 0000000..83906dc --- /dev/null +++ b/scripts/g2-scope-manifest.json @@ -0,0 +1,36 @@ +{ + "schema_version": 1, + "protected": [ + ".github/workflows/ci.yml", + "Cargo.lock", + "Cargo.toml", + "clippy.toml", + "crates/**", + "deny.toml", + "docs/ARCHITECTURE.md", + "docs/G2-EVIDENCE.md", + "docs/SCHEMAS.md", + "docs/TESTING.md", + "packages/psyche-npm/**", + "rust-toolchain.toml", + "scripts/check-g2-evidence-test.py", + "scripts/check-g2-evidence.py", + "scripts/g2-scope-manifest.json", + "scripts/g2-test-manifest.json" + ], + "unscoped": [ + ".gitignore", + "AGENTS.md", + "CONTRIBUTING.md", + "LICENSE", + "README.md", + "SECURITY.md", + "agent/**", + "docs/CLI.md", + "docs/CONFIGURATION.md", + "docs/PROTOCOL-OWNERSHIP.md", + "docs/ROADMAP.md", + "scripts/agent-bootstrap", + "scripts/agent-check" + ] +} From 00430d27ca96925002e3f1714b10ad9a70e90efe Mon Sep 17 00:00:00 2001 From: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com> Date: Sun, 30 Aug 2026 16:15:41 +0000 Subject: [PATCH 2/2] chore: re-trigger pull_request CI on the fork vehicle Signed-off-by: Timothy Wayne Gregg <5861166+CompleteDotTech@users.noreply.github.com>