diff --git a/docs/CI.md b/docs/CI.md index 8540322..9d3fcaf 100644 --- a/docs/CI.md +++ b/docs/CI.md @@ -31,22 +31,28 @@ Individual commands are available when iterating: | `make fuzz-smoke` | Run each discovered fuzz target for a fixed one-second smoke window, or print an explicit `SKIPPED` result when no target exists. | | `make deps` | Require a clean `go mod tidy -diff`, verified module sums and a read-only dependency load. | | `make licenses` | Compare the exact runtime module/version/replacement graph with its inventory and require a top-level license file. | -| `make experiments` | Require both established G01/G02 modules, run their default race/vet suites, then exercise the two explicitly reviewed G01 CLI packages with `g01_live,g01_worker` tags and the reviewed `g01_pair_fixture` livecanary collection/terminal partitions with tagged vet. | +| `make experiments` | Require both established G01/G02 modules, run their default race/vet suites, then exercise the two explicitly reviewed G01 CLI packages with `g01_live,g01_worker` tags and the reviewed `g01_pair_fixture` livecanary collection/listener and terminal partitions with tagged vet. | | `make vuln` | Run the exact `golang.org/x/vuln/cmd/govulncheck@v1.7.0` tool. | No hardware, live GitHub, Docker or daemon suite is part of this public check. Those profiles remain explicit future or maintainer-controlled runs; they are not silently converted into passing tests here. G04 introduces the first application behavior contracts and should add meaningful unit and fuzz targets before claiming those forms of coverage. The tagged CLI tests use synthetic input/subprocess fixtures and static plan or refusal paths. The `g01_pair_fixture` livecanary checks use private synthetic -fixtures: one collection run excludes `^TestPairedTerminal`, one terminal run -selects that prefix while skipping the reviewed persistence set, and a third run -selects the exact persistence set +fixtures: one paired-collection run selects `^TestPaired` while excluding +`^TestPairedTerminal`, one remaining collection/listener run has no `-run` +filter while excluding `^TestPaired`, one terminal run selects that prefix while +skipping the reviewed persistence set, and a fourth run selects the exact +persistence set `^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$`. `FixtureStorageFailure` is the unique generated sentinel from `scripts/tooling_test.go`; it is intentionally distinct from the real `TestPairedTerminalClosedReplayActualFile` test. This exact expression is the script's `storage_regex` value and is also the `STORAGE` alias in issue #54. -One tagged vet follows those three race-tested runs. None of these tagged checks +The two collection/listener partitions are explicit and disjoint: every +non-terminal `TestPaired*` test is in the first, and every other test, example or +fuzz seed is in the second. The unfiltered second command preserves Go's normal +execution of tagged examples and fuzz seeds. One tagged vet follows those four +race-tested runs. None of these tagged checks executes approved live controller/worker operations or exposes a public terminal phase/API. The implementation and evidence boundaries are recorded in the [G01 paired terminal guide](evidence/g01-paired-terminal.md). The script permits diff --git a/docs/evidence/g01-paired-baseline.md b/docs/evidence/g01-paired-baseline.md index ef368e0..b50aa20 100644 --- a/docs/evidence/g01-paired-baseline.md +++ b/docs/evidence/g01-paired-baseline.md @@ -151,15 +151,20 @@ Root independently reproduced that tooling red and green. From `experiments/g01-scaleset`, required fixture checks are: ```text -GOTOOLCHAIN=go1.26.8 go test -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip '^TestPairedTerminal' ./livecanary +GOTOOLCHAIN=go1.26.8 go test -race -count=1 -timeout=120s -tags=g01_pair_fixture -run '^TestPaired' -skip '^TestPairedTerminal' ./livecanary +GOTOOLCHAIN=go1.26.8 go test -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip '^TestPaired' ./livecanary ``` -The terminal behavior and terminal persistence/identity partitions, including -their exact storage expression, are maintained in the canonical +The first command selects the non-terminal paired collection tests; the second +leaves `-run` unrestricted so tagged tests, examples and fuzz seeds remain +eligible while excluding the paired prefix. The commands are exhaustive and +disjoint for the collection/listener work. The terminal behavior and terminal +persistence/identity partitions, including their exact storage expression, are +maintained in the canonical [terminal evidence guide](g01-paired-terminal.md). Use the repository's -`scripts/check-offline-experiments.sh` for the complete three-way invocation; -the collection/listener command above is shown here only to identify this -collection entry's partition. +`scripts/check-offline-experiments.sh` for the complete four-way invocation; +the collection/listener commands above are shown here only to identify the +collection entry's partitions. ### Current correction checkpoint diff --git a/docs/evidence/g01-paired-terminal.md b/docs/evidence/g01-paired-terminal.md index c8a535a..00fe79c 100644 --- a/docs/evidence/g01-paired-terminal.md +++ b/docs/evidence/g01-paired-terminal.md @@ -194,14 +194,16 @@ Sync-error fixtures inject failures at actual file-write/sync boundaries and do not claim physical power-loss durability. The trust model remains reviewed Go code and private local files, not hostile same-UID code or copying a used mutex. -The independently reviewed Luna tooling fragment runs three complementary tagged -partitions: collection, terminal excluding the explicitly named persistence -tests, and those persistence tests. Each retains the exact toolchain, race -detector, count one and a 120-second timeout, plus one tagged vet pass: +The independently reviewed Luna tooling fragment runs four complementary tagged +partitions: paired collection, remaining collection/listener work, terminal +excluding the explicitly named persistence tests, and those persistence tests. +Each retains the exact toolchain, race detector, count one and a 120-second +timeout, plus one tagged vet pass: ```sh terminal_storage_tests='^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$' -GOTOOLCHAIN=go1.26.8 go test -C experiments/g01-scaleset -tags=g01_pair_fixture -race -count=1 -timeout=120s ./livecanary -skip '^TestPairedTerminal' +GOTOOLCHAIN=go1.26.8 go test -C experiments/g01-scaleset -tags=g01_pair_fixture -race -count=1 -timeout=120s ./livecanary -run '^TestPaired' -skip '^TestPairedTerminal' +GOTOOLCHAIN=go1.26.8 go test -C experiments/g01-scaleset -tags=g01_pair_fixture -race -count=1 -timeout=120s ./livecanary -skip '^TestPaired' GOTOOLCHAIN=go1.26.8 go test -C experiments/g01-scaleset -tags=g01_pair_fixture -race -count=1 -timeout=120s ./livecanary -run '^TestPairedTerminal' -skip "$terminal_storage_tests" GOTOOLCHAIN=go1.26.8 go test -C experiments/g01-scaleset -tags=g01_pair_fixture -race -count=1 -timeout=120s ./livecanary -run "$terminal_storage_tests" GOTOOLCHAIN=go1.26.8 go vet -C experiments/g01-scaleset -tags=g01_pair_fixture ./livecanary @@ -211,9 +213,14 @@ GOTOOLCHAIN=go1.26.8 go vet -C experiments/g01-scaleset -tags=g01_pair_fixture . `scripts/check-offline-experiments.sh` and is the `STORAGE` alias in issue #54. The generated tooling sentinel is named `TestPairedTerminalFixtureStorageFailure`, so its top-level regex term is `FixtureStorageFailure`; it must remain in the -storage-only group. The three groups are exhaustive and disjoint: every other -top-level `TestPairedTerminal` test stays in terminal behavior, while tests -outside that prefix stay in collection/listener. +storage-only group. The two terminal groups remain exhaustive and disjoint: +every other top-level `TestPairedTerminal` test stays in terminal behavior, +while only the named persistence tests stay in storage. The two +collection/listener groups are likewise exhaustive and disjoint: non-terminal +`TestPaired*` tests stay in paired collection, and every other test, example or +fuzz seed stays in remaining collection/listener. The unrestricted remaining +command is intentional because Go's `-run` filter also selects examples and +fuzz tests; `-skip '^TestPaired'` excludes only the paired test prefix. The fixture tag remains excluded with either live command tag. The earlier two-part tooling red diff --git a/scripts/check-offline-experiments.sh b/scripts/check-offline-experiments.sh index 9b2781d..dfa5540 100644 --- a/scripts/check-offline-experiments.sh +++ b/scripts/check-offline-experiments.sh @@ -4,6 +4,7 @@ set -euo pipefail go_cmd="${GO:-go}" exact_toolchain="go1.26.8" +paired_collection_regex='^TestPaired' storage_regex='^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$' # These are the two established offline gate modules. Keep this list explicit so @@ -35,7 +36,11 @@ for module_dir in "${offline_modules[@]}"; do # paths only. Do not discover arbitrary opt-in tags or platform probes. GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" test -race -count=1 -timeout=45s -tags=g01_live,g01_worker ./cmd/g01-live ./cmd/g01-worker GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" vet -tags=g01_live,g01_worker ./cmd/g01-live ./cmd/g01-worker - GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" test -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip '^TestPairedTerminal' ./livecanary + # Keep the non-terminal collection/listener tests exhaustive and disjoint: + # paired collection first, then every non-paired test. The terminal + # partitions below remain the only consumers of TestPairedTerminal. + GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" test -race -count=1 -timeout=120s -tags=g01_pair_fixture -run "${paired_collection_regex}" -skip '^TestPairedTerminal' ./livecanary + GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" test -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip "${paired_collection_regex}" ./livecanary GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" test -race -count=1 -timeout=120s -tags=g01_pair_fixture -run '^TestPairedTerminal' -skip "${storage_regex}" ./livecanary GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" test -race -count=1 -timeout=120s -tags=g01_pair_fixture -run "${storage_regex}" ./livecanary GOTOOLCHAIN="${exact_toolchain}" "${go_cmd}" vet -tags=g01_pair_fixture ./livecanary diff --git a/scripts/tooling_test.go b/scripts/tooling_test.go index 46b92da..4df9504 100644 --- a/scripts/tooling_test.go +++ b/scripts/tooling_test.go @@ -163,20 +163,133 @@ func TestToolingTaggedCLIRegressionRuns(t *testing.T) { func TestToolingTaggedPairFixturePartitionsRun(t *testing.T) { root := toolingFixture(t) - const collectionSentinel = "tagged-pair-fixture-collection-regression" + const pairedCollectionSentinel = "tagged-pair-fixture-paired-collection-regression" + const listenerSentinel = "tagged-pair-fixture-listener-regression" + const exampleSentinel = "tagged-pair-fixture-example-regression" + const fuzzSentinel = "tagged-pair-fixture-fuzz-seed-regression" const terminalSentinel = "tagged-pair-fixture-terminal-regression" const storageSentinel = "tagged-pair-fixture-storage-regression" - collectionPath := "experiments/g01-scaleset/livecanary/pair_fixture_collection_regression_test.go" + pairedCollectionPath := "experiments/g01-scaleset/livecanary/pair_fixture_paired_collection_regression_test.go" + listenerPath := "experiments/g01-scaleset/livecanary/pair_fixture_listener_regression_test.go" + examplePath := "experiments/g01-scaleset/livecanary/pair_fixture_example_regression_test.go" + fuzzPath := "experiments/g01-scaleset/livecanary/pair_fixture_fuzz_regression_test.go" terminalPath := "experiments/g01-scaleset/livecanary/pair_fixture_terminal_regression_test.go" storagePath := "experiments/g01-scaleset/livecanary/pair_fixture_storage_regression_test.go" - collectionSource := `//go:build g01_pair_fixture && !g01_live && !g01_worker + remainingCollectionInvocation := "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip ^TestPaired ./livecanary" + taggedPositiveSource := func(testName, marker string) string { + return `//go:build g01_pair_fixture && !g01_live && !g01_worker + +package livecanary + +import ( + "os" + "testing" +) + +func ` + testName + `(t *testing.T) { + path := os.Getenv("TOOLING_SENTINEL_LOG") + if path == "" { + t.Fatal("TOOLING_SENTINEL_LOG is not set") + } + file, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600) + if err != nil { + t.Fatal(err) + } + defer file.Close() + if _, err := file.WriteString("` + marker + `\n"); err != nil { + t.Fatal(err) + } +} +` + } + pairedCollectionPositiveSource := taggedPositiveSource("TestPairedCollectionFixturePass", "paired-collection-pass") + listenerPositiveSource := taggedPositiveSource("TestListenerFixturePass", "listener-pass") + taggedExampleSource := func(exampleName, marker, expected string) string { + return `//go:build g01_pair_fixture && !g01_live && !g01_worker + +package livecanary + +import ( + "fmt" + "os" +) + +func ` + exampleName + `() { + path := os.Getenv("TOOLING_SENTINEL_LOG") + if path == "" { + panic("TOOLING_SENTINEL_LOG is not set") + } + file, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600) + if err != nil { + panic(err) + } + defer file.Close() + if _, err := file.WriteString("` + marker + `\n"); err != nil { + panic(err) + } + fmt.Println("` + marker + `") + // Output: ` + expected + ` +} +` + } + taggedFuzzSource := func(fuzzName, marker, failure string) string { + failureLine := "" + if failure != "" { + failureLine = ` + t.Fatal("` + failure + `")` + } + return `//go:build g01_pair_fixture && !g01_live && !g01_worker + +package livecanary + +import ( + "os" + "testing" +) + +func ` + fuzzName + `(f *testing.F) { + f.Add("fixture-seed") + f.Fuzz(func(t *testing.T, _ string) { + path := os.Getenv("TOOLING_SENTINEL_LOG") + if path == "" { + t.Fatal("TOOLING_SENTINEL_LOG is not set") + } + file, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600) + if err != nil { + t.Fatal(err) + } + defer file.Close() + if _, err := file.WriteString("` + marker + `\n"); err != nil { + t.Fatal(err) + }` + failureLine + ` + }) +} +` + } + examplePositiveSource := taggedExampleSource("Example_fixturePass", "example-pass", "example-pass") + exampleSource := taggedExampleSource("Example_fixtureFailure", exampleSentinel, "unexpected-example-output") + fuzzPositiveSource := taggedFuzzSource("FuzzRemainingFixturePass", "fuzz-pass", "") + fuzzSource := taggedFuzzSource("FuzzRemainingFixtureFailure", fuzzSentinel, fuzzSentinel) + terminalPositiveSource := taggedPositiveSource("TestPairedTerminalFixturePass", "terminal-pass") + storagePositiveSource := taggedPositiveSource("TestPairedTerminalFixtureStorageFailure", "storage-pass") + pairedCollectionSource := `//go:build g01_pair_fixture && !g01_live && !g01_worker package livecanary import "testing" -func TestTaggedPairFixtureCollectionFailure(t *testing.T) { - t.Fatal("tagged-pair-fixture-collection-regression") +func TestPairedCollectionFixtureFailure(t *testing.T) { + t.Fatal("tagged-pair-fixture-paired-collection-regression") +} +` + listenerSource := `//go:build g01_pair_fixture && !g01_live && !g01_worker + +package livecanary + +import "testing" + +func TestListenerFixtureFailure(t *testing.T) { + t.Fatal("tagged-pair-fixture-listener-regression") } ` terminalSource := `//go:build g01_pair_fixture && !g01_live && !g01_worker @@ -200,26 +313,92 @@ func TestPairedTerminalFixtureStorageFailure(t *testing.T) { } ` wrapper, logPath, realGo := toolingGoWrapper(t, root) - env := []string{"GO=" + wrapper, "TOOLING_REAL_GO=" + realGo, "TOOLING_GO_LOG=" + logPath} + sentinelLogPath := filepath.Join(root, "tagged-sentinel.log") + env := []string{"GO=" + wrapper, "TOOLING_REAL_GO=" + realGo, "TOOLING_GO_LOG=" + logPath, "TOOLING_SENTINEL_LOG=" + sentinelLogPath} // Each generated witness must fail through its own reviewed partition. The - // earlier script already executed all tagged terminal witnesses together; - // these controls retain that coverage while the log assertion below proves - // the three-way partition contract. - for _, tc := range []struct { - path, sentinel, name, source string + // terminal witnesses retain their established behavior while the log + // assertions below prove the four-way partition contract. + partitions := []struct { + path, sentinel, name, source, positiveSource, invocation string }{ - {collectionPath, collectionSentinel, "collection", collectionSource}, - {terminalPath, terminalSentinel, "terminal", terminalSource}, - {storagePath, storageSentinel, "storage", storageSource}, - } { + { + pairedCollectionPath, + pairedCollectionSentinel, + "paired-collection", + pairedCollectionSource, + pairedCollectionPositiveSource, + "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -run ^TestPaired -skip ^TestPairedTerminal ./livecanary", + }, + { + listenerPath, + listenerSentinel, + "listener", + listenerSource, + listenerPositiveSource, + remainingCollectionInvocation, + }, + { + examplePath, + exampleSentinel, + "example", + exampleSource, + examplePositiveSource, + remainingCollectionInvocation, + }, + { + fuzzPath, + fuzzSentinel, + "fuzz", + fuzzSource, + fuzzPositiveSource, + remainingCollectionInvocation, + }, + { + terminalPath, + terminalSentinel, + "terminal", + terminalSource, + terminalPositiveSource, + "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -run ^TestPairedTerminal -skip ^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$ ./livecanary", + }, + { + storagePath, + storageSentinel, + "storage", + storageSource, + storagePositiveSource, + "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -run ^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$ ./livecanary", + }, + } + for _, tc := range partitions { toolingFile(t, root, tc.path, tc.source, 0600) - if out, err := toolingRun(t, root, env, "make", "check"); err == nil || !strings.Contains(out, tc.sentinel) { - t.Fatalf("tagged %s failure was skipped: %s", tc.name, out) + if err := os.WriteFile(logPath, nil, 0600); err != nil { + t.Fatal(err) + } + if out, err := toolingRun(t, root, env, "bash", "scripts/check-offline-experiments.sh"); err == nil || !strings.Contains(out, tc.sentinel) { + t.Errorf("tagged %s failure was skipped: %s", tc.name, out) + } + data, err := os.ReadFile(logPath) + if err != nil { + t.Fatal(err) + } + lines := strings.Split(strings.TrimSpace(string(data)), "\n") + count := 0 + for _, line := range lines { + if line == tc.invocation { + count++ + } + } + if count != 1 { + t.Errorf("tagged %s failure used invocation %q %d times; wrapper log:\n%s", tc.name, tc.invocation, count, data) } if err := os.Remove(filepath.Join(root, tc.path)); err != nil { t.Fatal(err) } } + for _, tc := range partitions { + toolingFile(t, root, tc.path, tc.positiveSource, 0600) + } if err := os.WriteFile(logPath, nil, 0600); err != nil { t.Fatal(err) } @@ -231,10 +410,27 @@ func TestPairedTerminalFixtureStorageFailure(t *testing.T) { if err != nil { t.Fatal(err) } + sentinelData, err := os.ReadFile(sentinelLogPath) + if err != nil { + t.Fatal(err) + } + sentinelLines := strings.Split(strings.TrimSpace(string(sentinelData)), "\n") + for _, marker := range []string{"paired-collection-pass", "listener-pass", "example-pass", "fuzz-pass", "terminal-pass", "storage-pass"} { + count := 0 + for _, line := range sentinelLines { + if line == marker { + count++ + } + } + if count != 1 { + t.Fatalf("positive sentinel %q ran %d times; sentinel log:\n%s", marker, count, sentinelData) + } + } log := string(data) lines := strings.Split(strings.TrimSpace(log), "\n") for _, invocation := range []string{ - "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip ^TestPairedTerminal ./livecanary", + "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -run ^TestPaired -skip ^TestPairedTerminal ./livecanary", + remainingCollectionInvocation, "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -run ^TestPairedTerminal -skip ^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$ ./livecanary", "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -run ^TestPairedTerminal(Actual(Controller|Worker)SyncFailures|PostIntent(JournalIdentity|AuthorityBoundaries)|ClosedReplayActualFile|WorkerReceiptSurvivesControllerWriteFailure|FixtureStorageFailure)$ ./livecanary", "go1.26.8\tvet -tags=g01_pair_fixture ./livecanary", @@ -249,6 +445,12 @@ func TestPairedTerminalFixtureStorageFailure(t *testing.T) { t.Fatalf("offline gate logged %q %d times; wrapper log:\n%s", invocation, count, log) } } + legacyCollectionInvocation := "go1.26.8\ttest -race -count=1 -timeout=120s -tags=g01_pair_fixture -skip ^TestPairedTerminal ./livecanary" + for _, line := range lines { + if line == legacyCollectionInvocation { + t.Fatalf("offline gate retained the unsplit collection invocation: %s", line) + } + } } func TestToolingLicenseIdentityAndStaleRows(t *testing.T) {