fix(ci): preserve helper scripts through Nix setup - #1092
Merged
schickling-assistant merged 4 commits intoAug 20, 2026
Conversation
schickling
force-pushed
the
schickling/2026-08-20-fix-ci-helper-lifetime
branch
from
August 20, 2026 13:43
5305b1a to
ae0b56e
Compare
Storybook Previews
Report historyPR 1092 · 2026-08-20 18:47 UTC
PR 1092 · 2026-08-20 18:19 UTC
PR 1092 · 2026-08-20 17:47 UTC
PR 1092 · 2026-08-20 13:57 UTC
|
schickling
force-pushed
the
schickling/2026-08-20-fix-ci-helper-lifetime
branch
from
August 20, 2026 13:52
ae0b56e to
c4d2ef7
Compare
schickling
marked this pull request as ready for review
August 20, 2026 14:16
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4d2ef753a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f11f30a88
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3657f24858
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
CI Measurementspartial - advisory gate - readiness
devenv / genie
nix / closures / packages
Unchanged / 0-impact measurements (16)These rows had compatible baseline data, but their semantic impact rounded to 0.00x because the movement was below the configured budget, below the noise floor, or inside the robust noise band. devenv / devenv cli
devenv / devenv shell
devenv / genie
devenv / quality gates
devenv / workspace setup
nix / closures / packages
Diagnostic / ungated measurements (7)source / effect-utils
devenv / devenv shell
All measurements
Previous runs
Source-of-truth JSON{
"schemaVersion": 1,
"title": "CI Measurements",
"status": "partial",
"gate": "advisory",
"readiness": "partial (17/26 enabled observations gateable)",
"commit": {
"shortSha": "eed40ce",
"sha": "eed40ce06942d30ebb4f2199047baddfe52a1a0d"
},
"run": {
"id": "32404763568",
"attempt": "1",
"url": "https://github.com/overengineeringstudio/effect-utils/actions/runs/32404763568"
},
"baseline": null,
"protocol": "devenv-perf-warm-median-v2",
"chart": {
"meaning": "semantic-impact",
"zeroImpactMeaning": "no actionable PR impact after budgets, noise floor, and robust evidence checks",
"svg": "https://raw.githubusercontent.com/overengineeringstudio/effect-utils/ci-measurement-assets/ci-measurements/pr-1092/eed40ce06942d30ebb4f2199047baddfe52a1a0d/run-32404763568-attempt-1/ci-measurements.svg",
"lightPng": "https://raw.githubusercontent.com/overengineeringstudio/effect-utils/ci-measurement-assets/ci-measurements/pr-1092/eed40ce06942d30ebb4f2199047baddfe52a1a0d/run-32404763568-attempt-1/ci-measurements.png",
"darkPng": "https://raw.githubusercontent.com/overengineeringstudio/effect-utils/ci-measurement-assets/ci-measurements/pr-1092/eed40ce06942d30ebb4f2199047baddfe52a1a0d/run-32404763568-attempt-1/ci-measurements-dark.png"
},
"measurements": [
{
"id": "task_genie_run",
"label": "Genie run task",
"group": "devenv / genie",
"path": [
"devenv",
"genie"
],
"groupPath": [
"devenv",
"genie"
],
"status": "pass",
"direction": "improved",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 1.549,
"current": 1.519,
"delta": -0.030000000000000027,
"ratio": 0.9806326662362814,
"semanticImpactScore": -0.244,
"semanticImpactKind": "improvement",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.119,
"evidenceDeltaUpper": -0.061,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "task_genie_run",
"probeLabel": "Genie run task",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "nix.closure.nar_size",
"label": "Total closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"total",
"closure-size",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 174111832,
"current": 174115928,
"delta": 4096,
"ratio": 1.0000235251099994,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10481664,
"evidenceDeltaUpper": 10489856,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.serialized_nar_size",
"label": "Total serialized NAR size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"total",
"serialized-nar-size",
"nix closure diagnostics"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "regressed",
"gateable": true,
"gateReason": "eligible",
"confidence": "within_budget",
"comparisonMode": "diagnostic",
"unit": "bytes",
"baseline": 174111832,
"current": 174115928,
"delta": 4096,
"ratio": 1.0000235251099994,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -17407087.2,
"evidenceDeltaUpper": 17415279.2,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total",
"sizeKind": "nar"
}
},
{
"id": "source.lines",
"label": "Genie CI workflow helpers lines",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"genie",
"ci-workflow",
"source / ci"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "lines",
"baseline": 7931,
"current": 8863,
"delta": 932,
"ratio": 1.1175135544067583,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 5,
"currentSamples": 11,
"pairedSamples": 0,
"evidenceDeltaLower": 138.89999999999998,
"evidenceDeltaUpper": 1725.1,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_ci_workflow"
}
},
{
"id": "source.lines",
"label": "Genie runtime lines",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"packages",
"genie",
"source / genie"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "lines",
"baseline": 27496,
"current": 27636,
"delta": 140,
"ratio": 1.005091649694501,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 5,
"currentSamples": 95,
"pairedSamples": 0,
"evidenceDeltaLower": -2609.6000000000004,
"evidenceDeltaUpper": 2889.6000000000004,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_runtime"
}
},
{
"id": "source.files",
"label": "Genie CI workflow helpers files",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"genie",
"ci-workflow",
"source / ci"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "count",
"baseline": 10,
"current": 11,
"delta": 1,
"ratio": 1.1,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 5,
"currentSamples": 11,
"pairedSamples": 0,
"evidenceDeltaLower": 0,
"evidenceDeltaUpper": 2,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_ci_workflow"
}
},
{
"id": "task_check_quick_warm",
"label": "Warm cached check:quick",
"group": "devenv / quality gates",
"path": [
"devenv",
"quality gates",
"check:quick"
],
"groupPath": [
"devenv",
"quality gates"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "within_budget",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 8.632,
"current": 8.412,
"delta": -0.21999999999999886,
"ratio": 0.9745134383688602,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.449,
"evidenceDeltaUpper": 0.247,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"workload": "cached-no-op",
"taskCacheMode": "warm",
"probe": "task_check_quick_warm",
"probeLabel": "Warm cached check:quick",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "shell_eval_warm",
"label": "Warm shell eval",
"group": "devenv / devenv shell",
"path": [
"devenv",
"devenv shell"
],
"groupPath": [
"devenv",
"devenv shell"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "within_budget",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 8.395,
"current": 8.257,
"delta": -0.1379999999999999,
"ratio": 0.9835616438356164,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.203,
"evidenceDeltaUpper": 0.168,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "shell_eval_warm",
"probeLabel": "Warm shell eval",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "task_pnpm_install",
"label": "pnpm install task",
"group": "devenv / workspace setup",
"path": [
"devenv",
"workspace setup"
],
"groupPath": [
"devenv",
"workspace setup"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 0.785,
"current": 0.696,
"delta": -0.08900000000000008,
"ratio": 0.886624203821656,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.14,
"evidenceDeltaUpper": 0.033,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "task_pnpm_install",
"probeLabel": "pnpm install task",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "task_check_quick_forced",
"label": "Forced check:quick",
"group": "devenv / quality gates",
"path": [
"devenv",
"quality gates",
"check:quick"
],
"groupPath": [
"devenv",
"quality gates"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 14.127,
"current": 14.161,
"delta": 0.03399999999999892,
"ratio": 1.0024067388688327,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 3,
"currentSamples": 3,
"pairedSamples": 3,
"evidenceDeltaLower": -0.38,
"evidenceDeltaUpper": 0.034,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"workload": "forced-task-cache",
"taskCacheMode": "refresh",
"probe": "task_check_quick_forced",
"probeLabel": "Forced check:quick",
"status": 0,
"sampleCount": 6,
"warmupCount": 0,
"measuredSampleCount": 3,
"pairedSampleCount": 3,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "genie_check_task",
"label": "Genie check task",
"group": "devenv / genie",
"path": [
"devenv",
"genie"
],
"groupPath": [
"devenv",
"genie"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 3.921,
"current": 3.894,
"delta": -0.02699999999999969,
"ratio": 0.993114001530222,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.061,
"evidenceDeltaUpper": 0.012,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "genie_check_task",
"probeLabel": "Genie check task",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "processes_help",
"label": "devenv processes --help",
"group": "devenv / devenv cli",
"path": [
"devenv",
"devenv cli"
],
"groupPath": [
"devenv",
"devenv cli"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 0.023,
"current": 0.019,
"delta": -0.004,
"ratio": 0.8260869565217391,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 9,
"currentSamples": 9,
"pairedSamples": 9,
"evidenceDeltaLower": -0.006,
"evidenceDeltaUpper": 0.001,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "processes_help",
"probeLabel": "devenv processes --help",
"status": 0,
"sampleCount": 19,
"warmupCount": 1,
"measuredSampleCount": 9,
"pairedSampleCount": 9,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "tasks_list",
"label": "devenv tasks list",
"group": "devenv / devenv cli",
"path": [
"devenv",
"devenv cli"
],
"groupPath": [
"devenv",
"devenv cli"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 0.045,
"current": 0.045,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 9,
"currentSamples": 9,
"pairedSamples": 9,
"evidenceDeltaLower": -0.005,
"evidenceDeltaUpper": 0.002,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "tasks_list",
"probeLabel": "devenv tasks list",
"status": 0,
"sampleCount": 19,
"warmupCount": 1,
"measuredSampleCount": 9,
"pairedSampleCount": 9,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "source.files",
"label": "Genie runtime files",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"packages",
"genie",
"source / genie"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "unchanged",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "count",
"baseline": 95,
"current": 95,
"delta": 0,
"ratio": 1,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 5,
"currentSamples": 95,
"pairedSamples": 0,
"evidenceDeltaLower": -9.5,
"evidenceDeltaUpper": 9.5,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_runtime"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Nix sources closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"buckets",
"nix-sources",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "nix-sources"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Nix sources closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"buckets",
"nix-sources",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "nix-sources"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Nix sources closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"buckets",
"nix-sources",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "nix-sources"
}
},
{
"id": "source.files",
"label": "Nix workspace tools files",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"nix",
"workspace-tools",
"source / nix"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "unchanged",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "count",
"baseline": 16,
"current": 16,
"delta": 0,
"ratio": 1,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 5,
"currentSamples": 16,
"pairedSamples": 0,
"evidenceDeltaLower": -1.6,
"evidenceDeltaUpper": 1.6,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "nix_workspace_tools"
}
},
{
"id": "source.lines",
"label": "Nix workspace tools lines",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"nix",
"workspace-tools",
"source / nix"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "unchanged",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "lines",
"baseline": 4291,
"current": 4291,
"delta": 0,
"ratio": 1,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 5,
"currentSamples": 16,
"pairedSamples": 0,
"evidenceDeltaLower": -429.1,
"evidenceDeltaUpper": 429.1,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "nix_workspace_tools"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Node / pnpm closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"buckets",
"node",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "node"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Node / pnpm closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"buckets",
"node",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "node"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Node / pnpm closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"buckets",
"node",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "node"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Rust closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"buckets",
"rust",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "rust"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Rust closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"buckets",
"rust",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "rust"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Rust closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"buckets",
"rust",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "rust"
}
},
{
"id": "nix.closure.path_count",
"label": "Total closure path count",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"total",
"path-count",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "count",
"baseline": 83,
"current": 83,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10,
"evidenceDeltaUpper": 10,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.path_count",
"label": "Total closure path count",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"total",
"path-count",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "count",
"baseline": 5,
"current": 5,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10,
"evidenceDeltaUpper": 10,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.path_count",
"label": "Total closure path count",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"total",
"path-count",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "count",
"baseline": 8,
"current": 8,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10,
"evidenceDeltaUpper": 10,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.nar_size",
"label": "Total closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"total",
"closure-size",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 612328280,
"current": 612328280,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -12246565.6,
"evidenceDeltaUpper": 12246565.6,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.nar_size",
"label": "Total closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"total",
"closure-size",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 159040112,
"current": 159040112,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.serialized_nar_size",
"label": "Total serialized NAR size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"total",
"serialized-nar-size",
"nix closure diagnostics"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "diagnostic",
"unit": "bytes",
"baseline": 612328280,
"current": 612328280,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -61232828,
"evidenceDeltaUpper": 61232828,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total",
"sizeKind": "nar"
}
},
{
"id": "nix.closure.serialized_nar_size",
"label": "Total serialized NAR size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"total",
"serialized-nar-size",
"nix closure diagnostics"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "diagnostic",
"unit": "bytes",
"baseline": 159040112,
"current": 159040112,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -15904011.200000001,
"evidenceDeltaUpper": 15904011.200000001,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total",
"sizeKind": "nar"
}
},
{
"id": "shell_eval_traced",
"label": "Shell eval with OTEL trace",
"group": "devenv / devenv shell",
"path": [
"devenv",
"devenv shell"
],
"groupPath": [
"devenv",
"devenv shell"
],
"status": "missing_baseline",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "missing_baseline",
"comparisonMode": "historical",
"unit": "seconds",
"baseline": null,
"current": 51.287,
"delta": null,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": null,
"baselineSources": 0,
"currentSamples": 1,
"pairedSamples": null,
"evidenceDeltaLower": null,
"evidenceDeltaUpper": null,
"pairedEvidenceQuantile": null,
"dimensions": {
"probe": "shell_eval_traced",
"probeLabel": "Shell eval with OTEL trace",
"status": 0,
"sampleCount": 2,
"warmupCount": 0,
"measuredSampleCount": 1,
"pairedSampleCount": 1,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "32404763568-1-4e8f359fe4cf486faac9903f4ab12aa49136f1fb",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
}
]
} |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eed40ce069
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for (const [index, step] of job.steps.entries()) { | ||
| if (stepName(step) === prepareCiScriptsStepName) prepareIndex = index | ||
| if (checkoutCleansWorkspaceRoot(step) === true) workspaceCleaningCheckoutIndex = index | ||
| if (stepRun(step)?.includes(preparedCiRetryScriptPath) !== true) continue |
There was a problem hiding this comment.
When a downstream job runs preparation → a workspace-cleaning checkout → validateNixStoreStep, the checkout removes .genie-ci-runtime, but this condition skips validation because it recognizes only run-with-nix-gc-race-retry.sh. validateNixStoreStepFor directly sources resolve-devenv.sh, and the exported prepareJobLocalRustState similarly sources another staged script, so either consumer can subsequently fail with a missing file while the generated workflow is accepted; validate uses of every staged runtime helper, not only the retry wrapper.
Useful? React with 👍 / 👎.
schickling-assistant
deleted the
schickling/2026-08-20-fix-ci-helper-lifetime
branch
August 20, 2026 21:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Generated workflows staged Genie CI helpers in
runner.temp, but persistent self-hosted runners do not preserve that directory across step boundaries. Downstream workflows then fail to sourceresolve-devenv.sheven when preparation succeeds in the preceding step.Goal
Keep generated CI helper scripts available for devenv resolution, retry wrappers, and alternate-checkout jobs.
Decisions
${{ github.workspace }}/.genie-ci-runtime, which survives step boundaries.Verification
devenv tasks run check:quickpassed on the final head.devenv tasks run check:allreached 348 passing Genie tests and then failed in the unrelated Restate scheduling integration test (wake: the id ROTATES per cycle and stays externally resolvable, empty second wake ID).schickling.devworkflow now uses the workspace-scoped runtime; its mandatory localcheck:quickgate passes.Complexity
No new abstraction or dependency; this changes the generated runtime location and strengthens existing invariants.
Concerns
The full-suite Restate failure prevents marking this PR ready despite the focused and quick gates passing.
Friction & bottlenecks
No new bottleneck introduced.
Follow-ups
References
schickling/schickling.devPRs Add cache.nixos.org as fallback Nix substituter in CI #134, Use Tailscale MagicDNS subdomain resolution for worktree-specific hostnames #136, and fix(genie): empty files array and missing error details in JSON output #137.