[skevetter]

Current draft status

Current signed head: 7f5f401e874e7b835119548ab1f21ec3e5c4dfd9. Its affected local format/compile/vet/configured precommit gates and regular PR CI all passed. The initial signed head dea8d011b4b011311a1f622ccc19167bc662c51f also passed regular PR CI, but its first actual native qualification dispatch 40325722590 failed in both CLI/Desktop rows: all six cells passed cold download/current reuse, then stale-fixture creation hit Linux Text file busy while truncating an executable still held by a process. Every cell cleaned up successfully. Both artifact rows passed provenance/version/digest and missing-candidate guards.

The local 36-spec pass below remains evidence from its stated pre-edge working tree and VM, not a current native-head pass. The signed source now fixes the stale fixture by staging/chmod/atomic rename and adds the approved literal-path and actual-sudo cases. No product installer change, retry, kill, alias, or acceptance reduction.

Native edge qualification is held for a corrected immutable core candidate. Exact beta.2 runHostTunnel source still uses fmt.Sprintf("'%s' internal ssh-server --stdio", AgentPath); a real /bin/sh -n -c check of its exact generated hostile-path command exited 2 with an unmatched quote. This is syntax-only parser proof, not a new full SSH result. The coordinator owns the narrow core fix separately; do not rerun the known-broken beta.2 artifact to manufacture another predictable failure. See candidate blocker evidence.

Scope

Refs devsy-org/devsy#1621. Implements handoff T1/T2 qualification and the explicitly approved narrow T3 literal-path/real-sudo test additions; preserves the already-merged core #1627 and provider #152 implementations. No release, merge, ready-state, or issue-closure action.

• Require an explicit candidate core binary/version/tag/SHA for release qualification; retain the separately named v1.24.0 compatibility suite.
• Add a manually dispatched release gate with independent standalone-CLI and actual Desktop-bundled-CLI artifact rows. Verify published asset digests, executable version, source ancestry, exact core/provider identities, and complete passing matrix reports.
• Use published provider release bytes, not a provider snapshot disguised as release qualification.
• Exercise Bash/Zsh/Fish over external/built-in SSH, each with isolated configuration, provider, workspace, and agent directory. Cover remote-download cold install, current reuse, stale replacement, stdin-transfer cold install/reopen, workspace SSH, unchanged login shell, and cleanup.
• Preserve the provider-command byte-stream/stderr/nonzero-exit checks. Restrict bootstrap transport log assertions to the remote-host injection, excluding the separate devcontainer-agent delivery.
• Reuse all six lifecycle cells with literal space/apostrophe/backslash/dollar/Unicode agent paths, quoting every remote path helper through the existing shellquote dependency. Keep the same five lifecycle scenarios per cell.
• Add one actual non-root sudo-required install with a dedicated fixture user, a unique root-owned 0755 agent directory, a denied pre-install write, root-owned installed candidate version/hash, and live workspace SSH. Report this proof separately from the six lifecycle cells and require its outcome and cleanup.
• Restrict the temporary sudoers rule to the owned fixture user and installer /bin/sh -c ... devsy-install ... command pattern; validate it, refuse an existing policy path, and remove only the policy this job created on success/failure.

Tested release artifact pair

Artifact	Identity
Core	v1.25.0-beta.2, b11eea4face29781c6ec977b8802dd31b5aed97c
Core Linux amd64 SHA256	8a5032ddbd390b34741da16504800fb63f3a1492cd6cd7f7ae235a46519cc0e5
Provider	v1.1.47, c53ba2742db162f587aa57fbc87572d19ec11cd7
Provider Linux amd64 SHA256	d8c96cc7a1afc6e19413e310fb2ff4aecd73127024c8dd80f24b095e35970886
Actual Desktop package	beta.2 Devsy_linux_amd64.deb, SHA256 a3e420d462d67c7a7f487431e3bb23a4483b30c1db5662aed3e4b64fa7b9089e
Embedded CLI	Extracted opt/Devsy/resources/bin/devsy; VM --version reported v1.25.0-beta.2; SHA256 equals the standalone CLI

Both fix merges were verified as ancestors of the selected release sources. Standalone and Desktop package hashes matched GitHub's published asset digests. The embedded CLI bytes were checked independently; no second local matrix was run for byte-identical bytes. This is not native Desktop UI acceptance.

Native qualification evidence

Actual isolated local OrbStack Ubuntu 24.04/linux-amd64 VM with SSH and Docker, not Kubernetes or a production/customer host. Go 1.26.10, Docker 29.1.3, approved fixture ghcr.io/devsy-org/test-images/base:ubuntu (observed manifest sha256:4bcb1b466771b1ba1ea110e2a27daea2f6093f9527fb75ee59703ec89b5561cb).

From e2e/, with the explicit pair metadata, candidate binary, no inherited agent transport overrides, GOMAXPROCS=1 GOFLAGS=-p=1:

go test -v -timeout 3600s --ginkgo.v \
  --ginkgo.label-filter='provider-command || release-qualification'

PASS: 36 selected specs, 0 failed. Twelve unrelated compatibility specs were excluded by the label filter; no mandatory qualification row was skipped.

Login shell	External SSH	Built-in SSH	Cleanup
Bash	5/5 scenarios pass	5/5 scenarios pass	both ok
Zsh	5/5 scenarios pass	5/5 scenarios pass	both ok
Fish	5/5 scenarios pass	5/5 scenarios pass	both ok

All six command-boundary specs also passed. Every host agent delivered by remote download or stdin matched the candidate SHA256 and version. Current-agent reopen preserved the inode; stale replacement changed it. Workspace SSH probes passed. Login shells remained unchanged.

Negative checks: missing DEVSY_E2E_BINARY with qualification enabled failed clearly before SSH/workspace startup (expected exit 1). The workflow report predicate rejected the actual failed/incomplete report (exit 1) and accepted the final six-cell passing report (exit 0).

Retained local artifacts: dist/qualification/t2-evidence.md, qualification-final.log, qualification-report-final.json, and missing-binary-guard-final.log. They are evidence from the exercised working tree before the signed integration commit, not fabricated final-commit CI artifacts.

The local runtime pass preceded those lint-only helper extractions. The native GitHub workflow is intended to exercise the final signed branch source; no prior report is relabelled as final-commit CI evidence.

Failed attempts and fixture deviation

Earlier invocations rejected unsupported embedded-Ginkgo parallel flags before specs. One harness precheck incorrectly required the agent directory to be absent after provider init; it now checks the agent binary. Another assertion included the later devcontainer transfer; it now scopes to host bootstrap.

Earlier command / attempt	Observed result and disposition
Early macOS go vet ./...	One dependency/compile attempt timed out; a later attempt exposed unfinished qualification field/type identifiers. Those were corrected; a subsequent vet passed before later harness edits. Neither is claimed as final lint.
Early actionlint .github/workflows/tests.yaml	Failed because job-level env cannot reference runner.temp; directory initialization moved to a step. Later actionlint passed.
First VM focused matrix invocation	Exit 1 after 878.60s with dependency-download output only; no Ginkgo results. Original cause unknown. A separate later OrbStack diagnostic timed out waiting for services; not asserted as the cause of the first exit.
orb -m devsy-1621-qual /bin/true	Passed after the service-timeout observation.
Matrix with --ginkgo.procs=1	Failed before specs: flag not defined by embedded Ginkgo.
Matrix with -test.parallel=1	Failed before specs: rejected by embedded Ginkgo. Subsequent runs used default single Ginkgo process plus GOMAXPROCS=1 GOFLAGS=-p=1.
First executable matrix	Six provider-command cases passed; six workspace setup failures because provider init creates the agent directory. Binary-absence precheck corrected.
Corrected matrix with VM systemd cgroups	Failed at container startup; six workspace cells unqualified, cleanup ok.
First cgroupfs matrix	Workspace startup succeeded; failed overly broad transport assertion that included the separate devcontainer transfer. Scope corrected to remote-host bootstrap.
Final cgroupfs matrix	Exit 0; 36 selected specs passed, all six cells complete, cleanup ok.
Missing-binary guards	Expected exit 1 before SSH/workspace startup; final focused guard selected one failing BeforeAll and reported the missing explicit binary.
Combined Desktop version/checksum command	Version output observed before timeout; not a full-command pass. Separate host package/binary SHA256 commands passed and matched release digest / standalone bytes.
VM restoration	Initial unprivileged config removal failed with permission denied; authorized sudo restoration and owned-daemon restart passed, restoring systemd cgroup v2 with no containers.
First configured final precommit	All non-lint hooks and formatter passed; three length/complexity findings in the new helpers. Addressed by behavior-preserving extraction of existing environment checks, cleanup guard, and Ginkgo callbacks; no suppressions or weaker lint configuration.
First signed-head native workflow	Both actual CLI/Desktop source rows passed provenance and missing-binary guards. Each cell passed cold download/current reuse, then stale fixture truncation failed with Linux Text file busy; cleanup all ok. Correcting the fixture to atomic staging/rename, not weakening assertions. CLI result: 18 passed, 6 failed, 24 skipped.

A real full run failed at runc/systemd cgroup startup in the disposable nested VM. Under explicit coordinator authorization, T2 preserved the VM's Docker settings, changed only that VM's daemon to cgroupfs, proved cached-image container startup, and ran the passing matrix. After evidence capture, the original systemd driver was restored, the owned daemon was verified active, and no containers remained. No global OrbStack/other VM configuration was changed.

Acceptance and release limits

Verified on the explicitly identified local pre-edge working tree: T1 pair identity/ancestry and fail-closed guard; T2 six plain-path workspace cells, both cold delivery modes, current/stale agent handling, SSH, shell preservation, and cleanup. This is not the new literal-path/actual-sudo acceptance.

Supplemental candidate-source core evidence from the coordinator: pkg/inject and pkg/agent/inject.go were compared byte-for-byte with beta.2 source b11eea4face29781c6ec977b8802dd31b5aed97c; no differences. GOMAXPROCS=1 GOFLAGS=-p=1 mise exec -- go test ./pkg/inject -run TestBootstrapSuite -count=1 -timeout=180s -v passed (package 2.345s). It covered real /bin/sh hostile literal paths and live binary streams, literal roundtrip, failed-download fallback to stdin, short-transfer/chmod replacement preservation, warm-agent reuse, NUL rejection, and Dockerless shell selection. The sudo/su mode live-stream and failure-status cases used explicit function doubles, not a real privilege-transition fixture.

Not run: native Desktop UI/install/update, reporter/customer-host reproduction, Windows/macOS full-workspace artifact smokes, Kubernetes, the new literal AGENT_PATH cells, or the actual non-root sudo fixture. First signed-head native jobs failed as recorded above; current-head native edge qualification is held for the corrected candidate artifact. This PR does not claim complete T3 acceptance, stable delivery, or customer confirmation.

The approved literal-path and real-sudo source is new and requires its own final signed-head native proof: 37 selected specs per artifact source (six command-boundary specs, thirty lifecycle specs, one actual sudo scenario). The updated report predicate deliberately rejects the older plain-path 36-spec passing JSON (observed expected exit 1), so it cannot stand in for the new edge evidence.

Still required before customer resolution: produce immutable candidate CLI/Desktop artifacts containing the corrected literal command boundaries, complete the final native edge qualification and reviews, then publish/qualify a stable core containing both e22be00149ace87d512de0ec953541e3f7642c69 and those additional boundary fixes. Pair it with the qualified provider release, verify downloadable artifacts and the installed external-provider version, complete reporter-equivalent fresh/reopen and supported-client smokes, attach traces/matrix evidence, and perform maintainer/customer communication and closure under project policy. Keep this PR draft.

Signed integration and final local checks

Current head: 7f5f401e874e7b835119548ab1f21ec3e5c4dfd9; initial head: dea8d011b4b011311a1f622ccc19167bc662c51f. Both commits are signed by Samuel K skevetter@pm.me; local git verify-commit reported good signatures. Pushed over SSH.

With the repository-pinned mise tools and GOMAXPROCS=1 GOFLAGS=-p=1, one command at a time:

Command	Final result
mise exec -- golangci-lint fmt ./e2e/...	PASS
mise exec -- go test ./e2e -run '^$'	PASS, final E2E compile
mise exec -- go test ./pkg/ssh/... ./cmd/...	PASS at the initial head; unchanged transport/command suites were not duplicated after test-harness-only edits
mise exec -- go vet ./e2e/...	PASS after literal/sudo/atomic-stale source changes
mise exec -- prek run --files <all eight qualification files>	PASS: lint, formatting, actionlint, secrets, whitespace, EOF, YAML, large-file and conflict checks; shell hooks had no matching files

The final changed-source serial chain passed in 52.10s; the global slot was released directly to the next owner. The pinned v2.15 hook explicitly has pass_filenames: false for lint/formatter: --files selects hooks, not their internal module scope. A validation checkpoint and exact mutation list contained only the four intended modified files plus the new sudo source; no unrelated file was staged. Current-head regular CI passed: lint, precommit, compatibility integration, Windows stdin, workflow approval, CLA and signed-commit checks. Manual release qualification was correctly skipped for the PR event, not counted as proof. CodeRabbit's success status explicitly says the draft review was skipped; it is not a completed review. Final Sol/Greptile/CodeRabbit gates remain held until corrected-artifact native runtime evidence exists.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[skevetter]

Separate T3 finding, not the first native stale-fixture ETXTBSY failure: the tested core candidate b11eea4face29781c6ec977b8802dd31b5aed97c still formats runHostTunnel as:

command := fmt.Sprintf("'%s' internal ssh-server --stdio", c.client.AgentPath())

Source: https://github.com/devsy-org/devsy/blob/b11eea4face29781c6ec977b8802dd31b5aed97c/pkg/tunnel/container.go#L134

A real local /bin/sh -n -c check of the exact generated command with the approved hostile path exited 2 with an unmatched-quote syntax error. This is syntax-only parser proof, not a new full SSH/native matrix result.

The newly approved literal/sudo test-source changes are not yet signed/pushed; no product code or candidate artifact was modified by this provider worker. The coordinator owns the narrow core-boundary fix separately. Preserve the literal requirements and hold additional native qualification until a corrected immutable candidate artifact and explicitly updated pair provenance are available. Do not weaken the test, sanitize the path, add an alias, or add another launcher. The known broken beta.2 artifact must not be predictably rerun and presented as new proof.

[skevetter]

Upstream dependency is now tracked by core DRAFT PR devsy-org/devsy#1725 at signed source 1c379a765070858a6912eeecfcbfec87d484c489.

The coordinator reports a complete command-boundary cutover, including the initial workspace-up SSH prefix and periodic refresh, with actual hostile-executable-path CLI startup, SSH, same-container reopen, deletion and empty inventory passing. Those binaries were explicitly engineering v0.0 smoke artifacts, not a published candidate and not inputs to this provider release gate. Core final CI/review/merge gates remain upstream-owned.

Provider source remains unchanged at signed 7f5f401e874e7b835119548ab1f21ec3e5c4dfd9; regular PR CI passed. The 37-case-per-source native qualification is still held until a real corrected published CLI/Desktop candidate is available with explicit tag, source SHA, binary version, digests and actual bundled-CLI identity. No beta.2 rerun, CI-artifact substitution, acceptance reduction, release, merge, ready-state change or issue closure is implied.