From 6cb2b83c2196365bec6aaf8924a08e8c82e7ba2c Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sat, 17 Oct 2026 11:28:20 +0900 Subject: [PATCH 01/20] feat(credentials): forward GitHub CLI authentication MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Provide host active-account credentials for GitHub CLI through the existing credentials relay. Enable SSH_INJECT_GITHUB_CREDENTIALS by default with an independent opt-out and workspace policy enforcement across services paths. Run the real gh against a private invocation configuration. Preserve local authentication and environment precedence, share the invocation with nested gh processes, and merge only sanitized user changes onto the latest original settings. Keep symlinks and unrelated concurrent edits; retain failed writes for later recovery even when the relay is unavailable. Forwarded tokens are never written into the original settings. Explicit container keyring logins remain local. File re-saves of identical forwarded values can remain relayed. Keep gh output and exit status unchanged, without command classification, host guessing or command retries. Record bounded credential-free local boundary and writeback decisions with an opt-out and no remote export. Validation: real gh 2.100.0 through actual tunnel/HTTP relay in an isolated Linux container, mock GitHub/GHE APIs, dummy tokens and Linux keyring passed. Covered auth status/login/logout/switch/refresh, local precedence, aliases, extensions, parallel/dotfiles edits, writeback failure, signals, orphan-child ownership, recovery without relay, non-root HOME and diagnostic privacy. Related race tests and changed-scope lint passed; Windows package build passed. The broader race suite still fails the pre-existing TestSnapshotWorkspacePolicyDoesNotFilterOtherMount; original HEAD overlay reproduces it. Live provider/workspace/IDE/reconnect behavior, host macOS keychain, other gh versions and real GitHub/GHE services were not tested. ## Transcript - U1: > # 実装依頼 > > ## 依頼 > `/Users/a13400/src/devsy-org/devsy` の現在のcheckoutで、既存のcredentials relayにGitHub CLI（`gh`）の認証転送を追加してください。 > > ## 目的 > - コンテナ内の通常の `gh` コマンドや拡張から、ホスト側のGitHub認証を利用できるようにする。 > - コンテナ内での独自ログイン、設定変更、dotfiles管理と共存する。 > > ## 成功条件 > - `github.com` とGitHub Enterpriseの複数ホストを扱え、対象ホストの選択は `gh` 本体が行う。 > - コンテナ側に既存認証がなければ、ホスト側の対象ホストのactive accountを利用する。 > - コンテナ側に明示された認証用環境変数やローカル認証があれば優先する。期限切れ・権限不足でも、ホスト認証へ自動切替しない。 > - `gh auth status` は実際の `gh` が転送込みの認証状態を表示する。独自の表示や結果合成は行わない。 > - コンテナ内の `gh auth login/logout/switch/refresh`、設定変更、alias、および拡張内部の操作を、コマンド分類なしで扱う。 > - ローカル認証をlogoutした場合、次回実行では転送認証を再び利用できる。ホスト側の認証は変更しない。 > - 利用者による設定・認証変更が保存され、Devsyが提供した転送情報は元設定へ混入しない。 > - dotfilesの再適用や並行する `gh` 操作について、無関係な変更を失わない。 > - 転送できない場合は、転送認証なしで通常の `gh` を実行する。 > - 以上を実際の `gh` とダミー認証を用いて検証する。静的確認・ローカル検証と、未検証の実環境境界を区別して報告する。 > > ## 決定事項 > - 既存credentials relayに追加する。 > - 理由: ホストの認証機能をコンテナ内ツールへ提供する既存の責務に合う。 > - 主な棄却案: 独立した別サービス。既存の接続・ライフサイクル管理を重複させるため。 > - 専用context設定を追加し、既定で有効にする。 > - 理由: Git・Docker・SSH署名の既存設定方針と揃え、個別に無効化できるようにする。 > - 主な棄却案: 既定無効、既存Git認証設定への連動。 > - ホスト側の認証は各ホストのactive accountに追従する。 > - 理由: アカウント選択を二重管理しないため。 > - 主な棄却案: Devsy設定によるアカウント固定。 > - 通常の `gh` をwrapper経由で透過的に利用する。 > - 理由: 既存スクリプトやツールの呼び出し変更を不要にするため。 > - 主な棄却案: Devsy専用コマンドによる明示実行。 > - wrapperにはホスト判定を持たせない。 > - 理由: URL、Git remote、環境変数、alias、拡張による `gh` の判断を再実装しないため。 > - 主な棄却案: 引数解析、認証エラーメッセージからのホスト推定と再実行。 > - 環境変数だけの転送ではなく、実行ごとに一時設定を用意する。 > - 理由: 複数ホストの認証を渡し、選択を `gh` 本体へ任せるため。 > - 主な棄却案: `GH_TOKEN`／`GH_ENTERPRISE_TOKEN` のみの注入。複数ホストの異なる認証を表現できないため。 > - 元設定を読み込み、ローカル認証を優先しつつ転送情報を加えた一時設定を使う。 > - 理由: Devsyによる元ファイルの直接編集とdotfilesとの競合を避けるため。 > - 主な棄却案: 元の `hosts.yml` への転送認証の追記。 > - コマンドを分類せず、一時設定上の利用者変更だけを書き戻す。 > - 理由: 拡張、alias、将来のコマンドにも対応するため。 > - 主な棄却案: 認証・設定変更コマンドだけ元設定で実行する方式。拡張内部の操作を捕捉できないため。 > - 拡張と子プロセスは同じ一時設定を共有し、最初の `gh` の終了時に変更を反映する。 > - 理由: 実行内の整合性を保ち、随時同期の保守負担を避けるため。 > - 主な棄却案: ファイル監視による随時反映。 > - 失敗終了時も、終了までに行われた利用者の設定変更は反映する。 > - 理由: コマンドの最終的な成否と、それ以前の設定変更を区別するため。 > - 主な棄却案: 成功終了時だけ反映。 > - 書き戻しでは最新の元設定に変更差分を適用し、同じ項目の競合は今回の変更を優先する。 > - 理由: 無関係な並行変更を維持しつつ、今回の利用者操作を保存するため。 > - 主な棄却案: 一時設定全体の上書き、競合時に今回の変更を破棄する方式。 > - 転送不能時は通常の `gh` へフォールバックし、workspace接続時には警告しない。 > - 理由: 既定有効でも、ホストの `gh` 未導入・未ログインやrelay障害が通常利用を妨げないため。 > - 主な棄却案: 転送不能で強制終了、接続時の警告。 > > ## 制約 > - wrapperによるホスト判定、コマンド分類、失敗したコマンド全体の自動再実行を行わない。 > - 実際の `gh` のstdout・stderr・終了コードを加工しない。 > - relayの独自表示や `gh auth status` の結果合成を追加しない。 > - Devsy由来の転送認証を元の設定ファイルへ保存しない。利用者自身のログイン・設定変更による保存は維持する。 > - 元設定の場所は `GH_CONFIG_DIR`、XDG設定、対象ユーザーのHOMEを含めて扱う。 > - symlinkやdotfiles管理を壊さず、ホスト側の認証ストアも変更しない。 > - トークンを診断ログや検証成果物に含めない。 > > ## 対象外 > - ホスト側での `gh` 自動インストール・自動ログイン。 > - ホスト側アカウントの切替・ログアウト・権限変更。 > - `gh` 本体のforkや独自ビルドの配布。 > - D-Bus Secret Serviceの新規実装。 > - 接続時の独自認証診断画面。 > - push、PR作成、release、cloud resourceの変更。 > > ## 未検証の前提 > - 対応対象の `gh` バージョンにおける設定形式、keyring連携、拡張への設定・環境変数の継承。 > - 既存のDevsy各起動経路におけるcontext設定、workspace側ポリシー、services起動・再取得との統合箇所。 > - 一時設定への書き込みから、転送情報と利用者自身の認証保存を区別できる具体的な方法。 > - 同一値の再保存、keyringへの保存、設定削除、入れ子実行、並行更新、異常終了時の保持・cleanupを含めた方式の成立性。 > > ## 実装時の指示 > - 主な既存実装: > - `cmd/internal/agentcontainer/credentials_server.go` > - `pkg/credentials/server.go` > - `pkg/agent/tunnelserver/tunnelserver.go` > - `pkg/agent/tunnel/tunnel.proto` > - `pkg/tunnel/services.go` > - `pkg/client/clientimplementation/services.go` > - `pkg/config/context.go` > - 最初に、一時設定と書き戻し方式が制約を満たすことを、小さな検証で確かめる。 > - 特に、転送したアカウントと同じアカウントへの `gh auth login`、keyring保存、alias・拡張内部の設定変更を確認する。単純な値の差分だけで利用者の保存意図を扱えると仮定しない。 > - 成立しない条件が見つかった場合は、コマンド分類やホスト判定を復活させず、具体的な反例と必要な設計変更を示す。 > - 検証は隔離した設定・ダミー認証を使い、実際の個人認証情報や通常利用の設定を変更しない。 - A: Applied the investigation, development and observability workflows. Inspected the requested checkout and preserved two unrelated existing changes. Used isolated Linux containers, actual gh 2.100.0 and dummy APIs/credentials to probe temporary settings and same-account login. Omitting per-account tokens broke real auth switch; comments did not reliably distinguish login from configuration rewrites. Initially treated same-value persistence ambiguity as a blocker. - U2: > 同一アカウント・同一トークンへのログインと、認証を変更しない設定の再保存が、同じファイル内容になる ← なにが問題なの？ - A: The user questioned the practical significance of identical final file contents; this steering arrived during the investigation. - U3: > そのケースは同じトークンなんだから転送のまま残っても挙動かわらないでしょ。実用上なにが問題なの？ - A: The user clarified that retaining identical authentication as forwarded has the same immediate behavior; no implementation change was made before the next steering message. - U4: > そんなエッジケースの利用するなら利用者はgh転送をしない想定だ - A: Accepted same-value re-save ambiguity as non-blocking. Users needing independence from subsequent host authentication changes can disable forwarding. Resumed implementation without introducing host guessing or command classification. - U5: > 続けて - A: Continued inspection of relay wiring and configuration while protecting existing unrelated edits. Began the new ghcredentials files in the originally requested checkout. - U6: > 既存の変更ってなに？ このworktree checkoutに残ってるの？ git worktreeを追加するときってcheckoutの編集もそのまま引き継がれるの？ - A: Inspected the originally requested checkout and the chat worktree read-only to locate the existing edits. Git worktree creation does not automatically copy another checkout's index or uncommitted edits. The user then explicitly selected this worktree. - U7: > 作業はこちらのworktreeで進める - A: Worked in /Users/a13400/orca/workspaces/devsy/gh-relay on atty303/gh-relay, initially clean. Moved only the two newly owned ghcredentials files from the original checkout, leaving its unrelated edits intact. Added the default-enabled context option, restricted relay RPC and HTTP endpoint, all services launch wiring, private transparent wrapper, per-invocation snapshots, local-auth precedence, three-way writeback, nested gh sharing, safe local diagnostics and recovery. Used existing dependencies and generated protocol bindings through the repository task. Added unit and actual gh integration coverage using fictional credentials, isolated settings, local mock GitHub APIs and guest Linux Secret Service. Fixed independent review findings covering recovery without relay, ownership surviving wrapper-only SIGKILL, dangling symlink preservation, failure classification, successful merge decision recording and diagnostic opt-out overhead. Clarified that unchanged file-token re-saves may remain forwarded while explicit guest keyring logins are retained; verified both same-token cases. The fresh reviewer accepted the user scope clarification and final delta with no actionable findings. Final real gh integration, related race tests, changed-scope lint including config and main, and Windows package cross-build succeeded. The broader race suite has one baseline failure, TestSnapshotWorkspacePolicyDoesNotFilterOtherMount, also reproduced using original HEAD files through a Go overlay. Live workspace/IDE/reconnect/provider behavior, host macOS keychain and actual GitHub/GHE servers remain unverified. Cleaned owned test resources and earlier empty cache locks; final snapshot and staging match the reviewed candidate. No push, PR, release or cloud mutation was performed. Co-authored-by: Codex --- Taskfile.yml | 4 + .../agentcontainer/credentials_server.go | 13 + cmd/workspace/ssh.go | 5 + main.go | 12 +- pkg/agent/tunnel/tunnel.pb.go | 65 +- pkg/agent/tunnel/tunnel.proto | 1 + pkg/agent/tunnel/tunnel_grpc.pb.go | 38 ++ .../tunnelserver/github_credentials_test.go | 21 + pkg/agent/tunnelserver/options.go | 4 + pkg/agent/tunnelserver/tunnelserver.go | 18 + pkg/client/clientimplementation/services.go | 6 + pkg/config/context.go | 7 + pkg/credentials/server.go | 38 ++ pkg/ghcredentials/config.go | 179 ++++++ pkg/ghcredentials/diagnostics.go | 191 ++++++ pkg/ghcredentials/host.go | 99 ++++ pkg/ghcredentials/install.go | 50 ++ pkg/ghcredentials/owner_linux.go | 34 ++ pkg/ghcredentials/owner_unix.go | 39 ++ pkg/ghcredentials/process_unix.go | 50 ++ pkg/ghcredentials/process_windows.go | 38 ++ pkg/ghcredentials/result.go | 89 +++ pkg/ghcredentials/session.go | 558 ++++++++++++++++++ pkg/ghcredentials/session_test.go | 164 +++++ pkg/ghcredentials/testdata/Dockerfile | 3 + pkg/ghcredentials/testdata/fixture.py | 294 +++++++++ pkg/ghcredentials/wrapper.go | 236 ++++++++ pkg/ghcredentials/wrapper_integration_test.go | 308 ++++++++++ pkg/ghcredentials/writeback.go | 125 ++++ pkg/tunnel/browser.go | 3 + pkg/tunnel/services.go | 5 + .../developing-in-workspaces/credentials.mdx | 18 +- 32 files changed, 2682 insertions(+), 33 deletions(-) create mode 100644 pkg/agent/tunnelserver/github_credentials_test.go create mode 100644 pkg/ghcredentials/config.go create mode 100644 pkg/ghcredentials/diagnostics.go create mode 100644 pkg/ghcredentials/host.go create mode 100644 pkg/ghcredentials/install.go create mode 100644 pkg/ghcredentials/owner_linux.go create mode 100644 pkg/ghcredentials/owner_unix.go create mode 100644 pkg/ghcredentials/process_unix.go create mode 100644 pkg/ghcredentials/process_windows.go create mode 100644 pkg/ghcredentials/result.go create mode 100644 pkg/ghcredentials/session.go create mode 100644 pkg/ghcredentials/session_test.go create mode 100644 pkg/ghcredentials/testdata/Dockerfile create mode 100644 pkg/ghcredentials/testdata/fixture.py create mode 100644 pkg/ghcredentials/wrapper.go create mode 100644 pkg/ghcredentials/wrapper_integration_test.go create mode 100644 pkg/ghcredentials/writeback.go diff --git a/Taskfile.yml b/Taskfile.yml index ec689e9faf..7f6b8f0cee 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -106,6 +106,10 @@ tasks: export PATH="$(go env GOPATH)/bin:$PATH" protoc -I . tunnel.proto --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative + cli:test:gh-relay: + desc: verify GitHub CLI relay with isolated real gh and Linux keyring (requires Docker) + cmd: go test -tags=ghrelay ./pkg/ghcredentials -run "^TestRealGitHubCLI$" -v -count=1 -timeout=10m + cli:test: desc: run devsy unit tests cmd: | diff --git a/cmd/internal/agentcontainer/credentials_server.go b/cmd/internal/agentcontainer/credentials_server.go index 8184c21519..e21da6f62e 100644 --- a/cmd/internal/agentcontainer/credentials_server.go +++ b/cmd/internal/agentcontainer/credentials_server.go @@ -20,6 +20,7 @@ import ( "github.com/devsy-org/devsy/pkg/dockercredentials" cliflags "github.com/devsy-org/devsy/pkg/flags" "github.com/devsy-org/devsy/pkg/flags/names" + "github.com/devsy-org/devsy/pkg/ghcredentials" "github.com/devsy-org/devsy/pkg/gitcredentials" "github.com/devsy-org/devsy/pkg/gitsshsigning" "github.com/devsy-org/devsy/pkg/log" @@ -34,6 +35,7 @@ type CredentialsServerCmd struct { User string ConfigureGitHelper bool + ConfigureGitHubHelper bool ConfigureDockerHelper bool ForwardPorts bool @@ -60,6 +62,12 @@ func NewCredentialsServerCmd(flags *flags.GlobalFlags) *cobra.Command { } cliflags.Add( credentialsServerCmd, + cliflags.Bool( + &cmd.ConfigureGitHubHelper, + "configure-github-helper", + false, + "If true will configure GitHub CLI authentication forwarding", + ), cliflags.Bool( &cmd.ConfigureGitHelper, names.ConfigureGitHelper, @@ -111,6 +119,11 @@ func (cmd *CredentialsServerCmd) Run(ctx context.Context, port int) error { } cmd.maybeForwardPorts(runCtx, tunnelClient) + if cmd.ConfigureGitHubHelper { + if err := ghcredentials.Install(port); err != nil { + log.Debug("GitHub CLI wrapper installation unavailable") + } + } if err := cmd.configureDockerHelper(port); err != nil { return err diff --git a/cmd/workspace/ssh.go b/cmd/workspace/ssh.go index 3b8f9d2245..e1a4a5c0b0 100644 --- a/cmd/workspace/ssh.go +++ b/cmd/workspace/ssh.go @@ -574,6 +574,9 @@ func (cmd *SSHCmd) startTunnelServices( ConfigureDockerCredentials: devsyConfig.ContextOption( config.ContextOptionSSHInjectDockerCredentials, ) == config.BoolTrue, + ConfigureGitHubCredentials: devsyConfig.ContextOption( + config.ContextOptionGitHubCLIForwarding, + ) == config.BoolTrue, ConfigureGitCredentials: devsyConfig.ContextOption( config.ContextOptionSSHInjectGitCredentials, ) == config.BoolTrue, @@ -656,6 +659,7 @@ func resolveMergedWorkspaceFolder( type startServicesOptions struct { ConfigureDockerCredentials bool ConfigureGitCredentials bool + ConfigureGitHubCredentials bool ConfigureGitSSHSignatureHelper bool GitSSHSigningKey string } @@ -680,6 +684,7 @@ func (cmd *SSHCmd) startServices( Workspace: workspace, ConfigureDockerCredentials: opts.ConfigureDockerCredentials, ConfigureGitCredentials: opts.ConfigureGitCredentials, + ConfigureGitHubCredentials: opts.ConfigureGitHubCredentials, ConfigureGitSSHSignatureHelper: opts.ConfigureGitSSHSignatureHelper, GitSSHSigningKey: opts.GitSSHSigningKey, }, diff --git a/main.go b/main.go index c562ff0227..574d4d89f0 100644 --- a/main.go +++ b/main.go @@ -1,7 +1,17 @@ package main -import "github.com/devsy-org/devsy/cmd" +import ( + "os" + + "github.com/devsy-org/devsy/cmd" + "github.com/devsy-org/devsy/pkg/ghcredentials" +) func main() { + // gh must receive the caller's exact environment and streams, without + // Devsy's envfile loading, argument parsing, logging, or telemetry. + if len(os.Args) >= 4 && os.Args[1] == "internal" && os.Args[2] == "gh" { + os.Exit(ghcredentials.Run(os.Args[3:])) + } cmd.Execute() } diff --git a/pkg/agent/tunnel/tunnel.pb.go b/pkg/agent/tunnel/tunnel.pb.go index 90ec40140e..e9afcb320e 100644 --- a/pkg/agent/tunnel/tunnel.pb.go +++ b/pkg/agent/tunnel/tunnel.pb.go @@ -772,7 +772,7 @@ const file_tunnel_proto_rawDesc = "" + "\x04INFO\x10\x01\x12\b\n" + "\x04DONE\x10\x02\x12\v\n" + "\aWARNING\x10\x03\x12\t\n" + - "\x05ERROR\x10\x042\xad\a\n" + + "\x05ERROR\x10\x042\xe4\a\n" + "\x06Tunnel\x12&\n" + "\x04Ping\x12\r.tunnel.Empty\x1a\r.tunnel.Empty\"\x00\x12*\n" + "\x03Log\x12\x12.tunnel.LogMessage\x1a\r.tunnel.Empty\"\x00\x12/\n" + @@ -781,6 +781,7 @@ const file_tunnel_proto_rawDesc = "" + "SendResult\x12\x0f.tunnel.Message\x1a\r.tunnel.Empty\"\x00\x127\n" + "\x11DockerCredentials\x12\x0f.tunnel.Message\x1a\x0f.tunnel.Message\"\x00\x124\n" + "\x0eGitCredentials\x12\x0f.tunnel.Message\x1a\x0f.tunnel.Message\"\x00\x125\n" + + "\x11GitHubCredentials\x12\r.tunnel.Empty\x1a\x0f.tunnel.Message\"\x00\x125\n" + "\x0fGitSSHSignature\x12\x0f.tunnel.Message\x1a\x0f.tunnel.Message\"\x00\x12+\n" + "\aGitUser\x12\r.tunnel.Empty\x1a\x0f.tunnel.Message\"\x00\x121\n" + "\vDevsyConfig\x12\x0f.tunnel.Message\x1a\x0f.tunnel.Message\"\x00\x123\n" + @@ -836,36 +837,38 @@ var file_tunnel_proto_depIdxs = []int32{ 6, // 7: tunnel.Tunnel.SendResult:input_type -> tunnel.Message 6, // 8: tunnel.Tunnel.DockerCredentials:input_type -> tunnel.Message 6, // 9: tunnel.Tunnel.GitCredentials:input_type -> tunnel.Message - 6, // 10: tunnel.Tunnel.GitSSHSignature:input_type -> tunnel.Message - 13, // 11: tunnel.Tunnel.GitUser:input_type -> tunnel.Empty - 6, // 12: tunnel.Tunnel.DevsyConfig:input_type -> tunnel.Message - 6, // 13: tunnel.Tunnel.GPGPublicKeys:input_type -> tunnel.Message - 6, // 14: tunnel.Tunnel.KubeConfig:input_type -> tunnel.Message - 13, // 15: tunnel.Tunnel.Secrets:input_type -> tunnel.Empty - 4, // 16: tunnel.Tunnel.ForwardPort:input_type -> tunnel.ForwardPortRequest - 2, // 17: tunnel.Tunnel.StopForwardPort:input_type -> tunnel.StopForwardPortRequest - 13, // 18: tunnel.Tunnel.StreamWorkspace:input_type -> tunnel.Empty - 1, // 19: tunnel.Tunnel.StreamMount:input_type -> tunnel.StreamMountRequest - 13, // 20: tunnel.Tunnel.StreamSnapshotVolumes:input_type -> tunnel.Empty - 13, // 21: tunnel.Tunnel.Ping:output_type -> tunnel.Empty - 13, // 22: tunnel.Tunnel.Log:output_type -> tunnel.Empty - 13, // 23: tunnel.Tunnel.Status:output_type -> tunnel.Empty - 13, // 24: tunnel.Tunnel.SendResult:output_type -> tunnel.Empty - 6, // 25: tunnel.Tunnel.DockerCredentials:output_type -> tunnel.Message - 6, // 26: tunnel.Tunnel.GitCredentials:output_type -> tunnel.Message - 6, // 27: tunnel.Tunnel.GitSSHSignature:output_type -> tunnel.Message - 6, // 28: tunnel.Tunnel.GitUser:output_type -> tunnel.Message - 6, // 29: tunnel.Tunnel.DevsyConfig:output_type -> tunnel.Message - 6, // 30: tunnel.Tunnel.GPGPublicKeys:output_type -> tunnel.Message - 6, // 31: tunnel.Tunnel.KubeConfig:output_type -> tunnel.Message - 8, // 32: tunnel.Tunnel.Secrets:output_type -> tunnel.SecretsResponse - 5, // 33: tunnel.Tunnel.ForwardPort:output_type -> tunnel.ForwardPortResponse - 3, // 34: tunnel.Tunnel.StopForwardPort:output_type -> tunnel.StopForwardPortResponse - 9, // 35: tunnel.Tunnel.StreamWorkspace:output_type -> tunnel.Chunk - 9, // 36: tunnel.Tunnel.StreamMount:output_type -> tunnel.Chunk - 9, // 37: tunnel.Tunnel.StreamSnapshotVolumes:output_type -> tunnel.Chunk - 21, // [21:38] is the sub-list for method output_type - 4, // [4:21] is the sub-list for method input_type + 13, // 10: tunnel.Tunnel.GitHubCredentials:input_type -> tunnel.Empty + 6, // 11: tunnel.Tunnel.GitSSHSignature:input_type -> tunnel.Message + 13, // 12: tunnel.Tunnel.GitUser:input_type -> tunnel.Empty + 6, // 13: tunnel.Tunnel.DevsyConfig:input_type -> tunnel.Message + 6, // 14: tunnel.Tunnel.GPGPublicKeys:input_type -> tunnel.Message + 6, // 15: tunnel.Tunnel.KubeConfig:input_type -> tunnel.Message + 13, // 16: tunnel.Tunnel.Secrets:input_type -> tunnel.Empty + 4, // 17: tunnel.Tunnel.ForwardPort:input_type -> tunnel.ForwardPortRequest + 2, // 18: tunnel.Tunnel.StopForwardPort:input_type -> tunnel.StopForwardPortRequest + 13, // 19: tunnel.Tunnel.StreamWorkspace:input_type -> tunnel.Empty + 1, // 20: tunnel.Tunnel.StreamMount:input_type -> tunnel.StreamMountRequest + 13, // 21: tunnel.Tunnel.StreamSnapshotVolumes:input_type -> tunnel.Empty + 13, // 22: tunnel.Tunnel.Ping:output_type -> tunnel.Empty + 13, // 23: tunnel.Tunnel.Log:output_type -> tunnel.Empty + 13, // 24: tunnel.Tunnel.Status:output_type -> tunnel.Empty + 13, // 25: tunnel.Tunnel.SendResult:output_type -> tunnel.Empty + 6, // 26: tunnel.Tunnel.DockerCredentials:output_type -> tunnel.Message + 6, // 27: tunnel.Tunnel.GitCredentials:output_type -> tunnel.Message + 6, // 28: tunnel.Tunnel.GitHubCredentials:output_type -> tunnel.Message + 6, // 29: tunnel.Tunnel.GitSSHSignature:output_type -> tunnel.Message + 6, // 30: tunnel.Tunnel.GitUser:output_type -> tunnel.Message + 6, // 31: tunnel.Tunnel.DevsyConfig:output_type -> tunnel.Message + 6, // 32: tunnel.Tunnel.GPGPublicKeys:output_type -> tunnel.Message + 6, // 33: tunnel.Tunnel.KubeConfig:output_type -> tunnel.Message + 8, // 34: tunnel.Tunnel.Secrets:output_type -> tunnel.SecretsResponse + 5, // 35: tunnel.Tunnel.ForwardPort:output_type -> tunnel.ForwardPortResponse + 3, // 36: tunnel.Tunnel.StopForwardPort:output_type -> tunnel.StopForwardPortResponse + 9, // 37: tunnel.Tunnel.StreamWorkspace:output_type -> tunnel.Chunk + 9, // 38: tunnel.Tunnel.StreamMount:output_type -> tunnel.Chunk + 9, // 39: tunnel.Tunnel.StreamSnapshotVolumes:output_type -> tunnel.Chunk + 22, // [22:40] is the sub-list for method output_type + 4, // [4:22] is the sub-list for method input_type 4, // [4:4] is the sub-list for extension type_name 4, // [4:4] is the sub-list for extension extendee 0, // [0:4] is the sub-list for field type_name diff --git a/pkg/agent/tunnel/tunnel.proto b/pkg/agent/tunnel/tunnel.proto index 7e710ff2a1..36c6441768 100644 --- a/pkg/agent/tunnel/tunnel.proto +++ b/pkg/agent/tunnel/tunnel.proto @@ -13,6 +13,7 @@ service Tunnel { rpc DockerCredentials(Message) returns (Message) {} rpc GitCredentials(Message) returns (Message) {} + rpc GitHubCredentials(Empty) returns (Message) {} rpc GitSSHSignature(Message) returns (Message) {} rpc GitUser(Empty) returns (Message) {} rpc DevsyConfig(Message) returns (Message) {} diff --git a/pkg/agent/tunnel/tunnel_grpc.pb.go b/pkg/agent/tunnel/tunnel_grpc.pb.go index 5d735fe276..96f4cfaf6b 100644 --- a/pkg/agent/tunnel/tunnel_grpc.pb.go +++ b/pkg/agent/tunnel/tunnel_grpc.pb.go @@ -27,6 +27,7 @@ const ( Tunnel_SendResult_FullMethodName = "/tunnel.Tunnel/SendResult" Tunnel_DockerCredentials_FullMethodName = "/tunnel.Tunnel/DockerCredentials" Tunnel_GitCredentials_FullMethodName = "/tunnel.Tunnel/GitCredentials" + Tunnel_GitHubCredentials_FullMethodName = "/tunnel.Tunnel/GitHubCredentials" Tunnel_GitSSHSignature_FullMethodName = "/tunnel.Tunnel/GitSSHSignature" Tunnel_GitUser_FullMethodName = "/tunnel.Tunnel/GitUser" Tunnel_DevsyConfig_FullMethodName = "/tunnel.Tunnel/DevsyConfig" @@ -50,6 +51,7 @@ type TunnelClient interface { SendResult(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Empty, error) DockerCredentials(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) GitCredentials(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) + GitHubCredentials(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*Message, error) GitSSHSignature(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) GitUser(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*Message, error) DevsyConfig(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) @@ -131,6 +133,16 @@ func (c *tunnelClient) GitCredentials(ctx context.Context, in *Message, opts ... return out, nil } +func (c *tunnelClient) GitHubCredentials(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*Message, error) { + cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) + out := new(Message) + err := c.cc.Invoke(ctx, Tunnel_GitHubCredentials_FullMethodName, in, out, cOpts...) + if err != nil { + return nil, err + } + return out, nil +} + func (c *tunnelClient) GitSSHSignature(ctx context.Context, in *Message, opts ...grpc.CallOption) (*Message, error) { cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) out := new(Message) @@ -278,6 +290,7 @@ type TunnelServer interface { SendResult(context.Context, *Message) (*Empty, error) DockerCredentials(context.Context, *Message) (*Message, error) GitCredentials(context.Context, *Message) (*Message, error) + GitHubCredentials(context.Context, *Empty) (*Message, error) GitSSHSignature(context.Context, *Message) (*Message, error) GitUser(context.Context, *Empty) (*Message, error) DevsyConfig(context.Context, *Message) (*Message, error) @@ -317,6 +330,9 @@ func (UnimplementedTunnelServer) DockerCredentials(context.Context, *Message) (* func (UnimplementedTunnelServer) GitCredentials(context.Context, *Message) (*Message, error) { return nil, status.Error(codes.Unimplemented, "method GitCredentials not implemented") } +func (UnimplementedTunnelServer) GitHubCredentials(context.Context, *Empty) (*Message, error) { + return nil, status.Error(codes.Unimplemented, "method GitHubCredentials not implemented") +} func (UnimplementedTunnelServer) GitSSHSignature(context.Context, *Message) (*Message, error) { return nil, status.Error(codes.Unimplemented, "method GitSSHSignature not implemented") } @@ -479,6 +495,24 @@ func _Tunnel_GitCredentials_Handler(srv interface{}, ctx context.Context, dec fu return interceptor(ctx, in, info, handler) } +func _Tunnel_GitHubCredentials_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(Empty) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(TunnelServer).GitHubCredentials(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: Tunnel_GitHubCredentials_FullMethodName, + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(TunnelServer).GitHubCredentials(ctx, req.(*Empty)) + } + return interceptor(ctx, in, info, handler) +} + func _Tunnel_GitSSHSignature_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(Message) if err := dec(in); err != nil { @@ -687,6 +721,10 @@ var Tunnel_ServiceDesc = grpc.ServiceDesc{ MethodName: "GitCredentials", Handler: _Tunnel_GitCredentials_Handler, }, + { + MethodName: "GitHubCredentials", + Handler: _Tunnel_GitHubCredentials_Handler, + }, { MethodName: "GitSSHSignature", Handler: _Tunnel_GitSSHSignature_Handler, diff --git a/pkg/agent/tunnelserver/github_credentials_test.go b/pkg/agent/tunnelserver/github_credentials_test.go new file mode 100644 index 0000000000..c8cabbe5db --- /dev/null +++ b/pkg/agent/tunnelserver/github_credentials_test.go @@ -0,0 +1,21 @@ +package tunnelserver + +import ( + "context" + "testing" + + "github.com/devsy-org/devsy/pkg/agent/tunnel" + "github.com/stretchr/testify/require" +) + +func TestGitHubCredentialsRequiresIndependentPermission(t *testing.T) { + for _, server := range []*tunnelServer{ + New(), + New(WithAllowGitCredentials(true), WithAllowDockerCredentials(true)), + New(WithAllowGitHubCredentials(false)), + } { + response, err := server.GitHubCredentials(context.Background(), &tunnel.Empty{}) + require.ErrorContains(t, err, "forbidden") + require.Nil(t, response) + } +} diff --git a/pkg/agent/tunnelserver/options.go b/pkg/agent/tunnelserver/options.go index 50b89bf675..07fa6c0b37 100644 --- a/pkg/agent/tunnelserver/options.go +++ b/pkg/agent/tunnelserver/options.go @@ -13,6 +13,10 @@ import ( type Option func(*tunnelServer) *tunnelServer +func WithAllowGitHubCredentials(allow bool) Option { + return func(s *tunnelServer) *tunnelServer { s.allowGitHubCredentials = allow; return s } +} + func WithWorkspace(workspace *provider2.Workspace) Option { return func(s *tunnelServer) *tunnelServer { s.workspace = workspace diff --git a/pkg/agent/tunnelserver/tunnelserver.go b/pkg/agent/tunnelserver/tunnelserver.go index 7386ad806b..d107ec36df 100644 --- a/pkg/agent/tunnelserver/tunnelserver.go +++ b/pkg/agent/tunnelserver/tunnelserver.go @@ -21,6 +21,7 @@ import ( "github.com/devsy-org/devsy/pkg/devsyconfig" "github.com/devsy-org/devsy/pkg/dockercredentials" "github.com/devsy-org/devsy/pkg/extract" + "github.com/devsy-org/devsy/pkg/ghcredentials" "github.com/devsy-org/devsy/pkg/gitcredentials" "github.com/devsy-org/devsy/pkg/gitsshsigning" "github.com/devsy-org/devsy/pkg/gpg" @@ -31,7 +32,9 @@ import ( "github.com/devsy-org/devsy/pkg/status" "github.com/devsy-org/devsy/pkg/stdio" "google.golang.org/grpc" + "google.golang.org/grpc/codes" "google.golang.org/grpc/reflection" + grpcstatus "google.golang.org/grpc/status" ) func RunServicesServer( @@ -140,6 +143,7 @@ type tunnelServer struct { forwarder netstat.Forwarder allowGitCredentials bool + allowGitHubCredentials bool allowDockerCredentials bool allowKubeConfig bool allowPlatformOptions bool @@ -299,6 +303,20 @@ func (t *tunnelServer) GitUser(ctx context.Context, empty *tunnel.Empty) (*tunne }, nil } +func (t *tunnelServer) GitHubCredentials( + ctx context.Context, + _ *tunnel.Empty, +) (*tunnel.Message, error) { + if !t.allowGitHubCredentials { + return nil, grpcstatus.Error(codes.PermissionDenied, "GitHub CLI credentials forbidden") + } + b, err := json.Marshal(ghcredentials.HostAccounts(ctx)) + if err != nil { + return nil, fmt.Errorf("encode GitHub CLI accounts") + } + return &tunnel.Message{Message: string(b)}, nil +} + func (t *tunnelServer) GitCredentials( ctx context.Context, message *tunnel.Message, diff --git a/pkg/client/clientimplementation/services.go b/pkg/client/clientimplementation/services.go index d6b8913dad..25eeaaf5ec 100644 --- a/pkg/client/clientimplementation/services.go +++ b/pkg/client/clientimplementation/services.go @@ -24,6 +24,7 @@ type StartServicesDaemonOptions struct { type credentialConfig struct { docker bool git bool + github bool gitSSHSignature bool } @@ -52,6 +53,7 @@ func StartServicesDaemon(ctx context.Context, opts StartServicesDaemonOptions) e Workspace: opts.Client.WorkspaceConfig(), ConfigureDockerCredentials: credConfig.docker, ConfigureGitCredentials: credConfig.git, + ConfigureGitHubCredentials: credConfig.github, ConfigureGitSSHSignatureHelper: credConfig.gitSSHSignature, GitSSHSigningKey: opts.GitSSHSigningKey, }, @@ -70,6 +72,9 @@ func getCredentialConfig( workspace *managementv1.DevsyWorkspaceInstance, ) credentialConfig { cfg := credentialConfig{ + github: devsyConfig.ContextOption( + config.ContextOptionGitHubCLIForwarding, + ) == config.BoolTrue, docker: devsyConfig.ContextOption( config.ContextOptionSSHInjectDockerCredentials, ) == config.BoolTrue, @@ -96,6 +101,7 @@ func getCredentialConfig( } if instanceCredentialForwarding.Git != nil { cfg.git = !instanceCredentialForwarding.Git.Disabled + cfg.github = cfg.github && !instanceCredentialForwarding.Git.Disabled cfg.gitSSHSignature = !instanceCredentialForwarding.Git.Disabled } diff --git a/pkg/config/context.go b/pkg/config/context.go index d63ef53321..5dbea35fd0 100644 --- a/pkg/config/context.go +++ b/pkg/config/context.go @@ -12,6 +12,7 @@ const ( ContextOptionGitSSHSignatureForwarding = "GIT_SSH_SIGNATURE_FORWARDING" ContextOptionSSHInjectDockerCredentials = "SSH_INJECT_DOCKER_CREDENTIALS" ContextOptionSSHInjectGitCredentials = "SSH_INJECT_GIT_CREDENTIALS" + ContextOptionGitHubCLIForwarding = "SSH_INJECT_GITHUB_CREDENTIALS" ContextOptionExitAfterTimeout = "EXIT_AFTER_TIMEOUT" ContextOptionTelemetry = "TELEMETRY" ContextOptionAgentURL = "AGENT_URL" @@ -30,6 +31,12 @@ const ( ) var ContextOptions = []ContextOption{ + { + Name: ContextOptionGitHubCLIForwarding, + Description: "Specifies if Devsy should forward host GitHub CLI authentication into the workspace", + Default: BoolTrue, + Enum: []string{BoolTrue, BoolFalse}, + }, { Name: ContextOptionSSHAddPrivateKeys, Description: "Specifies if Devsy should automatically add ssh-keys to the ssh-agent", diff --git a/pkg/credentials/server.go b/pkg/credentials/server.go index 46ea8d39fc..f01a6ce672 100644 --- a/pkg/credentials/server.go +++ b/pkg/credentials/server.go @@ -17,6 +17,8 @@ import ( "github.com/devsy-org/devsy/pkg/config" "github.com/devsy-org/devsy/pkg/log" "google.golang.org/grpc" + "google.golang.org/grpc/codes" + grpcstatus "google.golang.org/grpc/status" ) const DefaultPort = "12049" @@ -120,6 +122,7 @@ func newCredentialsHandler( return err }, "/git-credentials": handleGitCredentialsRequest, + "/github-credentials": handleGitHubCredentialsRequest, "/docker-credentials": handleDockerCredentialsRequest, "/git-ssh-signature": handleGitSSHSignatureRequest, "/devsy-platform-credentials": handleDevsyPlatformCredentialsRequest, @@ -143,6 +146,41 @@ func newCredentialsHandler( }) } +func handleGitHubCredentialsRequest( + ctx context.Context, + writer http.ResponseWriter, + request *http.Request, + client CredentialsClient, +) error { + if request.Method != http.MethodPost { + writer.WriteHeader(http.StatusMethodNotAllowed) + return nil + } + githubClient, ok := client.(interface { + GitHubCredentials(context.Context, *tunnel.Empty, ...grpc.CallOption) (*tunnel.Message, error) + }) + if !ok { + return json.NewEncoder(writer).Encode(map[string]string{"status": "rpc_unavailable"}) + } + response, err := githubClient.GitHubCredentials(request.Context(), &tunnel.Empty{}) + if err != nil { + kind := "rpc_error" + switch grpcstatus.Code(err) { + case codes.PermissionDenied: + kind = "rpc_denied" + case codes.DeadlineExceeded: + kind = "rpc_timeout" + case codes.Canceled: + kind = "rpc_cancelled" + } + return json.NewEncoder(writer).Encode(map[string]string{"status": kind}) + } + writer.Header().Set("Content-Type", "application/json") + writer.Header().Set("Cache-Control", "no-store") + _, err = io.WriteString(writer, response.Message) + return err +} + const fetchOwnerTimeout = 2 * time.Second // maxOwnerResponseSize bounds how much of the /owner response FetchOwner diff --git a/pkg/ghcredentials/config.go b/pkg/ghcredentials/config.go new file mode 100644 index 0000000000..7cf314b162 --- /dev/null +++ b/pkg/ghcredentials/config.go @@ -0,0 +1,179 @@ +// Package ghcredentials supplies temporary GitHub CLI authentication without +// putting relayed credentials in the user's configuration. +package ghcredentials + +import ( + "errors" + "io" + "os" + "path/filepath" + "reflect" + "runtime" + + "gopkg.in/yaml.v3" +) + +const ( + hostsFile = "hosts.yml" + configFile = "config.yml" +) + +func ConfigDir() (string, error) { + if dir := os.Getenv("GH_CONFIG_DIR"); dir != "" { + return filepath.Abs(dir) + } + if dir := os.Getenv("XDG_CONFIG_HOME"); dir != "" { + return filepath.Abs(filepath.Join(dir, "gh")) + } + if dir := os.Getenv("APPDATA"); runtime.GOOS == "windows" && dir != "" { + return filepath.Abs(filepath.Join(dir, "GitHub CLI")) + } + home, err := os.UserHomeDir() + if err != nil { + return "", err + } + return filepath.Join(home, ".config", "gh"), nil +} + +func resolvedFile(path string) (*os.File, error) { + resolved, err := filepath.EvalSymlinks(path) + if err != nil { + return nil, err + } + return os.OpenInRoot(filepath.Dir(resolved), filepath.Base(resolved)) +} + +func readFile(path string) ([]byte, error) { + f, err := resolvedFile(path) + if errors.Is(err, os.ErrNotExist) { + return nil, nil + } + if err != nil { + return nil, err + } + defer func() { _ = f.Close() }() + data, err := io.ReadAll(io.LimitReader(f, 16<<20+1)) + if len(data) > 16<<20 { + return nil, errors.New("GitHub CLI configuration file too large") + } + return data, err +} + +func decode(data []byte) (*yaml.Node, error) { + var doc yaml.Node + if err := yaml.Unmarshal(data, &doc); err != nil { + // YAML errors may include credential values. + return nil, errInvalidConfig + } + if len(doc.Content) == 0 { + return mapping(), nil + } + n := doc.Content[0] + if n.Kind != yaml.MappingNode { + return nil, errInvalidConfig + } + return n, nil +} + +func get(n *yaml.Node, key string) *yaml.Node { + if n == nil || n.Kind != yaml.MappingNode { + return nil + } + for i := 0; i+1 < len(n.Content); i += 2 { + if n.Content[i].Value == key { + return n.Content[i+1] + } + } + return nil +} + +func put(n *yaml.Node, key string, value *yaml.Node) { + for i := 0; i+1 < len(n.Content); i += 2 { + if n.Content[i].Value == key { + if value == nil { + n.Content = append(n.Content[:i], n.Content[i+2:]...) + } else { + n.Content[i+1] = value + } + return + } + } + if value != nil { + n.Content = append(n.Content, scalar(key), value) + } +} + +func scalar( + s string, +) *yaml.Node { + return &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: s} +} + +func equal(a, b *yaml.Node) bool { + if a == nil || b == nil { + return a == b + } + var av, bv any + if a.Decode(&av) != nil || b.Decode(&bv) != nil { + return false + } + return reflect.DeepEqual(av, bv) +} + +// merge applies only this invocation's changes to the most recent original. +// Mapping entries are independent; sequences and scalars are single items. +func merge(before, after, latest *yaml.Node) *yaml.Node { + if equal(before, after) { + return latest + } + if mappingCompatible(before, after) { + if before == nil { + before = mapping() + } + if latest == nil || latest.Kind != yaml.MappingNode { + latest = mapping() + } + for key := range mappingKeys(before, after) { + put(latest, key, merge(get(before, key), get(after, key), get(latest, key))) + } + return latest + } + return after +} + +func mergeYAML(before, after, latest []byte) ([]byte, error) { + b, err := decode(before) + if err != nil { + return nil, err + } + a, err := decode(after) + if err != nil { + return nil, err + } + l, err := decode(latest) + if err != nil { + return nil, err + } + result := merge(b, a, l) + if equal(result, l) && equal(b, a) { + return latest, nil + } + return yaml.Marshal(result) +} + +func mappingKeys(nodes ...*yaml.Node) map[string]bool { + keys := map[string]bool{} + for _, n := range nodes { + for i := 0; i+1 < len(n.Content); i += 2 { + keys[n.Content[i].Value] = true + } + } + return keys +} + +func mappingCompatible(before, after *yaml.Node) bool { + if after == nil || after.Kind != yaml.MappingNode { + return false + } + return before == nil || before.Kind == yaml.MappingNode +} diff --git a/pkg/ghcredentials/diagnostics.go b/pkg/ghcredentials/diagnostics.go new file mode 100644 index 0000000000..824279be58 --- /dev/null +++ b/pkg/ghcredentials/diagnostics.go @@ -0,0 +1,191 @@ +package ghcredentials + +import ( + "encoding/json" + "os" + "path/filepath" + "sort" + "time" + + "github.com/devsy-org/devsy/pkg/version" +) + +type hostDecision struct { + Host string `json:"host"` + Source string `json:"source"` +} + +// Only these source-allowlisted fields may enter local diagnostics. Command +// arguments, config contents, accounts, error text, and token values are never +// recorded. gh owns target selection, so its arguments need no diagnostic copy. +type diagnostic struct { + Version string `json:"version"` + Started time.Time `json:"started"` + Finished time.Time `json:"finished,omitzero"` + Complete bool `json:"complete"` + Binary string `json:"binary"` + ConfigDir string `json:"config_dir,omitempty"` + Inputs map[string]string `json:"inputs"` + Hosts []hostDecision `json:"hosts,omitempty"` + Stage string `json:"stage"` + Code int `json:"exit_code"` + Signal int `json:"signal,omitempty"` + HostStatus string `json:"host_status,omitempty"` + HostContext HostContext `json:"host_context"` + HostDecisions []hostDecision `json:"host_decisions,omitempty"` + Operations []operation `json:"operations,omitempty"` + Writeback *writebackSummary `json:"writeback,omitempty"` + Recovery []recoveryDecision `json:"recovery,omitempty"` + path string +} + +func beginDiagnostic(binary string) *diagnostic { + d := &diagnostic{ + Version: version.GetVersion(), + Binary: binary, + Started: time.Now(), + Stage: "started", + } + if os.Getenv("DEVSY_GH_DIAGNOSTICS") == "false" { + return d + } + root, err := sessionRoot() + if err != nil { + return d + } + dir := filepath.Join(root, "diagnostics") + if os.MkdirAll(dir, 0o700) != nil { + return d + } + f, err := os.CreateTemp(dir, "run-*.json") + if err != nil { + return d + } + d.path = f.Name() + _ = f.Close() + d.Inputs = credentialInputs() + d.save(false) + return d +} + +func (d *diagnostic) run(binary string, args, env []string, owner *os.File) int { + started := time.Now() + result := runGH(binary, args, env, owner) + d.Code = result.Code + d.Signal = result.Signal + if d.path != "" { + op := operation{ + Name: "gh.execute", + Status: "ok", + DurationMS: time.Since(started).Milliseconds(), + ErrorType: result.ErrorType, + } + if result.Code != 0 { + op.Status = "error" + } + if result.Signal != 0 { + op.Status = "cancelled" + } + d.Operations = append(d.Operations, op) + } + return d.Code +} + +func (d *diagnostic) save(complete bool) { + if d.path == "" { + return + } + d.Complete = complete + b, err := json.Marshal(d) + if err == nil { + _ = os.WriteFile(d.path, b, 0o600) + } +} + +func (d *diagnostic) finish() { + if d.path == "" { + return + } + d.Finished = time.Now() + d.save(true) + entries, _ := os.ReadDir(filepath.Dir(d.path)) + // Fixed-size records and a count bound cap local disk usage. Incomplete + // records identify killed runs; no remote diagnostic export exists. + sort.Slice(entries, func(i, j int) bool { + a, b := entries[i], entries[j] + successA := diagnosticSuccess(filepath.Join(filepath.Dir(d.path), a.Name())) + successB := diagnosticSuccess(filepath.Join(filepath.Dir(d.path), b.Name())) + if successA != successB { + return successA + } + ai, _ := a.Info() + bi, _ := b.Info() + return ai != nil && bi != nil && ai.ModTime().Before(bi.ModTime()) + }) + for len(entries) > 64 { + _ = os.Remove(filepath.Join(filepath.Dir(d.path), entries[0].Name())) + entries = entries[1:] + } +} + +func diagnosticSuccess(path string) bool { + b, err := readFile(path) + if err != nil { + return false + } + var d diagnostic + if json.Unmarshal(b, &d) != nil { + return false + } + return d.Complete && d.Code == 0 && d.Stage != "writeback_pending" +} + +type operation struct { + Name string `json:"name"` + Status string `json:"status"` + ErrorType string `json:"error_type,omitempty"` + DurationMS int64 `json:"duration_ms"` +} + +func (d *diagnostic) observe(name string, started time.Time, err error) { + if d.path == "" { + return + } + op := operation{Name: name, Status: "ok", DurationMS: time.Since(started).Milliseconds()} + if err != nil { + op.Status = "error" + op.Name, op.ErrorType = describeError(name, err) + } + d.Operations = append(d.Operations, op) +} + +func credentialInputs() map[string]string { + inputs := map[string]string{} + for _, name := range []string{ + "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", + "GH_HOST", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "HOME", "APPDATA", + } { + value, present := os.LookupEnv(name) + state := "unset" + if present { + state = "empty" + if value != "" { + state = "set" + } + } + inputs[name] = state + } + return inputs +} + +func (d *diagnostic) hostResult(result HostResult) { + if d.path == "" || result.Status == "" { + return + } + op := operation{Name: "host.resolve", Status: "ok", DurationMS: result.DurationMS} + if result.Status != "available" { + op.Status = "unavailable" + op.ErrorType = result.Status + } + d.Operations = append(d.Operations, op) +} diff --git a/pkg/ghcredentials/host.go b/pkg/ghcredentials/host.go new file mode 100644 index 0000000000..223c9444ac --- /dev/null +++ b/pkg/ghcredentials/host.go @@ -0,0 +1,99 @@ +package ghcredentials + +import ( + "context" + "os" + "os/exec" + "path/filepath" + "strings" + "time" + + "gopkg.in/yaml.v3" +) + +type Account struct { + Host string `json:"host"` + User string `json:"user"` + Token string `json:"token"` +} + +// HostAccounts reads active accounts through the host's unmodified gh. It never +// runs login, refresh, switch, logout, or an API request. +func HostAccounts(ctx context.Context) (result HostResult) { + started := time.Now() + info := HostContext{Inputs: credentialInputs()} + defer func() { result.Context = info; result.DurationMS = time.Since(started).Milliseconds() }() + binary, err := exec.LookPath("gh") + if err != nil { + return HostResult{Status: "gh_not_found"} + } + info.Binary = binary + dir, err := ConfigDir() + if err != nil { + return HostResult{Status: "config_path_failed"} + } + info.ConfigDir = dir + b, err := readFile(filepath.Join(dir, hostsFile)) + if err != nil { + return HostResult{Status: "config_read_" + errorKind(err)} + } + hosts, err := decode(b) + if err != nil { + return HostResult{Status: "config_decode_failed"} + } + return collectHostAccounts(ctx, binary, hosts) +} + +func collectHostAccounts(ctx context.Context, binary string, hosts *yaml.Node) HostResult { + result := HostResult{Status: "no_active_accounts"} + for i := 0; i+1 < len(hosts.Content); i += 2 { + host := hosts.Content[i].Value + u := get(hosts.Content[i+1], "user") + if u == nil || u.Value == "" { + result.Decisions = append( + result.Decisions, + hostDecision{Host: host, Source: "no_active_account"}, + ) + continue + } + token, err := resolveToken(ctx, binary, host, os.Environ()) + if err != nil { + result.Decisions = append( + result.Decisions, + hostDecision{Host: host, Source: "token_" + errorKind(err)}, + ) + continue + } + if token != "" { + result.Accounts = append( + result.Accounts, + Account{Host: host, User: u.Value, Token: token}, + ) + } + } + if len(result.Accounts) > 0 { + result.Status = "available" + } + return result +} + +func tokenForHost(ctx context.Context, binary, host string, env []string) string { + token, _ := resolveToken(ctx, binary, host, env) + return token +} + +func resolveToken(ctx context.Context, binary, host string, env []string) (string, error) { + ctx, cancel := context.WithTimeout(ctx, 3*time.Second) + defer cancel() + //nolint:gosec // binary is resolved from the user's PATH; host is one argument. + cmd := exec.CommandContext(ctx, binary, "auth", "token", "--hostname", host) + cmd.Env = env + out, err := cmd.Output() + if err != nil { + if ctx.Err() != nil { + return "", ctx.Err() + } + return "", err + } + return strings.TrimSpace(string(out)), nil +} diff --git a/pkg/ghcredentials/install.go b/pkg/ghcredentials/install.go new file mode 100644 index 0000000000..ae37820894 --- /dev/null +++ b/pkg/ghcredentials/install.go @@ -0,0 +1,50 @@ +package ghcredentials + +import ( + "fmt" + "os" + "path/filepath" + "strings" + + "al.essio.dev/pkg/shellescape" + "github.com/devsy-org/devsy/pkg/envfile" +) + +const WrapperDir = "/usr/local/share/devsy/gh/bin" + +// Install owns only its private executable and a system profile fragment. The +// user's gh binaries, configuration, and shell dotfiles are never replaced. +func Install(port int) error { + binary, err := os.Executable() + if err != nil { + return err + } + // #nosec G301 -- Public executable path; it contains no authentication data. + if err := os.MkdirAll(WrapperDir, 0o755); err != nil { + return err + } + script := fmt.Sprintf( + "#!/bin/sh\n# devsy gh relay\nexport DEVSY_CREDENTIALS_SERVER_PORT=%d\nexec %s internal gh \"$0\" \"$@\"\n", + port, + shellescape.Quote(binary), + ) + //nolint:gosec // This is the executable wrapper, containing no credentials. + if err := os.WriteFile(filepath.Join(WrapperDir, "gh"), []byte(script), 0o755); err != nil { + return err + } + // Devsy SSH/IDE sessions read envfile; login shells also need the wrapper. + path := os.Getenv("PATH") + if !strings.HasPrefix(path, WrapperDir+":") { + path = WrapperDir + ":" + path + } + envfile.MergeAndApply(map[string]string{"PATH": path}) + // #nosec G301 -- System shell profiles must be readable by workspace users. + if err := os.MkdirAll("/etc/profile.d", 0o755); err != nil { + return err + } + profile := fmt.Sprintf(`# devsy gh relay +case ":$PATH:" in *:%s:*) ;; *) export PATH=%s:"$PATH" ;; esac +`, WrapperDir, WrapperDir) + // #nosec G306 -- Public PATH configuration, containing no credentials. + return os.WriteFile("/etc/profile.d/devsy-gh.sh", []byte(profile), 0o644) +} diff --git a/pkg/ghcredentials/owner_linux.go b/pkg/ghcredentials/owner_linux.go new file mode 100644 index 0000000000..11cc85456a --- /dev/null +++ b/pkg/ghcredentials/owner_linux.go @@ -0,0 +1,34 @@ +package ghcredentials + +import ( + "crypto/sha256" + "errors" + "fmt" + "os" + + "golang.org/x/sys/unix" +) + +type ownerLock struct{ file *os.File } + +// Kernel ownership follows the inherited descriptor, independently of the +// cache filesystem's lock semantics. The real gh keeps this session alive +// even if its wrapper is killed; no socket accepts or transmits credentials. +func tryOwner(path string) (*ownerLock, bool, error) { + key := sha256.Sum256([]byte(fmt.Sprintf("%d:%s", os.Getuid(), path))) + fd, err := unix.Socket(unix.AF_UNIX, unix.SOCK_DGRAM|unix.SOCK_CLOEXEC, 0) + if err != nil { + return nil, false, err + } + err = unix.Bind(fd, &unix.SockaddrUnix{Name: fmt.Sprintf("@devsy-gh-%x", key[:20])}) + if err == nil { + return &ownerLock{file: os.NewFile(uintptr(fd), "gh-session-owner")}, true, nil + } + _ = unix.Close(fd) + if errors.Is(err, unix.EADDRINUSE) { + return nil, false, nil + } + return nil, false, err +} + +func (l *ownerLock) Close() error { return l.file.Close() } diff --git a/pkg/ghcredentials/owner_unix.go b/pkg/ghcredentials/owner_unix.go new file mode 100644 index 0000000000..c3b8ca4042 --- /dev/null +++ b/pkg/ghcredentials/owner_unix.go @@ -0,0 +1,39 @@ +//go:build !linux && !windows + +package ghcredentials + +import ( + "os" + "path/filepath" + + "golang.org/x/sys/unix" +) + +type ownerLock struct{ file *os.File } + +func tryOwner(path string) (*ownerLock, bool, error) { + f, err := ownerFile(path) + if err != nil { + return nil, false, err + } + err = unix.Flock(int(f.Fd()), unix.LOCK_EX|unix.LOCK_NB) + if err == nil { + return &ownerLock{file: f}, true, nil + } + _ = f.Close() + if err == unix.EWOULDBLOCK { + return nil, false, nil + } + return nil, false, err +} + +func (l *ownerLock) Close() error { return l.file.Close() } + +func ownerFile(path string) (*os.File, error) { + root, err := os.OpenRoot(filepath.Dir(path)) + if err != nil { + return nil, err + } + defer func() { _ = root.Close() }() + return root.OpenFile(filepath.Base(path), os.O_CREATE|os.O_RDWR, 0o600) +} diff --git a/pkg/ghcredentials/process_unix.go b/pkg/ghcredentials/process_unix.go new file mode 100644 index 0000000000..af9ba73ff6 --- /dev/null +++ b/pkg/ghcredentials/process_unix.go @@ -0,0 +1,50 @@ +//go:build !windows + +package ghcredentials + +import ( + "os" + "os/exec" + "os/signal" + "syscall" +) + +func runGH(binary string, args, env []string, owner *os.File) processResult { + //nolint:gosec // The user chose this executable and these unchanged arguments. + cmd := exec.Command(binary, args...) + cmd.Env, cmd.Stdin, cmd.Stdout, cmd.Stderr = env, os.Stdin, os.Stdout, os.Stderr + if owner != nil { + cmd.ExtraFiles = []*os.File{owner} + } + signals := make(chan os.Signal, 8) + signal.Notify(signals, os.Interrupt, syscall.SIGTERM, syscall.SIGHUP) + defer signal.Stop(signals) + if err := cmd.Start(); err != nil { + return processResult{Code: 126, ErrorType: errorKind(err)} + } + done := make(chan struct{}) + defer close(done) + go forwardSignals(cmd.Process, signals, done) + if err := cmd.Wait(); err != nil { + if status, ok := cmd.ProcessState.Sys().(syscall.WaitStatus); ok && status.Signaled() { + return processResult{ + Code: 128 + int(status.Signal()), + Signal: int(status.Signal()), + ErrorType: "signal", + } + } + return processResult{Code: cmd.ProcessState.ExitCode(), ErrorType: "exit_nonzero"} + } + return processResult{} +} + +func forwardSignals(process *os.Process, signals <-chan os.Signal, done <-chan struct{}) { + for { + select { + case sig := <-signals: + _ = process.Signal(sig) + case <-done: + return + } + } +} diff --git a/pkg/ghcredentials/process_windows.go b/pkg/ghcredentials/process_windows.go new file mode 100644 index 0000000000..8ae0263bc8 --- /dev/null +++ b/pkg/ghcredentials/process_windows.go @@ -0,0 +1,38 @@ +package ghcredentials + +import ( + "os" + "os/exec" + + "github.com/gofrs/flock" +) + +type ownerLock struct { + file *os.File + lock *flock.Flock +} + +func tryOwner(path string) (*ownerLock, bool, error) { + l := flock.New(path) + ok, err := l.TryLock() + if err != nil || !ok { + _ = l.Close() + return nil, ok, err + } + return &ownerLock{lock: l}, true, nil +} + +func (l *ownerLock) Close() error { return l.lock.Close() } + +func runGH(binary string, args, env []string, _ *os.File) processResult { + //nolint:gosec // The user chose this executable and these unchanged arguments. + cmd := exec.Command(binary, args...) + cmd.Env, cmd.Stdin, cmd.Stdout, cmd.Stderr = env, os.Stdin, os.Stdout, os.Stderr + if err := cmd.Run(); err != nil { + if cmd.ProcessState != nil { + return processResult{Code: cmd.ProcessState.ExitCode(), ErrorType: "exit_nonzero"} + } + return processResult{Code: 126, ErrorType: errorKind(err)} + } + return processResult{} +} diff --git a/pkg/ghcredentials/result.go b/pkg/ghcredentials/result.go new file mode 100644 index 0000000000..99786e3630 --- /dev/null +++ b/pkg/ghcredentials/result.go @@ -0,0 +1,89 @@ +package ghcredentials + +import ( + "context" + "errors" + "os" + "os/exec" + "syscall" +) + +// HostResult separates an unavailable authentication source from a broken +// transport. Only Accounts contains secrets; Status and Decisions are safe for +// the local diagnostic recorder. +type HostResult struct { + Accounts []Account `json:"accounts"` + Status string `json:"status"` + Decisions []hostDecision `json:"decisions,omitempty"` + Context HostContext `json:"context"` + DurationMS int64 `json:"duration_ms"` +} + +type HostContext struct { + Binary string `json:"binary,omitempty"` + ConfigDir string `json:"config_dir,omitempty"` + Inputs map[string]string `json:"inputs,omitempty"` +} + +type boundaryError struct { + operation, kind string + cause error +} + +func (e *boundaryError) Error() string { return e.operation + ": " + e.kind } +func (e *boundaryError) Unwrap() error { return e.cause } + +func boundary(operation string, err error) error { + if err == nil { + return nil + } + return &boundaryError{operation: operation, kind: errorKind(err), cause: err} +} + +func errorKind(err error) string { + classes := []struct { + cause error + kind string + }{ + {context.DeadlineExceeded, "timeout"}, + {context.Canceled, "cancelled"}, + {os.ErrPermission, "permission_denied"}, + {os.ErrNotExist, "not_found"}, + {syscall.ECONNREFUSED, "connection_refused"}, + {errInvalidConfig, "schema_invalid"}, + } + for _, class := range classes { + if errors.Is(err, class.cause) { + return class.kind + } + } + if _, ok := errors.AsType[*exec.ExitError](err); ok { + return "process_exit" + } + return "io_error" +} + +var errInvalidConfig = errors.New("invalid GitHub CLI configuration") + +func lockOwner(path string) (*ownerLock, error) { + l, ok, err := tryOwner(path) + if err != nil { + return nil, err + } + if !ok { + return nil, errors.New("GitHub CLI session is active") + } + return l, nil +} + +func describeError(name string, err error) (string, string) { + if boundary, ok := errors.AsType[*boundaryError](err); ok { + return boundary.operation, boundary.kind + } + return name, errorKind(err) +} + +type processResult struct { + Code, Signal int + ErrorType string +} diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go new file mode 100644 index 0000000000..c2986d2e08 --- /dev/null +++ b/pkg/ghcredentials/session.go @@ -0,0 +1,558 @@ +package ghcredentials + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "strings" + "time" + + "github.com/gofrs/flock" + "gopkg.in/yaml.v3" +) + +type session struct { + Original string `json:"original"` + Accounts []Account `json:"accounts"` + dir string + lock *ownerLock + decisions []hostDecision + writeback *writebackSummary +} + +func sessionRoot() (string, error) { + dir, err := os.UserCacheDir() + if err != nil { + return "", err + } + dir = filepath.Join(dir, "devsy", "gh-relay") + if err := os.MkdirAll(dir, 0o700); err != nil { + return "", err + } + return dir, nil +} + +func lock(ctx context.Context, path string) (*flock.Flock, error) { + l := flock.New(path) + ok, err := l.TryLockContext(ctx, 20*time.Millisecond) + if err != nil { + return nil, err + } + if !ok { + return nil, errors.New("GitHub CLI configuration lock unavailable") + } + return l, nil +} + +func startSession(ctx context.Context, original string, accounts []Account) (*session, error) { + root, err := sessionRoot() + if err != nil { + return nil, err + } + l, err := lock(ctx, filepath.Join(root, "sessions.lock")) + if err != nil { + return nil, boundary("session.catalog_lock", err) + } + defer func() { _ = l.Close() }() + recoverSessions(ctx, root, false) + return newSession(ctx, root, original, accounts) +} + +func newSession(ctx context.Context, root, original string, accounts []Account) (*session, error) { + dir, err := os.MkdirTemp(root, "session-") + if err != nil { + return nil, err + } + owner, err := lockOwner(filepath.Join(dir, "owner.lock")) + if err != nil { + _ = os.RemoveAll(dir) + return nil, boundary("session.owner_lock", err) + } + s := &session{Original: original, Accounts: accounts, dir: dir, lock: owner} + ready := false + defer func() { + if !ready { + _ = owner.Close() + _ = os.RemoveAll(dir) + } + }() + if err := copyTree(ctx, original, filepath.Join(dir, "before"), 0); err != nil { + return nil, boundary("snapshot.read", err) + } + if err := copyTree(ctx, filepath.Join(dir, "before"), s.work(), 0); err != nil { + return nil, boundary("snapshot.copy", err) + } + initial := *s + initial.Accounts = nil // No forwarded values exist until inject completes. + if err := initial.save(); err != nil { + return nil, boundary("session.metadata", err) + } + ready = true + return s, nil +} + +func (s *session) work() string { return filepath.Join(s.dir, "work") } + +func copyTree(ctx context.Context, src, dst string, depth int) error { + if err := ctx.Err(); err != nil { + return err + } + if depth > 32 { + return errors.New("GitHub CLI configuration symlink cycle") + } + if err := os.MkdirAll(dst, 0o700); err != nil { + return err + } + entries, err := os.ReadDir(src) + if errors.Is(err, os.ErrNotExist) { + return nil + } + if err != nil { + return err + } + for _, e := range entries { + if err := copyEntry( + ctx, + filepath.Join(src, e.Name()), + filepath.Join(dst, e.Name()), + depth, + ); err != nil { + return err + } + } + return nil +} + +func copyEntry(ctx context.Context, src, dst string, depth int) error { + info, err := os.Stat(src) // Follow symlinks into a private copy, never a link to the original. + if err != nil { + return err + } + if info.IsDir() { + return copyTree(ctx, src, dst, depth+1) + } + if !info.Mode().IsRegular() { + return nil + } + if info.Size() > 16<<20 { + return errors.New("GitHub CLI configuration file too large") + } + data, err := readFile(src) + if err != nil { + return err + } + mode := fs.FileMode(0o600) + if info.Mode()&0o111 != 0 { + mode = 0o700 + } + return os.WriteFile(dst, data, mode) +} + +type recoveryDecision struct { + Session string `json:"session"` + ConfigDir string `json:"config_dir,omitempty"` + Outcome string `json:"outcome"` + Operation string `json:"operation,omitempty"` + ErrorType string `json:"error_type,omitempty"` + Writeback *writebackSummary `json:"writeback,omitempty"` +} + +func recoverSessions(ctx context.Context, root string, recording bool) []recoveryDecision { + var results []recoveryDecision + entries, _ := os.ReadDir(root) + for _, e := range entries { + if e.IsDir() && strings.HasPrefix(e.Name(), "session-") { + results = append(results, recoverSession(ctx, filepath.Join(root, e.Name()), recording)) + } + } + return results +} + +func recoverSession(ctx context.Context, dir string, recording bool) recoveryDecision { + result := recoveryDecision{Session: filepath.Base(dir), Outcome: "active"} + owner, ok, err := tryOwner(filepath.Join(dir, "owner.lock")) + if err != nil { + return recoveryFailure(result, "recovery.owner_lock", err) + } + if !ok { + return result + } + defer func() { _ = owner.Close() }() + return recoverUnlocked(ctx, dir, recording) +} + +func recoverUnlocked(ctx context.Context, dir string, recording bool) recoveryDecision { + result := recoveryDecision{Session: filepath.Base(dir), Outcome: "recovered"} + old, err := loadSession(dir) + if err != nil { + return recoveryFailure(result, "recovery.metadata", err) + } + if old == nil { + _ = os.RemoveAll(dir) + return result + } + result.ConfigDir = old.Original + if recording { + old.writeback = &writebackSummary{} + } + err = old.sync(ctx) + result.Writeback = old.writeback + if err != nil { + return recoveryFailure(result, "recovery.writeback", err) + } + _ = os.RemoveAll(dir) + return result +} + +func loadSession(dir string) (*session, error) { + b, err := readFile(filepath.Join(dir, "session.json")) + if err != nil { + return nil, err + } + if len(b) == 0 { + return nil, nil + } + var old session + if json.Unmarshal(b, &old) != nil || !filepath.IsAbs(old.Original) { + return nil, errInvalidConfig + } + old.dir = dir + return &old, nil +} + +func recoveryFailure(result recoveryDecision, name string, err error) recoveryDecision { + result.Outcome = "pending" + result.Operation, result.ErrorType = describeError(name, err) + return result +} + +func (s *session) save() error { + b, err := json.Marshal(s) + if err != nil { + return err + } + return os.WriteFile(filepath.Join(s.dir, "session.json"), b, 0o600) +} + +func localAuthSource(n *yaml.Node) string { + if value := get(n, "oauth_token"); value != nil && value.Value != "" { + return "local_config" + } + users := get(n, "users") + if users != nil { + for i := 1; i < len(users.Content); i += 2 { + if get(users.Content[i], "oauth_token") != nil { + return "local_config" + } + } + } + if user := get(n, "user"); user != nil && user.Value != "" { + return "local_registration" + } + return "" +} + +func mapping() *yaml.Node { return &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"} } + +func ensure(n *yaml.Node, key string) *yaml.Node { + v := get(n, key) + if v == nil || v.Kind != yaml.MappingNode { + v = mapping() + put(n, key, v) + } + return v +} + +func (s *session) inject(ctx context.Context, binary string, env []string) error { + path := filepath.Join(s.work(), hostsFile) + b, err := readFile(path) + if err != nil { + return boundary("config.read", err) + } + hosts, err := decode(b) + if err != nil { + return boundary("config.decode", err) + } + var injected []Account + for _, a := range s.Accounts { + source := a.inject(ctx, binary, env, hosts) + s.decisions = append(s.decisions, hostDecision{Host: a.Host, Source: source}) + if source == "forwarded" { + injected = append(injected, a) + } + } + s.Accounts = injected + if err := s.save(); err != nil { + return err + } + if len(injected) == 0 { + return nil + } + b, err = yaml.Marshal(hosts) + if err != nil { + return err + } + return os.WriteFile(path, b, 0o600) +} + +func (s *session) strip(data []byte) ([]byte, error) { + hosts, err := decode(data) + if err != nil { + return nil, err + } + for _, a := range s.Accounts { + stripAccount(hosts, a) + } + return yaml.Marshal(hosts) +} + +func files(dir string) (map[string]bool, error) { + result := map[string]bool{} + err := filepath.WalkDir(dir, func(path string, e fs.DirEntry, err error) error { + if errors.Is(err, os.ErrNotExist) { + return nil + } + if err != nil { + return err + } + if !e.IsDir() { + rel, _ := filepath.Rel(dir, path) + result[rel] = true + } + return nil + }) + return result, err +} + +func (s *session) sync(ctx context.Context) error { + root := filepath.Dir(s.dir) + key := sha256.Sum256([]byte(s.Original)) + l, err := lock(ctx, filepath.Join(root, fmt.Sprintf("config-%x.lock", key[:16]))) + if err != nil { + return boundary("config.lock", err) + } + defer func() { _ = l.Close() }() + beforeDir := filepath.Join(s.dir, "before") + names, err := files(beforeDir) + if err != nil { + return err + } + afterNames, err := files(s.work()) + if err != nil { + return err + } + for name := range afterNames { + names[name] = true + } + for name := range names { + if err := s.syncFile(name); err != nil { + return err + } + } + return nil +} + +func writeOriginal(path string, data []byte, mode fs.FileMode) error { + var err error + path, err = resolveOriginal(path) + if err != nil { + return err + } + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + return err + } + f, err := os.CreateTemp(filepath.Dir(path), ".devsy-gh-") + if err != nil { + return err + } + name := f.Name() + defer func() { _ = os.Remove(name) }() + if err := f.Chmod(mode); err != nil { + _ = f.Close() + return err + } + if _, err := f.Write(data); err != nil { + _ = f.Close() + return err + } + if err := f.Close(); err != nil { + return err + } + return os.Rename(name, path) +} + +func (s *session) finish(ctx context.Context) error { + err := s.sync(ctx) + if err == nil { + _ = os.RemoveAll(s.dir) + } + _ = s.lock.Close() + return err +} + +func (a Account) inject(ctx context.Context, binary string, env []string, hosts *yaml.Node) string { + if a.Host == "" || a.User == "" || a.Token == "" { + return "invalid_account" + } + if source := localAuthSource(get(hosts, a.Host)); source != "" { + return source + } + if token := tokenForHost(ctx, binary, a.Host, env); token != "" { + return localTokenSource(token, env) + } + h := ensure(hosts, a.Host) + put(h, "user", scalar(a.User)) + put(h, "oauth_token", scalar(a.Token)) + put(ensure(ensure(h, "users"), a.User), "oauth_token", scalar(a.Token)) + return "forwarded" +} + +func stripAccount(hosts *yaml.Node, a Account) { + h := get(hosts, a.Host) + if h == nil { + return + } + u := get(h, "users") + account := get(u, a.User) + activeToken := get(h, "oauth_token") + userToken := get(account, "oauth_token") + // Successful keyring login removes both file tokens. Keep its account + // registration; the keyring itself remains entirely under gh's control. + keyringLogin := activeToken == nil && userToken == nil + removeForwardedToken(h, a.Token) + removeForwardedToken(account, a.Token) + if !keyringLogin { + stripRegistration(h, u, account, a.User) + } + if len(h.Content) == 0 { + put(hosts, a.Host, nil) + } +} + +func removeForwardedToken(n *yaml.Node, token string) { + if t := get(n, "oauth_token"); t != nil && t.Value == token { + put(n, "oauth_token", nil) + } +} + +func stripRegistration(host, users, account *yaml.Node, user string) { + if t := get(host, "user"); t != nil && t.Value == user && get(account, "oauth_token") == nil { + put(host, "user", nil) + } + if account != nil && len(account.Content) == 0 { + put(users, user, nil) + } + if users != nil && len(users.Content) == 0 { + put(host, "users", nil) + } +} + +func (s *session) changedFile(name string) ([]byte, []byte, error) { + before, err := readFile(filepath.Join(s.dir, "before", name)) + if err != nil { + return nil, nil, boundary("config.snapshot_read", err) + } + after, err := readFile(filepath.Join(s.work(), name)) + if err != nil { + return nil, nil, boundary("config.changed_read", err) + } + if name == hostsFile && len(after) != 0 { + after, err = s.strip(after) + if err != nil { + return nil, nil, boundary("config.strip", err) + } + } + return before, after, nil +} + +func (s *session) syncFile(name string) error { + before, after, err := s.changedFile(name) + if err != nil { + return err + } + + if bytes.Equal(before, after) { + s.writeback.unchanged() + return nil + } + path := filepath.Join(s.Original, name) + latest, err := readFile(path) + if err != nil { + return boundary("config.latest_read", err) + } + after, err = s.mergeFile(name, before, after, latest) + if err != nil { + return boundary("config.merge", err) + } + if bytes.Equal(latest, after) { + return nil + } + err = s.saveOriginal(name, after) + s.writeback.written(err) + return boundary("config.write", err) +} + +func (s *session) mergeFile(name string, before, after, latest []byte) ([]byte, error) { + if err := s.writeback.inspect(name, before, after, latest); err != nil { + return nil, err + } + if name == hostsFile || name == configFile { + return mergeYAML(before, after, latest) + } + return after, nil +} + +func (s *session) saveOriginal(name string, after []byte) error { + path := filepath.Join(s.Original, name) + if after == nil { + err := os.Remove(path) + if errors.Is(err, os.ErrNotExist) { + return nil + } + return err + } + mode := fs.FileMode(0o600) + info, err := os.Stat(filepath.Join(s.work(), name)) + if err == nil && info.Mode()&0o111 != 0 { + mode = 0o700 + } + return writeOriginal(path, after, mode) +} + +func localTokenSource(token string, env []string) string { + for _, entry := range env { + name, value, _ := strings.Cut(entry, "=") + if authEnvironment[name] && value == token { + return "environment" + } + } + return "keyring" +} + +var authEnvironment = map[string]bool{ + "GH_TOKEN": true, + "GITHUB_TOKEN": true, + "GH_ENTERPRISE_TOKEN": true, + "GITHUB_ENTERPRISE_TOKEN": true, +} + +func resolveOriginal(path string) (string, error) { + // Resolve the current link target at writeback time, including symlinked + // parent directories. Never replace a dotfiles manager's symlink. + if resolved, err := filepath.EvalSymlinks(path); err == nil { + path = resolved + } else if !errors.Is(err, os.ErrNotExist) { + return "", err + } else if _, linkErr := os.Lstat(path); linkErr == nil { + // A dangling link is still the user's link. Keep the changes pending + // rather than replacing it with a regular file. + return "", errors.New("GitHub CLI configuration target unavailable") + } + return path, nil +} diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go new file mode 100644 index 0000000000..07901ba043 --- /dev/null +++ b/pkg/ghcredentials/session_test.go @@ -0,0 +1,164 @@ +package ghcredentials + +import ( + "context" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" +) + +func writeTestFile(t *testing.T, path, content string) { + t.Helper() + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o700)) + require.NoError(t, os.WriteFile(path, []byte(content), 0o600)) +} + +func TestSessionMergePreservesDotfilesAndConcurrentChanges(t *testing.T) { + root := t.TempDir() + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + t.Setenv("HOME", root) + original := filepath.Join(root, "gh") + target := filepath.Join(root, "dotfiles", configFile) + writeTestFile(t, target, "editor: vi\npager: less\n") + require.NoError(t, os.MkdirAll(original, 0o700)) + require.NoError(t, os.Symlink(target, filepath.Join(original, configFile))) + s, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + writeTestFile(t, filepath.Join(s.work(), configFile), "editor: nano\npager: less\n") + s.writeback = &writebackSummary{} + // Dotfiles reapplies a new original during the command. + writeTestFile(t, target, "editor: emacs\npager: cat\nprompt: disabled\n") + require.NoError(t, s.finish(context.Background())) + link, err := os.Readlink(filepath.Join(original, configFile)) + require.NoError(t, err) + require.Equal(t, target, link) + got, err := readFile(target) + require.NoError(t, err) + n, err := decode(got) + require.NoError(t, err) + require.Equal(t, "nano", get(n, "editor").Value) + require.Equal(t, "cat", get(n, "pager").Value) + require.Equal(t, "disabled", get(n, "prompt").Value) + require.Equal(t, 1, s.writeback.WrittenFiles) + require.Equal(t, changeCount{Modified: 1, Conflicts: 1}, s.writeback.Changes["config.editor"]) + require.NotContains(t, s.writeback.Changes, "config.pager") +} + +func TestSessionConcurrentInvocationsAndRecovery(t *testing.T) { + root := t.TempDir() + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + t.Setenv("HOME", root) + original := filepath.Join(root, "gh") + writeTestFile(t, filepath.Join(original, configFile), "editor: vi\npager: less\n") + a, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + b, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + writeTestFile(t, filepath.Join(a.work(), configFile), "editor: nano\npager: less\n") + writeTestFile(t, filepath.Join(b.work(), configFile), "editor: vi\npager: cat\n") + require.NoError(t, a.finish(context.Background())) + // Simulate a killed wrapper releasing its ownership lock without cleanup. + require.NoError(t, b.lock.Close()) + c, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + got, err := os.ReadFile(filepath.Join(c.work(), configFile)) + require.NoError(t, err) + n, err := decode(got) + require.NoError(t, err) + require.Equal(t, "nano", get(n, "editor").Value) + require.Equal(t, "cat", get(n, "pager").Value) + require.NoError(t, c.finish(context.Background())) +} + +func TestStripForwardedCredentialsAndPreserveLocalLogin(t *testing.T) { + a := Account{Host: "github.example", User: "relay-user", Token: "dummy-forwarded-value"} + s := &session{Accounts: []Account{a}} + for _, tc := range []struct { + name, input string + local, keyring bool + }{ + {"unchanged", `github.example: + user: relay-user + oauth_token: dummy-forwarded-value + users: + relay-user: + oauth_token: dummy-forwarded-value +`, false, false}, + {"local login", `github.example: + user: relay-user + oauth_token: dummy-local-value + users: + relay-user: + oauth_token: dummy-local-value +`, true, false}, + {"keyring login", "github.example:\n user: relay-user\n users:\n relay-user: {}\n", true, true}, + } { + t.Run(tc.name, func(t *testing.T) { + got, err := s.strip([]byte(tc.input)) + require.NoError(t, err) + // Do not include the produced configuration in assertion failures. + n, err := decode(got) + require.NoError(t, err) + h := get(n, a.Host) + require.Equal(t, tc.local, h != nil) + if tc.local { + require.NotNil(t, get(h, "user")) + } + if tc.keyring { + require.Nil(t, get(h, "oauth_token")) + } + }) + } +} + +func TestMergeAddedMappingsAndDeletions(t *testing.T) { + got, err := mergeYAML( + []byte("aliases: {}\neditor: vi\n"), + []byte("aliases:\n mine: repo view\n"), + []byte("aliases:\n other: issue list\neditor: vi\npager: cat\n"), + ) + require.NoError(t, err) + n, err := decode(got) + require.NoError(t, err) + require.Nil(t, get(n, "editor")) + require.Equal(t, "cat", get(n, "pager").Value) + require.NotNil(t, get(get(n, "aliases"), "mine")) + require.NotNil(t, get(get(n, "aliases"), "other")) +} + +func TestDanglingDotfilesSymlinkKeepsPendingChanges(t *testing.T) { + root := t.TempDir() + t.Setenv("HOME", root) + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + original, target := filepath.Join(root, "gh"), filepath.Join(root, "dotfiles.yml") + writeTestFile(t, target, "editor: vi\n") + require.NoError(t, os.MkdirAll(original, 0o700)) + link := filepath.Join(original, configFile) + require.NoError(t, os.Symlink(target, link)) + s, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + writeTestFile(t, filepath.Join(s.work(), configFile), "editor: nano\n") + require.NoError(t, os.Remove(target)) + require.Error(t, s.finish(context.Background())) + actual, err := os.Readlink(link) + require.NoError(t, err) + require.Equal(t, target, actual) + writeTestFile(t, target, "editor: emacs\npager: cat\n") + c, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + data, err := readFile(filepath.Join(c.work(), configFile)) + require.NoError(t, err) + n, err := decode(data) + require.NoError(t, err) + require.Equal(t, "nano", get(n, "editor").Value) + require.Equal(t, "cat", get(n, "pager").Value) + require.NoError(t, c.finish(context.Background())) +} + +func TestDisabledWritebackDiagnosticsDoNotParseConfiguration(t *testing.T) { + var summary *writebackSummary + // Invalid YAML would fail diagnostic parsing if the disabled sink were entered. + require.NoError(t, summary.inspect(configFile, []byte("["), []byte("["), []byte("["))) +} diff --git a/pkg/ghcredentials/testdata/Dockerfile b/pkg/ghcredentials/testdata/Dockerfile new file mode 100644 index 0000000000..9229a5a1b1 --- /dev/null +++ b/pkg/ghcredentials/testdata/Dockerfile @@ -0,0 +1,3 @@ +FROM mcr.microsoft.com/devcontainers/base:ubuntu +USER root +RUN apt-get update && apt-get install -y --no-install-recommends python3 dbus gnome-keyring openssl ca-certificates && rm -rf /var/lib/apt/lists/* diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py new file mode 100644 index 0000000000..265b21e3f5 --- /dev/null +++ b/pkg/ghcredentials/testdata/fixture.py @@ -0,0 +1,294 @@ +"""Isolated real-gh oracle. Never print captured auth output or token values.""" +import http.server +import json +import os +from pathlib import Path +import signal +import ssl +import subprocess +import sys +import time + +ROOT = Path('/fixture') +HOSTS = ['github.com', 'enterprise.one.test', 'enterprise.two.test'] +secrets = json.loads((ROOT / 'accounts.json').read_text()) + +def server(): + subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-keyout', '/tmp/fixture-key.pem', '-out', '/tmp/fixture-cert.pem', '-days', '1', '-subj', '/CN=github.com', '-addext', 'subjectAltName=DNS:github.com,DNS:api.github.com,DNS:enterprise.one.test,DNS:enterprise.two.test'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) + tls = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + tls.load_cert_chain('/tmp/fixture-cert.pem', '/tmp/fixture-key.pem') + tokens = {} + for record in secrets.values(): + tokens[record['forwarded']] = 'host-user' + tokens[record['local']] = 'local-user' + tokens[record['second']] = 'second-user' + tokens[record['refresh']] = 'host-user' + + class API(http.server.BaseHTTPRequestHandler): + def log_message(self, *args): + pass + def handle_api(self): + if self.command == 'POST': + self.rfile.read(int(self.headers.get('Content-Length', 0))) + token = self.headers.get('Authorization', '').split(' ')[-1] + user = tokens.get(token) + body = {'login': user, 'id': 123, 'data': {'viewer': {'login': user}}} if user else {'message': 'Bad credentials'} + if self.path == '/login/device/code': + user = 'host-user' + body = {'device_code': 'fixture-device', 'user_code': 'fixture-code', 'verification_uri': 'https://github.com/login/device', 'expires_in': 600, 'interval': 1} + elif self.path == '/login/oauth/access_token': + user = 'host-user' + body = {'access_token': secrets[HOSTS[0]]['refresh'], 'token_type': 'bearer', 'scope': 'repo,read:org,gist'} + data = json.dumps(body).encode() + self.send_response(200 if user else 401) + self.send_header('X-OAuth-Scopes', 'repo, read:org, gist') + self.send_header('Content-Type', 'application/json') + self.send_header('Content-Length', str(len(data))) + self.end_headers() + self.wfile.write(data) + do_GET = handle_api + do_POST = handle_api + + class Proxy(http.server.BaseHTTPRequestHandler): + def log_message(self, *args): + pass + def do_CONNECT(self): + if self.path not in [h + ':443' for h in HOSTS + ['api.github.com']]: + self.send_error(403) + return + self.send_response(200) + self.end_headers() + self.wfile.flush() + try: + with tls.wrap_socket(self.connection, server_side=True) as conn: + API(conn, self.client_address, self.server) + except (ssl.SSLError, ConnectionError): + pass + srv = http.server.ThreadingHTTPServer(('127.0.0.1', 8123), Proxy) + (ROOT / 'api-ready').touch() + srv.serve_forever() + + +def check(): + os.environ.update({ + 'HTTPS_PROXY': 'http://127.0.0.1:8123', 'HTTP_PROXY': 'http://127.0.0.1:8123', + 'NO_PROXY': 'localhost,127.0.0.1', 'SSL_CERT_FILE': '/tmp/fixture-cert.pem', + 'GH_CONFIG_DIR': '/fixture/guest-gh', 'XDG_DATA_HOME': '/fixture/guest-data', + 'XDG_CACHE_HOME': '/fixture/guest-cache', 'GH_PROMPT_DISABLED': '1', + 'DBUS_SESSION_BUS_ADDRESS': (ROOT / 'bus').read_text().strip(), + }) + results = [] + def verify(name, ok): + results.append({'check': name, 'ok': bool(ok)}) + if not ok: + print(json.dumps(results), flush=True) + raise SystemExit(1) + def gh(*args, token=None, env=None): + e = dict(os.environ) + if env: + e.update(env) + return subprocess.run(['/bin/bash', '-lc', 'exec gh "$@"', '--', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + def api(host, expected='host-user', **kw): + r = gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) + ok = r.returncode == 0 and r.stdout.strip() == expected + return ok + original = ROOT / 'guest-gh' + original.mkdir(exist_ok=True) + def saved(): + return (original / 'hosts.yml').read_text() if (original / 'hosts.yml').exists() else '' + def no_forwarded_saved(): + return all(v['forwarded'] not in saved() for v in secrets.values()) + def last_diagnostic(): + folder = ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics' + return json.loads(max(folder.glob('*.json'), key=lambda f: f.stat().st_mtime_ns).read_text()) + + + verify('all hosts forwarded through real RPC and HTTP relay', all(api(h) for h in HOSTS)) + verify('real auth status', gh('auth', 'status').returncode == 0) + verify('forwarding never saved', no_forwarded_saved()) + verify('unchanged authentication run recorded without user delta', not last_diagnostic()['writeback'].get('changes')) + r = gh('api', 'user', '--jq', '.login', env={'GH_HOST': HOSTS[2]}) + verify('gh chooses GH_HOST', r.returncode == 0 and r.stdout.strip() == 'host-user') + verify('explicit invalid GH_TOKEN never falls back', gh('api', 'user', env={'GH_TOKEN': secrets[HOSTS[0]]['invalid']}).returncode != 0) + verify('explicit invalid GITHUB_TOKEN never falls back', gh('api', 'user', env={'GITHUB_TOKEN': secrets[HOSTS[0]]['invalid']}).returncode != 0) + verify('explicit enterprise environment never falls back', gh('api', 'user', '--hostname', HOSTS[1], env={'GH_ENTERPRISE_TOKEN': secrets[HOSTS[1]]['invalid']}).returncode != 0) + r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['forwarded']) + verify('same token login stays forwarded', r.returncode == 0 and no_forwarded_saved() and api(HOSTS[0])) + verify('auth switch uses forwarded account', gh('auth', 'switch', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and no_forwarded_saved()) + r = gh('auth', 'refresh', '--hostname', HOSTS[0], '--insecure-storage', env={'GH_BROWSER': '/bin/true'}) + verify('refresh saves a newly issued credential', r.returncode == 0 and secrets[HOSTS[0]]['refresh'] in saved() and api(HOSTS[0])) + verify('logout refreshed credential', gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and api(HOSTS[0])) + r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) + verify('different token local login persists', r.returncode == 0 and secrets[HOSTS[0]]['local'] in saved() and api(HOSTS[0], 'local-user')) + local_text = saved() + (original / 'hosts.yml').write_text(local_text.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) + verify('invalid local credential never falls back', not api(HOSTS[0])) + (original / 'hosts.yml').write_text(local_text) + r = gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user') + verify('logout restores forwarding on next invocation', r.returncode == 0 and api(HOSTS[0]) and no_forwarded_saved()) + + r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['local']) + verify('keyring login succeeds without plaintext token', r.returncode == 0 and secrets[HOSTS[1]]['local'] not in saved() and api(HOSTS[1], 'local-user')) + r = gh('auth', 'status', '--hostname', HOSTS[1]) + verify('real auth status identifies keyring', r.returncode == 0 and 'keyring' in r.stdout + r.stderr) + verify('keyring logout restores forwarding', gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'local-user').returncode == 0 and api(HOSTS[1])) + + r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['forwarded']) + verify('same token explicit keyring login retained', r.returncode == 0 and 'host-user' in saved() and no_forwarded_saved() and api(HOSTS[1])) + verify('same token keyring logout restores forwarding', gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'host-user').returncode == 0 and api(HOSTS[1])) + + verify('alias saves', gh('alias', 'set', 'save-editor', '!gh config set editor nano').returncode == 0) + verify('alias nested configuration persists', gh('save-editor').returncode == 0 and gh('config', 'get', 'editor').stdout.strip() == 'nano') + ext = ROOT / 'guest-data' / 'gh' / 'extensions' / 'gh-probe' + ext.mkdir(parents=True) + (ext / 'gh-probe').write_text('#!/bin/sh\ngh config set pager cat\ngh api user --hostname enterprise.two.test --jq .login\nprintf "extension-out\\n"\nprintf "extension-err\\n" >&2\nexit 7\n') + (ext / 'gh-probe').chmod(0o755) + r = gh('probe') + verify('extension streams and failure exit pass unchanged', r.returncode == 7 and r.stdout == 'host-user\nextension-out\n' and r.stderr == 'extension-err\n') + verify('extension change persists after failure', gh('config', 'get', 'pager').stdout.strip() == 'cat') + + # Symlinked config and dotfiles replacement between snapshot and writeback. + config = original / 'config.yml' + target = ROOT / 'dotfiles-config.yml' + config.rename(target) + config.symlink_to(target) + ready, release = ROOT / 'command-ready', ROOT / 'command-release' + verify('concurrent alias saves', gh('alias', 'set', 'pending-editor', '!gh config set editor pending; touch "$GH_TEST_READY"; while test ! -f "$GH_TEST_RELEASE"; do sleep 0.05; done').returncode == 0) + def pending(): + e = dict(os.environ, GH_TEST_READY=str(ready), GH_TEST_RELEASE=str(release)) + return subprocess.Popen(['/bin/bash', '-lc', 'exec gh pending-editor'], env=e, stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True) + def wait_ready(): + for _ in range(200): + if ready.exists(): + return True + time.sleep(0.05) + return False + p = pending() + try: + verify('concurrent invocation reached user mutation', wait_ready()) + verify('parallel gh saves unrelated pager', gh('config', 'set', 'pager', 'less').returncode == 0) + target.write_text(target.read_text().replace('editor: nano', 'editor: emacs') + '\nprompt: disabled\n') + release.touch() + p.communicate(timeout=20) + changes = last_diagnostic()['writeback']['changes'] + verify('successful current-wins merge recorded', changes.get('config.editor', {}).get('current_wins') == 1 and last_diagnostic()['writeback']['written_files'] > 0) + verify('parallel and dotfiles changes survive', p.returncode == 0 and gh('config', 'get', 'editor').stdout.strip() == 'pending' and gh('config', 'get', 'pager').stdout.strip() == 'less' and gh('config', 'get', 'prompt').stdout.strip() == 'disabled' and config.is_symlink()) + finally: + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + ready.unlink() + release.unlink() + verify('prepare interrupted mutation', gh('config', 'set', 'editor', 'vi').returncode == 0) + p = pending() + try: + verify('interrupted invocation reached mutation', wait_ready()) + finally: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + verify('next invocation recovers killed wrapper mutation', gh('config', 'get', 'editor').stdout.strip() == 'pending') + verify('recovered session contains no forwarded credentials', no_forwarded_saved()) + + # A writeback failure must preserve gh's exit status and retain the changes. + ready.unlink() + verify('prepare pending writeback', gh('config', 'set', 'editor', 'vi').returncode == 0) + p = pending() + latest = None + try: + verify('writeback failure reached user mutation', wait_ready()) + latest = target.read_text() + target.write_text('[broken yaml') + release.touch() + p.communicate(timeout=20) + verify('writeback failure does not change real gh exit', p.returncode == 0) + diagnostic = last_diagnostic() + verify('writeback schema failure retained in diagnostics', diagnostic['stage'] == 'writeback_pending' and any(op['name'] == 'config.merge' and op.get('error_type') == 'schema_invalid' for op in diagnostic['operations'])) + finally: + if latest is not None: + target.write_text(latest) + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + verify('pending writeback recovered after original repaired', gh('config', 'get', 'editor').stdout.strip() == 'pending') + + + # Killing only the wrapper must not reap the still-running real gh. + ready.unlink() + release.unlink() + verify('prepare live-child ownership', gh('config', 'set', 'editor', 'vi').returncode == 0) + p = pending() + try: + verify('live child reached mutation', wait_ready()) + os.kill(p.pid, signal.SIGKILL) + r = gh('config', 'get', 'editor') + verify('live child prevents premature recovery', r.returncode == 0 and r.stdout.strip() == 'vi' and bool(list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*')))) + release.touch() + p.communicate(timeout=20) + verify('completed orphan child changes recover', gh('config', 'get', 'editor').stdout.strip() == 'pending') + finally: + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + + verify('host auth unchanged by guest operations', all(v['forwarded'] in (ROOT / 'host-gh' / 'hosts.yml').read_text() for v in secrets.values())) + + # A host active-account change must be observed on the very next invocation. + hp = ROOT / 'host-gh' / 'hosts.yml' + + host_before = hp.read_text() + try: + hp.write_text('{}\n') + r = gh('config', 'get', 'editor') + first = last_diagnostic() + hp.write_text('[broken yaml') + other = gh('config', 'get', 'editor') + second = last_diagnostic() + verify('same exit distinguishes host authentication failures', r.returncode == other.returncode == 0 and first['host_status'] == 'no_active_accounts' and second['host_status'] == 'config_decode_failed') + finally: + hp.write_text(host_before) + text = hp.read_text().replace('host-user', 'second-user') + for v in secrets.values(): + text = text.replace(v['forwarded'], v['second']) + hp.write_text(text) + verify('host active account followed', api(HOSTS[2], 'second-user')) + r = subprocess.run(['runuser', '-u', 'vscode', '--', 'env', 'HOME=/home/vscode', 'GH_CONFIG_DIR=', 'XDG_CONFIG_HOME=', 'XDG_CACHE_HOME=/home/vscode/.cache', '/bin/bash', '-lc', 'exec gh api user --hostname enterprise.two.test --jq .login'], env=dict(os.environ, DBUS_SESSION_BUS_ADDRESS='unix:path=/nonexistent'), text=True, capture_output=True, timeout=20) + verify('non-root target user inherits wrapper and uses own HOME', r.returncode == 0 and r.stdout.strip() == 'second-user') + + verify('local config dir override', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '/fixture/custom-gh'}).returncode == 0 and (ROOT / 'custom-gh' / 'config.yml').exists()) + verify('XDG config location', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '/fixture/custom-xdg'}).returncode == 0 and (ROOT / 'custom-xdg' / 'gh' / 'config.yml').exists()) + verify('HOME config location', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '', 'HOME': '/fixture/custom-home'}).returncode == 0 and (ROOT / 'custom-home' / '.config' / 'gh' / 'config.yml').exists()) + records = list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics').glob('*.json')) + verify('bounded local diagnostics', 0 < len(records) <= 64) + verify('diagnostics contain no credential values', all(all(secret not in f.read_text() for values in secrets.values() for secret in values.values()) for f in records)) + before = {f.name for f in records} + r = gh('config', 'set', 'browser', '/bin/true', env={'DEVSY_GH_DIAGNOSTICS': 'false'}) + verify('disabled diagnostics preserve configuration writes', r.returncode == 0 and gh('config', 'get', 'browser', env={'DEVSY_GH_DIAGNOSTICS': 'false'}).stdout.strip() == '/bin/true') + verify('diagnostics opt out', api(HOSTS[2], 'second-user', env={'DEVSY_GH_DIAGNOSTICS': 'false'}) and before == {f.name for f in records[0].parent.glob('*.json')}) + + diagnostic_dir = ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics' + held = diagnostic_dir.with_name('held-diagnostics') + diagnostic_dir.rename(held) + diagnostic_dir.symlink_to('/proc/1') + try: + verify('recording failure does not change authentication', api(HOSTS[2], 'second-user')) + finally: + diagnostic_dir.unlink() + held.rename(diagnostic_dir) + verify('normal sessions cleaned', not list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*'))) + + # Leave one interrupted change for the caller to recover after stopping RPC. + ready.unlink() + release.unlink() + verify('prepare relay-loss recovery', gh('config', 'set', 'editor', 'vi').returncode == 0) + p = pending() + try: + verify('relay-loss mutation reached', wait_ready()) + finally: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + print(json.dumps(results), flush=True) + +if sys.argv[1] == 'server': + server() +else: + check() diff --git a/pkg/ghcredentials/wrapper.go b/pkg/ghcredentials/wrapper.go new file mode 100644 index 0000000000..361aa9790d --- /dev/null +++ b/pkg/ghcredentials/wrapper.go @@ -0,0 +1,236 @@ +package ghcredentials + +import ( + "context" + "encoding/json" + "io" + "net" + "net/http" + "os" + "os/exec" + "path/filepath" + "strconv" + "strings" + "time" +) + +const sessionEnv = "DEVSY_GH_SESSION" + +// Run is the wrapper entry point. args starts with the wrapper's path, followed +// by untouched gh arguments. It bypasses Devsy's CLI/env/log initialization. +func Run(args []string) int { + if len(args) == 0 { + return 127 + } + binary, err := realBinary(args[0]) + if err != nil { + return 127 + } + env := os.Environ() + if active := os.Getenv(sessionEnv); active != "" && active == os.Getenv("GH_CONFIG_DIR") { + return runGH(binary, args[1:], env, nil).Code + } + inv := &invocation{binary: binary, args: args[1:], env: env, diag: beginDiagnostic(binary)} + defer inv.diag.finish() + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + inv.recover(ctx) + session := inv.prepare(ctx) + if session == nil { + return inv.diag.run(binary, inv.args, env, nil) + } + return inv.withSession(session) +} + +type invocation struct { + binary string + args, env []string + diag *diagnostic +} + +func (inv *invocation) recover(ctx context.Context) { + started := time.Now() + root, err := sessionRoot() + if err != nil { + inv.diag.observe("session.recover", started, err) + return + } + l, err := lock(ctx, filepath.Join(root, "sessions.lock")) + if err != nil { + inv.diag.observe("session.recover_lock", started, err) + return + } + defer func() { _ = l.Close() }() + inv.diag.Recovery = recoverSessions(ctx, root, inv.diag.path != "") + inv.diag.observe("session.recover", started, nil) +} + +func (inv *invocation) prepare(ctx context.Context) *session { + started := time.Now() + result, err := fetch(ctx) + inv.diag.observe("relay.fetch", started, err) + inv.diag.HostStatus = result.Status + inv.diag.HostContext = result.Context + inv.diag.hostResult(result) + inv.diag.HostDecisions = result.Decisions + if err != nil || len(result.Accounts) == 0 { + inv.diag.Stage = "relay_unavailable" + return nil + } + original, err := ConfigDir() + if err != nil { + inv.diag.Stage = "config_unavailable" + return nil + } + inv.diag.ConfigDir = original + started = time.Now() + s, err := startSession(ctx, original, result.Accounts) + inv.diag.observe("session.snapshot", started, err) + if err != nil { + inv.diag.Stage = "snapshot_failed" + return nil + } + started = time.Now() + if err := s.inject(ctx, inv.binary, inv.env); err != nil { + inv.diag.observe("config.overlay", started, err) + _ = os.RemoveAll(s.dir) + _ = s.lock.Close() + inv.diag.Stage = "overlay_failed" + return nil + } + inv.diag.observe("config.overlay", started, nil) + inv.diag.Hosts = s.decisions + return s +} + +func (inv *invocation) withSession(s *session) int { + inv.diag.Stage = "overlay_ready" + inv.diag.save(false) + env := replaceEnv(inv.env, "GH_CONFIG_DIR", s.work()) + env = replaceEnv(env, sessionEnv, s.work()) + code := inv.diag.run(inv.binary, inv.args, env, s.lock.file) + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + started := time.Now() + if inv.diag.path != "" { + s.writeback = &writebackSummary{} + } + err := s.finish(ctx) + inv.diag.Writeback = s.writeback + inv.diag.observe("config.writeback", started, err) + if err != nil { + inv.diag.Stage = "writeback_pending" + } else { + inv.diag.Stage = "written_back" + } + return code +} + +func realBinary(wrapper string) (string, error) { + self, _ := os.Stat(wrapper) + for _, dir := range filepath.SplitList(os.Getenv("PATH")) { + path := filepath.Join(dir, "gh") + info, err := executableInfo(path) + if err != nil { + continue + } + if self != nil && os.SameFile(self, info) { + continue + } + return filepath.Abs(path) + } + return "", exec.ErrNotFound +} + +func replaceEnv(env []string, name, value string) []string { + out := make([]string, 0, len(env)+1) + for _, e := range env { + if !strings.HasPrefix(e, name+"=") { + out = append(out, e) + } + } + return append(out, name+"="+value) +} + +func fetch(ctx context.Context) (HostResult, error) { + number, err := relayPort() + if err != nil { + return HostResult{}, err + } + // A loopback-only transport must not send tokens or relay requests through + // a user-configured HTTP proxy, redirect, or an external endpoint. + client := &http.Client{ + Transport: &http.Transport{ + Proxy: nil, + DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) { + dialer := net.Dialer{} + return dialer.DialContext( + ctx, + "tcp", + net.JoinHostPort("127.0.0.1", strconv.Itoa(number)), + ) + }, + }, + Timeout: 5 * time.Second, + CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }, + } + defer client.CloseIdleConnections() + req, err := http.NewRequestWithContext( + ctx, + http.MethodPost, + "http://localhost/github-credentials", + nil, + ) + if err != nil { + return HostResult{}, boundary("relay.http", err) + } + resp, err := client.Do(req) + if err != nil { + return HostResult{}, boundary("relay.http", err) + } + defer func() { _ = resp.Body.Close() }() + if resp.StatusCode != http.StatusOK { + return HostResult{}, &boundaryError{ + operation: "relay.http", + kind: "http_" + strconv.Itoa(resp.StatusCode), + } + } + var result HostResult + err = json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(&result) + if err != nil { + return HostResult{}, &boundaryError{ + operation: "relay.decode", + kind: "schema_invalid", + cause: err, + } + } + return result, nil +} + +func executableInfo(path string) (os.FileInfo, error) { + f, err := resolvedFile(path) + if err != nil { + return nil, err + } + defer func() { _ = f.Close() }() + info, err := f.Stat() + if err != nil { + return nil, err + } + if !info.Mode().IsRegular() || info.Mode()&0o111 == 0 { + return nil, exec.ErrNotFound + } + return info, nil +} + +func relayPort() (int, error) { + port := os.Getenv("DEVSY_CREDENTIALS_SERVER_PORT") + if port == "" { + port = "12049" + } + number, err := strconv.Atoi(port) + if err != nil || number < 1 || number > 65535 { + return 0, &boundaryError{operation: "relay.port", kind: "config_invalid"} + } + return number, nil +} diff --git a/pkg/ghcredentials/wrapper_integration_test.go b/pkg/ghcredentials/wrapper_integration_test.go new file mode 100644 index 0000000000..d9d7233154 --- /dev/null +++ b/pkg/ghcredentials/wrapper_integration_test.go @@ -0,0 +1,308 @@ +//go:build ghrelay + +package ghcredentials_test + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "context" + "crypto/rand" + "crypto/sha256" + "embed" + "encoding/hex" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "testing" + "time" + + "github.com/devsy-org/devsy/pkg/agent/tunnelserver" + "github.com/stretchr/testify/require" +) + +//go:embed testdata/Dockerfile testdata/fixture.py +var fixtureFiles embed.FS + +// Run with task cli:test:gh-relay. All authentication is fictional, API traffic +// stays in a network-disabled container, and the host gh reads a disposable config. +func TestRealGitHubCLI(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 8*time.Minute) + defer cancel() + dir := t.TempDir() + name := prepareFixture(t, ctx, dir) + prepareHostAuthentication(t, dir) + stop, done := startFixture(t, ctx, dir, name) + defer stop() + waitFixture(t, ctx, name, done) + // stdout contains only boolean test results. Raw gh streams and credentials + // stay inside the disposable fixture and are never attached to failures. + //nolint:gosec // name is the generated test container. + check := exec.CommandContext( + ctx, + "docker", + "exec", + name, + "python3", + "/fixture/fixture.py", + "check", + ) + report, err := check.CombinedOutput() + require.NoError(t, err, "real gh checks: %s", report) + verifyRealGHReport(t, report) + + verifyRelayFallback(t, ctx, name, stop) +} + +func downloadGH(t *testing.T, ctx context.Context, dir string) { + t.Helper() + const version = "2.100.0" + filename := "gh_" + version + "_linux_" + runtime.GOARCH + ".tar.gz" + get := func(name string) []byte { + req, err := http.NewRequestWithContext( + ctx, + http.MethodGet, + "https://github.com/cli/cli/releases/download/v"+version+"/"+name, + nil, + ) + require.NoError(t, err) + response, err := http.DefaultClient.Do(req) + require.NoError(t, err) + defer func() { _ = response.Body.Close() }() + require.Equal(t, http.StatusOK, response.StatusCode) + b, err := io.ReadAll(io.LimitReader(response.Body, 64<<20)) + require.NoError(t, err) + return b + } + archive := get(filename) + sums := string(get("gh_" + version + "_checksums.txt")) + hash := sha256.Sum256(archive) + require.Contains(t, sums, fmt.Sprintf("%x %s", hash, filename)) + z, err := gzip.NewReader(bytes.NewReader(archive)) + require.NoError(t, err) + defer func() { _ = z.Close() }() + tarReader := tar.NewReader(z) + for { + header, err := tarReader.Next() + require.NoError(t, err) + if strings.HasSuffix(header.Name, "/bin/gh") { + binary, err := io.ReadAll(io.LimitReader(tarReader, 128<<20)) + require.NoError(t, err) + // #nosec G306 -- Executable release binary, verified against its published checksum. + require.NoError(t, os.WriteFile(filepath.Join(dir, "gh"), binary, 0o755)) + return + } + } +} + +func verifyRealGHReport(t *testing.T, report []byte) { + t.Helper() + var results []struct { + Check string `json:"check"` + OK bool `json:"ok"` + } + require.NoError(t, json.Unmarshal(report, &results)) + for _, r := range results { + require.True(t, r.OK, r.Check) + t.Log(r.Check) + } +} + +func prepareFixture(t *testing.T, ctx context.Context, dir string) string { + t.Helper() + repo, err := filepath.Abs("../..") + require.NoError(t, err) + for _, name := range []string{"Dockerfile", "fixture.py"} { + b, err := fixtureFiles.ReadFile("testdata/" + name) + require.NoError(t, err) + root, err := os.OpenRoot(dir) + require.NoError(t, err) + require.NoError(t, root.WriteFile(name, b, 0o600)) + require.NoError(t, root.Close()) + } + random := make([]byte, 12) + _, err = rand.Read(random) + require.NoError(t, err) + name := "devsy-gh-test-" + hex.EncodeToString(random) + image := name + ":test" + //nolint:gosec // image and directory are generated fixture resources. + buildImage := exec.CommandContext(ctx, "docker", "build", "-q", "-t", image, dir) + imageOut, err := buildImage.CombinedOutput() + require.NoError(t, err, "fixture image: %s", imageOut) + t.Cleanup(func() { + //nolint:gosec // image is the random, test-owned tag. + _ = exec.Command("docker", "image", "rm", image).Run() + }) + //nolint:gosec // The output is inside t.TempDir; the source is this repository. + build := exec.CommandContext(ctx, "go", "build", "-o", filepath.Join(dir, "devsy"), ".") + build.Dir = repo + build.Env = append(os.Environ(), "GOOS=linux", "GOARCH="+runtime.GOARCH, "CGO_ENABLED=0") + out, err := build.CombinedOutput() + require.NoError(t, err, "build: %s", out) + downloadGH(t, ctx, dir) + return name +} + +func prepareHostAuthentication(t *testing.T, dir string) { + t.Helper() + type tokens struct { + Forwarded string `json:"forwarded"` + Local string `json:"local"` + Second string `json:"second"` + Invalid string `json:"invalid"` + Refresh string `json:"refresh"` + } + accounts := map[string]tokens{} + var hosts strings.Builder + for _, host := range []string{"github.com", "enterprise.one.test", "enterprise.two.test"} { + token := func() string { + b := make([]byte, 20) + _, err := rand.Read(b) + require.NoError(t, err) + return "ghp_" + hex.EncodeToString(b) + } + v := tokens{ + Forwarded: token(), + Local: token(), + Second: token(), + Invalid: token(), + Refresh: token(), + } + accounts[host] = v + fmt.Fprintf( + &hosts, + "%s:\n user: host-user\n oauth_token: %s\n users:\n host-user:\n oauth_token: %s\n", + host, + v.Forwarded, + v.Forwarded, + ) + } + require.NoError(t, os.MkdirAll(filepath.Join(dir, "host-gh"), 0o700)) + require.NoError( + t, + os.WriteFile(filepath.Join(dir, "host-gh", "hosts.yml"), []byte(hosts.String()), 0o600), + ) + data, err := json.Marshal(accounts) + require.NoError(t, err) + require.NoError(t, os.WriteFile(filepath.Join(dir, "accounts.json"), data, 0o600)) + for _, key := range []string{"GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST"} { + t.Setenv(key, "") + } + t.Setenv("GH_CONFIG_DIR", filepath.Join(dir, "host-gh")) + t.Setenv("GIT_CONFIG_GLOBAL", filepath.Join(dir, "no-gitconfig")) + t.Setenv("GIT_CONFIG_NOSYSTEM", "1") +} + +func startFixture( + t *testing.T, + ctx context.Context, + dir, name string, +) (context.CancelFunc, <-chan error) { + t.Helper() + image := name + ":test" + command := `cp /fixture/gh /usr/bin/gh +cp /fixture/devsy /usr/local/bin/devsy +mkdir -p /root/.local/share/keyrings +export PATH=/fixture:$PATH +exec dbus-run-session -- sh -c ' + printf "%s" "$DBUS_SESSION_BUS_ADDRESS" > /fixture/bus + printf fixture-password | gnome-keyring-daemon --unlock >/dev/null 2>/tmp/keyring.log + python3 /fixture/fixture.py server >/dev/null 2>/tmp/api.log & + /usr/local/bin/devsy internal agent container credentials-server --user root --configure-github-helper + sleep 600 +'` + //nolint:gosec // All mounts, commands, and image names belong to this disposable fixture. + guest := exec.CommandContext( + ctx, + "docker", + "run", + "--rm", + "--name", + name, + "--network", + "none", + "-i", + "-v", + dir+":/fixture", + image, + "sh", + "-c", + command, + ) + reader, err := guest.StdoutPipe() + require.NoError(t, err) + writer, err := guest.StdinPipe() + require.NoError(t, err) + require.NoError(t, guest.Start()) + t.Cleanup(func() { + //nolint:gosec // name is the random, test-owned container name. + _ = exec.Command("docker", "rm", "-f", name).Run() + _ = guest.Wait() + }) + tunnelCtx, tunnelCancel := context.WithCancel(ctx) + done := make(chan error, 1) + go func() { + done <- tunnelserver.RunServicesServer(tunnelCtx, reader, writer, false, false, nil, nil, + tunnelserver.WithAllowGitHubCredentials(true)) + }() + + return tunnelCancel, done +} + +func waitFixture(t *testing.T, ctx context.Context, name string, done <-chan error) { + t.Helper() + ready := false + for range 150 { + //nolint:gosec // name is the generated test container. + c := exec.CommandContext( + ctx, + "docker", + "exec", + name, + "sh", + "-c", + "test -f /fixture/api-ready && test -f /usr/local/share/devsy/gh/bin/gh", + ) + if c.Run() == nil { + ready = true + break + } + select { + case err := <-done: + t.Fatalf("relay ended before readiness: %v", err) + case <-time.After(100 * time.Millisecond): + } + } + require.True(t, ready, "fixture was not ready") +} + +func verifyRelayFallback(t *testing.T, ctx context.Context, name string, stop context.CancelFunc) { + t.Helper() + // A transport failure must execute normal gh once, without relayed auth. + stop() + //nolint:gosec // name is the test-owned container; only its relay process is stopped. + _ = exec.CommandContext(ctx, "docker", "exec", name, "sh", "-c", "pkill -x devsy || true"). + Run() + //nolint:gosec // name is the test-owned container, with an isolated configuration. + fallback := exec.CommandContext( + ctx, + "docker", + "exec", + name, + "bash", + "-lc", + "export XDG_CACHE_HOME=/fixture/guest-cache; "+ + "GH_CONFIG_DIR=/fixture/guest-gh gh config get editor && "+ + "GH_CONFIG_DIR=/fixture/fallback-gh gh config set editor nano && "+ + "GH_CONFIG_DIR=/fixture/fallback-gh gh config get editor", + ) + plain, err := fallback.CombinedOutput() + require.NoError(t, err, "fallback: %s", plain) + require.Equal(t, "pending\nnano\n", string(plain)) +} diff --git a/pkg/ghcredentials/writeback.go b/pkg/ghcredentials/writeback.go new file mode 100644 index 0000000000..9ffc224559 --- /dev/null +++ b/pkg/ghcredentials/writeback.go @@ -0,0 +1,125 @@ +package ghcredentials + +import "gopkg.in/yaml.v3" + +// Categories are a fixed allowlist: extension paths, aliases, account names, +// arbitrary configuration keys and values must never enter diagnostics. +type writebackSummary struct { + UnchangedFiles int `json:"unchanged_files"` + WrittenFiles int `json:"written_files"` + Changes map[string]changeCount `json:"changes,omitempty"` +} + +func (s *writebackSummary) unchanged() { + if s != nil { + s.UnchangedFiles++ + } +} + +func (s *writebackSummary) written(err error) { + if s != nil && err == nil { + s.WrittenFiles++ + } +} + +type changeCount struct { + Added int `json:"added"` + Modified int `json:"modified"` + Deleted int `json:"deleted"` + Conflicts int `json:"current_wins"` + AlreadyApplied int `json:"already_applied"` +} + +func (s *writebackSummary) inspect(name string, before, after, latest []byte) error { + if s == nil { + return nil + } + if name != configFile && name != hostsFile { + s.record("other_files", scalarBytes(before), scalarBytes(after), scalarBytes(latest)) + return nil + } + nodes, err := decodeConfigs(before, after, latest) + if err != nil { + return err + } + b, a, l := nodes[0], nodes[1], nodes[2] + if equal(b, a) { + s.UnchangedFiles++ + return nil + } + for key := range mappingKeys(b, a) { + s.inspectNodes(changeCategory(name, key), get(b, key), get(a, key), get(l, key)) + } + return nil +} + +func decodeConfigs(configs ...[]byte) ([]*yaml.Node, error) { + nodes := make([]*yaml.Node, 0, len(configs)) + for _, data := range configs { + n, err := decode(data) + if err != nil { + return nil, err + } + nodes = append(nodes, n) + } + return nodes, nil +} + +func changeCategory(name, key string) string { + if name == hostsFile { + return "hosts" + } + switch key { + case "editor", "pager", "aliases", "git_protocol", "prompt", "browser": + return "config." + key + default: + return "config.other" + } +} + +func scalarBytes(b []byte) *yaml.Node { + if b == nil { + return nil + } + return scalar(string(b)) +} + +func (s *writebackSummary) inspectNodes(category string, before, after, latest *yaml.Node) { + if equal(before, after) { + return + } + if mappingCompatible(before, after) { + if before == nil { + before = mapping() + } + for key := range mappingKeys(before, after) { + s.inspectNodes(category, get(before, key), get(after, key), get(latest, key)) + } + return + } + s.record(category, before, after, latest) +} + +func (s *writebackSummary) record(category string, before, after, latest *yaml.Node) { + if equal(before, after) { + return + } + if s.Changes == nil { + s.Changes = map[string]changeCount{} + } + c := s.Changes[category] + switch { + case before == nil: + c.Added++ + case after == nil: + c.Deleted++ + default: + c.Modified++ + } + if equal(after, latest) { + c.AlreadyApplied++ + } else if !equal(before, latest) { + c.Conflicts++ + } + s.Changes[category] = c +} diff --git a/pkg/tunnel/browser.go b/pkg/tunnel/browser.go index 80a71fcfb2..1e502c7b62 100644 --- a/pkg/tunnel/browser.go +++ b/pkg/tunnel/browser.go @@ -110,6 +110,9 @@ func runBrowserTunnelServices( ConfigureDockerCredentials: p.DevsyConfig.ContextOption( config.ContextOptionSSHInjectDockerCredentials, ) == config.BoolTrue, + ConfigureGitHubCredentials: p.DevsyConfig.ContextOption( + config.ContextOptionGitHubCLIForwarding, + ) == config.BoolTrue, ConfigureGitCredentials: p.DevsyConfig.ContextOption( config.ContextOptionSSHInjectGitCredentials, ) == config.BoolTrue, diff --git a/pkg/tunnel/services.go b/pkg/tunnel/services.go index f611815304..f54ddec37a 100644 --- a/pkg/tunnel/services.go +++ b/pkg/tunnel/services.go @@ -50,6 +50,7 @@ type RunServicesOptions struct { Workspace *provider.Workspace ConfigureDockerCredentials bool ConfigureGitCredentials bool + ConfigureGitHubCredentials bool ConfigureGitSSHSignatureHelper bool GitSSHSigningKey string // ExtraListeners holds pre-bound listeners keyed by "host:port" so the @@ -112,6 +113,7 @@ func runTunnelServer(ctx context.Context, cancel context.CancelFunc, p tunnelSer p.forwarder, p.opts.Workspace, tunnelserver.WithPlatformOptions(p.opts.PlatformOptions), + tunnelserver.WithAllowGitHubCredentials(p.opts.ConfigureGitHubCredentials), ) if err != nil { p.errChan <- fmt.Errorf("run tunnel server: %w", err) @@ -156,6 +158,9 @@ func buildCredentialsCommand(ctx context.Context, opts RunServicesOptions) strin if opts.ConfigureGitCredentials { command += " --configure-git-helper" } + if opts.ConfigureGitHubCredentials { + command += " --configure-github-helper" + } if opts.ConfigureGitSSHSignatureHelper { workingDir := "" if opts.Workspace != nil { diff --git a/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx b/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx index d6a8472b42..32cfe5d7c2 100644 --- a/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx +++ b/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx @@ -3,7 +3,7 @@ title: Reuse local credentials sidebar_label: Reuse local credentials --- -Devsy makes some of your local credentials available inside the dev container, so you do not configure them in each workspace. It supports git credentials, Docker registry credentials, and GPG keys. +Devsy makes some of your local credentials available inside the dev container, so you do not configure them in each workspace. It supports Git credentials, GitHub CLI authentication, Docker registry credentials, and GPG keys. Code that runs in the container, including lifecycle commands and features from `devcontainer.json`, can use these credentials. Do not enable them for repositories or `devcontainer.json` files you have not reviewed. @@ -27,6 +27,22 @@ devsy context set default -o SSH_AGENT_FORWARDING=false Disabling HTTPS credential injection does not disable SSH agent forwarding. +## GitHub CLI + +With `gh` installed on the host and in the container, normal `gh` commands, aliases, and extensions can use the host's active account for each configured GitHub.com or GitHub Enterprise host. `gh` selects the target host. Explicit authentication environment variables and container-local authentication take precedence, even when invalid or expired. + +Container logins and settings changes are saved locally. Logging out of a local account makes host authentication available again on the next invocation; it does not log out the host. Re-saving an unchanged forwarded token in the configuration file does not create independent local authentication. An explicit login saved in the container’s keyring is retained. If host authentication is unavailable, `gh` runs with its normal local configuration. + +To disable GitHub CLI forwarding independently of Git credentials: + +```sh +devsy context set default -o SSH_INJECT_GITHUB_CREDENTIALS=false +``` + +The relay uses a private temporary configuration per invocation; nested commands share it. Unfinished changes are recovered on the next invocation after the previous `gh` stops, including when forwarding is unavailable. + +Recent credential-free diagnostic records are kept under the user's cache directory in `devsy/gh-relay/diagnostics` (at most 64 runs). Set `DEVSY_GH_DIAGNOSTICS=false` to disable recording, or delete that diagnostics directory to remove the records. Temporary authentication and pending changes live separately in `devsy/gh-relay/session-*`; deleting those directories discards pending changes. + ## Docker Devsy provides registry credentials through a [Docker credential helper](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers), so you can pull and push private images from the container. To turn it off for all workspaces: From deeab482b2a276a963835fd2520e388a94e03bb8 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 01:08:12 +0900 Subject: [PATCH 02/20] fix(credentials): forward gh auth for tokenless account settings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Treat user/users registrations without tokens as account preferences. Present only the forwarded account during gh execution, retain its per-account settings, and restore hidden preferences and original blank token fields on writeback. Preserve the original host after logout of its synthetic forwarded account. Probe local authentication in disposable configuration snapshots: real gh auth token can migrate configuration even when no token is found, so it must never receive the original directory or the immutable writeback baseline. Keep local file, environment and obtainable keyring authentication ahead of forwarding. Do not forward after lookup execution failure or timeout. The gh CLI reports missing and inaccessible keyring entries identically; document and verify that boundary instead of inferring authentication from account registrations. Retain credential-free preference/probe/restoration decisions in existing local diagnostics. No mise-specific product behavior or dependency is introduced. Validation: related go test -race and changed-scope lint passed. The isolated real gh 2.100.0 suite passed against actual relay, mock APIs and Linux keyring, including metadata round trips, local and forwarded logout, parallel updates, 401/403 local precedence, backend-unavailable limits and existing recovery and privacy checks. The broader race suite retains the known baseline failure in TestSnapshotWorkspacePolicyDoesNotFilterOtherMount. No live workspace update or deployment was performed; real provider/IDE/mise integration remains outside this validation. Repo mise settings are provided for manual application only. ## Transcript - U1: > # 実装依頼 > > ## 依頼 > `/Users/a13400/orca/workspaces/devsy/gh-relay` の現在のcheckoutで、GitHub CLI credentials relayが、認証情報を含まない `hosts.yml` のuser設定だけを理由に転送を抑止する不具合を修正してください。 > > repo側のmise設定は編集せず、後述の編集内容と適用方法を報告してください。Devsy本体にはmise専用処理を追加しないでください。 > > ## 目的 > - dotfilesでGitHubのuser設定を管理しながら、ホストの認証を利用できるようにする。 > - 実際のローカル認証を優先する既存の契約を維持する。 > - miseとの連携は利用するrepo側で設定する。 > > ## 成功条件 > - `hosts.yml` にuser・usersのアカウント登録があっても、認証情報がなければホスト側のactive accountを転送する。 > - 元のuser設定とその他の設定を保持し、転送情報を元設定に保存しない。 > - 明示された認証用環境変数、ローカルトークン、keyring認証があれば優先する。期限切れ・権限不足を理由にホスト認証へ切り替えない。 > - 転送時のアカウント表示・選択・認証状態は実際の `gh` に任せる。 > - user設定だけがある状態での転送、設定再保存、ローカルlogin/logout、異なるアカウントへの転送、並行更新を、実際の `gh` と隔離したダミー認証で検証する。 > - 既存の設定保存、symlink保持、拡張・alias・子プロセス、異常終了後の回復を維持する。 > - repoを編集せず、mise設定の追記内容、`mise reshim` による反映方法、確認方法を示す。 > - ローカル検証と実workspaceでの検証範囲を区別して報告する。 > > ## 決定事項 > - user・usersのアカウント登録だけではローカル認証ありと判定しない。 > - 理由: 利用者は認証情報を含まないuser設定をdotfilesで管理している。 > - 主な棄却案: アカウント登録があれば無条件に転送を抑止する現行方式。 > - Devsyは汎用wrapperを提供し、miseを考慮しない。 > - 理由: ツール管理との連携は利用環境側の責務とする。 > - 主な棄却案: Devsyによるmise設定の自動変更、shell別のmise対策、mise管理下のgh本体の差し替え。 > - repo側ではmiseの公式command wrapper機能を利用する。 > - 理由: repoが選ぶghの版を維持し、activation・mise exec・taskから同じ連携を利用できる。 > - 主な棄却案: PATHへの一度きりの追加、対話shellだけのalias。 > - Devsy wrapperが存在すれば委譲し、なければ通常のghを実行する。 > - 理由: Devsy以外でも同じrepo設定を使い、追加の有効化フラグを不要にする。 > - 主な棄却案: repo側の別フラグによる有効化。 > - 今回の実装対象はDevsyの認証判定修正だけとし、repo側は編集内容の提示までとする。 > - 理由: 利用者が実装範囲を明示的に分離した。 > - 主な棄却案: 対象repoのmise設定も同時に編集する。 > - gh wrapperは設置時のDevsyの絶対パスを呼ぶ方式を維持する。 > - 理由: repoがtoolsに別版のdevsyを宣言していても、relayを提供するDevsyを呼べる。 > - 主な棄却案: PATH上のdevsyを呼ぶ方式。 > > ## 制約 > - wrapperでホスト判定、コマンド分類、失敗したコマンド全体の自動再実行を行わない。 > - ghのstdout・stderr・終了コードを加工せず、独自の認証状態表示を作らない。 > - 元設定を削除して解決しない。dotfilesの管理対象・適用済みファイルを変更しない。 > - 転送元のホスト認証ストアを変更しない。 > - トークンをログ、診断記録、検証成果物へ含めない。 > - Devsyにmise依存、mise専用設定、mise管理下のバイナリ変更を追加しない。 > - repo側のmise設定は提案だけとし、編集・適用しない。 > - workspaceの再作成、認証変更、再起動、agent更新、push、release、cloud resource変更は今回の依頼に含めない。 > > ## 対象外 > - 任意のツール管理ソフトによるPATH変更への自動対応。 > - gh本体やmise本体の変更。 > - ホストでのghインストール・ログイン・アカウント切替。 > - 同一トークンの再保存を独立した認証保存として必ず識別すること。 > - repo側設定のcommitや適用。 > > ## 未検証の前提 > - user設定だけがある場合の転送に必要な、現在の設定overlay・転送情報除去・差分書き戻しの変更箇所。 > - 元のuser設定と転送アカウントが異なる場合の、利用者変更と転送情報を区別する具体的な処理。 > - 対象ghバージョンでの、keyring認証の存在・取得不能・未登録を識別できる範囲。 > - 対象repo・VMで稼働するmiseの版と、提示設定の実環境での動作。 > > ## 実装時の指示 > - 主な対象: > - `pkg/ghcredentials/session.go` の `localAuthSource`、`Account.inject`、転送情報除去、書き戻し。 > - `pkg/ghcredentials/session_test.go` > - `pkg/ghcredentials/testdata/fixture.py` > - `local_registration` の早期returnを削除するだけで完了としない。転送中に上書きしたuser情報を除去するとき、元から存在したuser設定を削除・変更してしまわないことを検証する。 > - 既存の実gh検証taskは `mise exec -- task cli:test:gh-relay`。 > - 既存調査で確認した対象workspace: > - Orca環境: `glaze VM 0a14a489` > - 保存済みenvironment ID: `7927b72a-8746-4bda-a161-7798c82eb95a` > - workspace: `/home/me/orca/workspaces/orca-93d3d24f-5196-420e-9f43-5f280a14a489/env2` > - 接続状態・識別子は再確認すること。 > - VM setup直後の調査では、以下を確認済み: > - 通常のghはrepoのmiseが選ぶ `gh 2.95.0` で、Devsy wrapperよりPATH上で優先されていた。 > - `/usr/local/share/devsy/gh/bin/gh` は存在し、`/usr/local/bin/devsy` を絶対パスで呼んでいた。 > - ホスト認証取得は成功し、診断の `host_status` は `available`。 > - wrapperは `github.com` を `local_registration` と判定し、転送を抑止していた。 > - `/home/me/.config/gh/hosts.yml` は、chezmoi source `/home/me/dotfiles/home/dot_config/gh/private_hosts.yml` と一致し、user・usersを含むがoauth_tokenを含まなかった。 > - 元設定で通常gh・明示wrapperの両方の `gh pr list` が401。 > - 空の一時 `GH_CONFIG_DIR` と明示wrapperの組合せでは、同じworkspaceの `gh auth status`・`gh pr list` が成功。 > - 調査前後で元のhosts.ymlが変わっていないことを確認済み。 > - repo側へ提示する設定案: > > ```toml > [wrappers.gh] > command = "/bin/sh" > args = [ > "-c", > 'if [ -x /usr/local/share/devsy/gh/bin/gh ]; then exec /usr/local/share/devsy/gh/bin/gh "$@"; else exec gh "$@"; fi', > "gh", > ] > ``` > > - 設定追加後の反映操作は `mise reshim`。公式資料は https://mise.jdx.dev/dev-tools/shims.html#command-wrappers 。 > - mise 2026.10.8の隔離環境では、ダミーwrapperの有無それぞれについて、activation・mise exec・taskの6ケースで委譲・フォールバック・引数保持を確認済み。実Devsy wrapperとの組合せを対象repoへ適用した検証は未実施。 > - miseの検証環境ではHOME・設定・data・cache・state・shimsをすべて隔離する。ホストのmise shimや既存activation状態を混入させない。 > - 既存の広い単体テストには、変更前でも再現した `TestSnapshotWorkspacePolicyDoesNotFilterOtherMount` の失敗がある。今回の変更による失敗と区別する。 - A: Worked only in the existing atty303/gh-relay checkout from 6cb2b83c2. Fetched code, preserved the clean baseline, confirmed no applicable automatic-release approval workflow, and applied investigation/development/observability/review guidance. Added a regression that failed before the fix with local_registration instead of forwarded. Changed tokenless-account handling, temporary user registration visibility and sanitized restoration while preserving actual local credentials. Isolated real-gh testing revealed that gh auth token itself rewrites tokenless configuration; a direct real-gh one-factor probe reproduced the original file becoming empty, while a non-mutating fake probe did not. Moved all token lookups into disposable two-file snapshots, separate from originals and immutable baselines. Preserved original blank/null token fields, per-account user settings and hidden host settings after forwarded logout. Added safe preference/source/probe/restoration diagnostics and conservative handling of lookup execution failures/timeouts. Reviewed the actual gh 2.100.0 implementation: missing keyring entries and backend errors collapse into the same no-token exit result. Documented and tested that limitation rather than introducing a new Secret Service implementation; obtainable keyring credentials remain preferred even for 401/403. Corrected fixture assertions to inspect host/account protocol settings separately and corrected the Enterprise API prefix used to generate 403. Fresh independent review and delta review completed with no remaining findings. Final related race tests and changed-scope lint passed. The full isolated actual-gh suite passed in 48.01s, including tokenless registrations, exact unchanged hosts preservation, blank token fields, forwarding/logout, local login/logout, setting re-save, parallel user and host updates, keyring 401/403, unavailable-backend behavior and credential preservation, existing aliases/extensions/signals/recovery/privacy cases. The broader scope race run retained only the documented pre-existing TestSnapshotWorkspacePolicyDoesNotFilterOtherMount failure. No repo mise settings, personal authentication, live VM/workspace state, agent deployment, remote refs or cloud resources were changed. The proposed mise edit remains a report-only deliverable. Verified the final index equals the reviewed snapshot and froze this transcript before committing. Co-authored-by: Codex --- pkg/ghcredentials/diagnostics.go | 7 +- pkg/ghcredentials/session.go | 189 ++++++++++++++---- pkg/ghcredentials/session_test.go | 111 +++++++++- pkg/ghcredentials/testdata/fixture.py | 67 ++++++- pkg/ghcredentials/writeback.go | 6 + .../developing-in-workspaces/credentials.mdx | 4 +- 6 files changed, 343 insertions(+), 41 deletions(-) diff --git a/pkg/ghcredentials/diagnostics.go b/pkg/ghcredentials/diagnostics.go index 824279be58..c073cc7a2a 100644 --- a/pkg/ghcredentials/diagnostics.go +++ b/pkg/ghcredentials/diagnostics.go @@ -11,8 +11,11 @@ import ( ) type hostDecision struct { - Host string `json:"host"` - Source string `json:"source"` + Host string `json:"host"` + Source string `json:"source"` + Preferences bool `json:"account_preferences,omitempty"` + SameUser bool `json:"same_user,omitempty"` + Probe string `json:"token_probe,omitempty"` } // Only these source-allowlisted fields may enter local diagnostics. Command diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index c2986d2e08..0d8b4796c6 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -9,6 +9,7 @@ import ( "fmt" "io/fs" "os" + "os/exec" "path/filepath" "strings" "time" @@ -247,14 +248,11 @@ func localAuthSource(n *yaml.Node) string { users := get(n, "users") if users != nil { for i := 1; i < len(users.Content); i += 2 { - if get(users.Content[i], "oauth_token") != nil { + if token := get(users.Content[i], "oauth_token"); token != nil && token.Value != "" { return "local_config" } } } - if user := get(n, "user"); user != nil && user.Value != "" { - return "local_registration" - } return "" } @@ -279,19 +277,13 @@ func (s *session) inject(ctx context.Context, binary string, env []string) error if err != nil { return boundary("config.decode", err) } - var injected []Account - for _, a := range s.Accounts { - source := a.inject(ctx, binary, env, hosts) - s.decisions = append(s.decisions, hostDecision{Host: a.Host, Source: source}) - if source == "forwarded" { - injected = append(injected, a) - } + if err := s.overlay(ctx, binary, env, hosts); err != nil { + return err } - s.Accounts = injected if err := s.save(); err != nil { return err } - if len(injected) == 0 { + if len(s.Accounts) == 0 { return nil } b, err = yaml.Marshal(hosts) @@ -301,13 +293,71 @@ func (s *session) inject(ctx context.Context, binary string, env []string) error return os.WriteFile(path, b, 0o600) } -func (s *session) strip(data []byte) ([]byte, error) { +// Even gh auth token can migrate configuration. Never point the lookup at +// original settings or the immutable writeback baseline. Each host gets a fresh +// copy because a migration may change registrations for other hosts as well. +func (s *session) overlay( + ctx context.Context, + binary string, + env []string, + hosts *yaml.Node, +) error { + var injected []Account + for _, a := range s.Accounts { + probe, err := s.probeConfig() + if err != nil { + return boundary("config.probe_snapshot", err) + } + decision := a.inject(ctx, binary, replaceEnv(env, "GH_CONFIG_DIR", probe), hosts) + _ = os.RemoveAll(probe) + s.decisions = append(s.decisions, decision) + if decision.Source == "forwarded" { + injected = append(injected, a) + } + } + s.Accounts = injected + return nil +} + +func (s *session) probeConfig() (string, error) { + dir, err := os.MkdirTemp(s.dir, "probe-") + if err != nil { + return "", err + } + for _, name := range []string{hostsFile, configFile} { + if err := s.copyProbeFile(dir, name); err != nil { + _ = os.RemoveAll(dir) + return "", err + } + } + return dir, nil +} + +func (s *session) copyProbeFile(dir, name string) error { + b, err := readFile(filepath.Join(s.dir, "before", name)) + if err != nil || b == nil { + return err + } + return os.WriteFile(filepath.Join(dir, name), b, 0o600) +} + +func (s *session) strip(before, data []byte) ([]byte, error) { + original, err := decode(before) + if err != nil { + return nil, err + } hosts, err := decode(data) if err != nil { return nil, err } for _, a := range s.Accounts { - stripAccount(hosts, a) + outcome := stripAccount(hosts, get(original, a.Host), a) + if s.writeback != nil { + s.writeback.Restorations = append( + s.writeback.Restorations, + hostRestoration{Host: a.Host, Outcome: outcome}, + ) + } } return yaml.Marshal(hosts) } @@ -396,27 +446,73 @@ func (s *session) finish(ctx context.Context) error { return err } -func (a Account) inject(ctx context.Context, binary string, env []string, hosts *yaml.Node) string { - if a.Host == "" || a.User == "" || a.Token == "" { - return "invalid_account" +func (a Account) inject( + ctx context.Context, + binary string, + env []string, + hosts *yaml.Node, +) hostDecision { + h := get(hosts, a.Host) + decision := hostDecision{Host: a.Host, Source: "invalid_account"} + if !a.valid() { + return decision } - if source := localAuthSource(get(hosts, a.Host)); source != "" { - return source + decision.Preferences = get(h, "user") != nil || get(h, "users") != nil + if user := get(h, "user"); user != nil { + decision.SameUser = user.Value == a.User + } + if source := localAuthSource(h); source != "" { + decision.Source = source + return decision + } + token, err := resolveToken(ctx, binary, a.Host, env) + decision.Probe = "empty" + if err != nil { + decision.Probe = errorKind(err) + if !noTokenResult(err) { + decision.Source = "local_lookup_failed" + return decision + } } - if token := tokenForHost(ctx, binary, a.Host, env); token != "" { - return localTokenSource(token, env) + if token != "" { + decision.Probe = "token_returned" + decision.Source = localTokenSource(token, env) + return decision } + a.overlay(hosts) + decision.Source = "forwarded" + return decision +} + +// gh auth token reports an absent token as exit 1. It also collapses keyring +// backend errors into that result; the CLI cannot distinguish those cases. +// Execution failures and timeouts are not evidence of absent authentication. +func noTokenResult(err error) bool { + exit, ok := errors.AsType[*exec.ExitError](err) + return ok && exit.ExitCode() == 1 +} + +func (a Account) valid() bool { + return a.Host != "" && a.User != "" && a.Token != "" +} + +func (a Account) overlay(hosts *yaml.Node) { h := ensure(hosts, a.Host) put(h, "user", scalar(a.User)) put(h, "oauth_token", scalar(a.Token)) - put(ensure(ensure(h, "users"), a.User), "oauth_token", scalar(a.Token)) - return "forwarded" + // Tokenless registrations are settings, not usable accounts. Hide them + // only in the overlay so gh auth status does not report them as failures. + users := mapping() + put(users, a.User, get(get(h, "users"), a.User)) + put(ensure(users, a.User), "oauth_token", scalar(a.Token)) + put(h, "users", users) } -func stripAccount(hosts *yaml.Node, a Account) { +func stripAccount(hosts, original *yaml.Node, a Account) string { h := get(hosts, a.Host) if h == nil { - return + put(hosts, a.Host, original) + return "original_host_restored" } u := get(h, "users") account := get(u, a.User) @@ -425,25 +521,38 @@ func stripAccount(hosts *yaml.Node, a Account) { // Successful keyring login removes both file tokens. Keep its account // registration; the keyring itself remains entirely under gh's control. keyringLogin := activeToken == nil && userToken == nil - removeForwardedToken(h, a.Token) - removeForwardedToken(account, a.Token) + removeForwardedToken(h, original, a.Token) + removeForwardedToken(account, get(get(original, "users"), a.User), a.Token) if !keyringLogin { - stripRegistration(h, u, account, a.User) + stripRegistration(h, get(original, "user"), a.User) } + // Restore hidden dotfiles registrations while retaining new local logins. + // No real original credential can be hidden: inject excludes those hosts. + users := get(h, "users") + if users == nil { + users = mapping() + } + put(h, "users", merge(mapping(), users, get(original, "users"))) if len(h.Content) == 0 { put(hosts, a.Host, nil) } + if keyringLogin { + return "keyring_registration" + } + return "preferences_restored" } -func removeForwardedToken(n *yaml.Node, token string) { +func removeForwardedToken(n, original *yaml.Node, token string) { if t := get(n, "oauth_token"); t != nil && t.Value == token { - put(n, "oauth_token", nil) + put(n, "oauth_token", get(original, "oauth_token")) } } -func stripRegistration(host, users, account *yaml.Node, user string) { - if t := get(host, "user"); t != nil && t.Value == user && get(account, "oauth_token") == nil { - put(host, "user", nil) +func stripRegistration(host, originalUser *yaml.Node, user string) { + users := get(host, "users") + account := get(users, user) + if forwardedRegistration(host, account, user) { + put(host, "user", originalUser) } if account != nil && len(account.Content) == 0 { put(users, user, nil) @@ -453,6 +562,16 @@ func stripRegistration(host, users, account *yaml.Node, user string) { } } +func hasToken(n *yaml.Node) bool { + t := get(n, "oauth_token") + return t != nil && t.Value != "" +} + +func forwardedRegistration(host, account *yaml.Node, user string) bool { + t := get(host, "user") + return t != nil && t.Value == user && !hasToken(account) && !hasToken(host) +} + func (s *session) changedFile(name string) ([]byte, []byte, error) { before, err := readFile(filepath.Join(s.dir, "before", name)) if err != nil { @@ -463,7 +582,7 @@ func (s *session) changedFile(name string) ([]byte, []byte, error) { return nil, nil, boundary("config.changed_read", err) } if name == hostsFile && len(after) != 0 { - after, err = s.strip(after) + after, err = s.strip(before, after) if err != nil { return nil, nil, boundary("config.strip", err) } diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index 07901ba043..ad11152e23 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -5,10 +5,13 @@ import ( "os" "path/filepath" "testing" + "time" "github.com/stretchr/testify/require" ) +const testPathEnv = "PATH=/usr/bin:/bin" + func writeTestFile(t *testing.T, path, content string) { t.Helper() require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o700)) @@ -96,7 +99,7 @@ func TestStripForwardedCredentialsAndPreserveLocalLogin(t *testing.T) { {"keyring login", "github.example:\n user: relay-user\n users:\n relay-user: {}\n", true, true}, } { t.Run(tc.name, func(t *testing.T) { - got, err := s.strip([]byte(tc.input)) + got, err := s.strip(nil, []byte(tc.input)) require.NoError(t, err) // Do not include the produced configuration in assertion failures. n, err := decode(got) @@ -162,3 +165,109 @@ func TestDisabledWritebackDiagnosticsDoNotParseConfiguration(t *testing.T) { // Invalid YAML would fail diagnostic parsing if the disabled sink were entered. require.NoError(t, summary.inspect(configFile, []byte("["), []byte("["), []byte("["))) } + +func TestSessionForwardsTokenlessRegistrations(t *testing.T) { + for _, tc := range []struct { + user, tokenField, userBody string + }{ + {"relay-user", "", "{}"}, + {"dotfiles-user", "", ""}, + {"relay-user", " oauth_token: \"\"\n", "\n oauth_token: \"\""}, + {"dotfiles-user", " oauth_token:\n", "\n oauth_token:"}, + } { + t.Run(tc.user+tc.tokenField, func(t *testing.T) { + root := t.TempDir() + t.Setenv("HOME", root) + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + original := filepath.Join(root, "gh") + before := "github.example:\n git_protocol: https\n user: " + tc.user + "\n" + tc.tokenField + + " users:\n " + tc.user + ": " + tc.userBody + "\n other-user:\n" + writeTestFile(t, filepath.Join(original, hostsFile), before) + a := Account{Host: "github.example", User: "relay-user", Token: "dummy-forwarded-value"} + s, err := startSession(context.Background(), original, []Account{a}) + require.NoError(t, err) + t.Cleanup(func() { _ = s.lock.Close() }) + env := []string{"HOME=" + root, "GH_CONFIG_DIR=" + original, testPathEnv} + require.NoError(t, s.inject(context.Background(), "/usr/bin/false", env)) + require.Equal(t, "forwarded", s.decisions[0].Source) + require.True(t, s.decisions[0].Preferences) + require.Equal(t, tc.user == a.User, s.decisions[0].SameUser) + data, err := readFile(filepath.Join(s.work(), hostsFile)) + require.NoError(t, err) + n, err := decode(data) + require.NoError(t, err) + h := get(n, a.Host) + require.Equal(t, a.User, get(h, "user").Value) + require.Len(t, get(h, "users").Content, 2) + require.NoError(t, s.finish(context.Background())) + got, err := readFile(filepath.Join(original, hostsFile)) + require.NoError(t, err) + // Never print credential-bearing configuration on assertion failure. + require.True(t, string(got) == before, "tokenless dotfiles registration changed") + }) + } +} + +func TestAuthenticationProbeCannotMigrateOriginalSettings(t *testing.T) { + root := t.TempDir() + t.Setenv("HOME", root) + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + original := filepath.Join(root, "gh") + before := "github.example:\n user: dotfiles-user\n users:\n dotfiles-user: {}\n" + writeTestFile(t, filepath.Join(original, hostsFile), before) + probe := filepath.Join(root, "auth") + writeTestFile(t, probe, `#!/bin/sh +printf '{}\n' > "$GH_CONFIG_DIR/hosts.yml" +printf 'version: "1"\n' > "$GH_CONFIG_DIR/config.yml" +exit 1 +`) + // resolveToken passes "auth" as the first argument, so sh reads this fixture. + t.Chdir(root) + s, err := startSession(context.Background(), original, []Account{ + {Host: "github.example", User: "relay-user", Token: "dummy-forwarded-value"}, + }) + require.NoError(t, err) + t.Cleanup(func() { _ = s.lock.Close() }) + env := []string{"HOME=" + root, "GH_CONFIG_DIR=" + original, testPathEnv} + require.NoError(t, s.inject(context.Background(), "/bin/sh", env)) + for _, path := range []string{filepath.Join(original, hostsFile), filepath.Join(s.dir, "before", hostsFile)} { + got, err := readFile(path) + require.NoError(t, err) + require.True(t, string(got) == before, "authentication probe modified original or baseline") + } + require.NoFileExists(t, filepath.Join(original, configFile)) + require.NoFileExists(t, filepath.Join(s.work(), configFile)) + require.NoError(t, s.finish(context.Background())) + got, err := readFile(filepath.Join(original, hostsFile)) + require.NoError(t, err) + require.True(t, string(got) == before, "probe migration was written back") +} + +func TestForwardedLogoutRestoresHiddenPreferences(t *testing.T) { + before := []byte( + "github.example:\n user: dotfiles-user\n git_protocol: ssh\n users:\n dotfiles-user: {}\n", + ) + s := &session{ + Accounts: []Account{ + {Host: "github.example", User: "relay-user", Token: "dummy-forwarded-value"}, + }, + } + after, err := s.strip(before, []byte("{}\n")) + require.NoError(t, err) + original, err := decode(before) + require.NoError(t, err) + result, err := decode(after) + require.NoError(t, err) + require.True(t, equal(original, result), "forwarded logout removed hidden preferences") +} + +func TestTokenLookupTimeoutDoesNotAllowForwarding(t *testing.T) { + ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(-time.Second)) + defer cancel() + a := Account{Host: "github.example", User: "relay-user", Token: "dummy-forwarded-value"} + hosts := mapping() + decision := a.inject(ctx, "/usr/bin/false", []string{testPathEnv}, hosts) + require.Equal(t, "local_lookup_failed", decision.Source) + require.Equal(t, "timeout", decision.Probe) + require.Nil(t, get(hosts, a.Host)) +} diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py index 265b21e3f5..2772d11062 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/pkg/ghcredentials/testdata/fixture.py @@ -32,6 +32,9 @@ def handle_api(self): self.rfile.read(int(self.headers.get('Content-Length', 0))) token = self.headers.get('Authorization', '').split(' ')[-1] user = tokens.get(token) + if user == 'local-user' and (ROOT / 'revoke-local').exists(): + user = None + forbidden = user == 'local-user' and (ROOT / 'forbid-local').exists() and self.path.startswith(('/repos/', '/api/v3/repos/')) body = {'login': user, 'id': 123, 'data': {'viewer': {'login': user}}} if user else {'message': 'Bad credentials'} if self.path == '/login/device/code': user = 'host-user' @@ -40,7 +43,7 @@ def handle_api(self): user = 'host-user' body = {'access_token': secrets[HOSTS[0]]['refresh'], 'token_type': 'bearer', 'scope': 'repo,read:org,gist'} data = json.dumps(body).encode() - self.send_response(200 if user else 401) + self.send_response(403 if forbidden else 200 if user else 401) self.send_header('X-OAuth-Scopes', 'repo, read:org, gist') self.send_header('Content-Type', 'application/json') self.send_header('Content-Length', str(len(data))) @@ -103,6 +106,52 @@ def last_diagnostic(): return json.loads(max(folder.glob('*.json'), key=lambda f: f.stat().st_mtime_ns).read_text()) + # Dotfiles contain account preferences, never authentication. + registrations = ( + 'github.com:\n git_protocol: https\n user: local-user\n users:\n local-user:\n' + 'enterprise.one.test:\n git_protocol: https\n user: host-user\n oauth_token: \"\"\n users:\n host-user:\n oauth_token: \"\"\n' + 'enterprise.two.test:\n git_protocol: https\n user: dotfiles-user\n oauth_token:\n users:\n dotfiles-user:\n oauth_token:\n other-user:\n' + ) + (original / 'hosts.yml').write_text(registrations) + verify('tokenless registrations use host active accounts', all(api(h) for h in HOSTS)) + verify('real auth status ignores hidden tokenless registrations', gh('auth', 'status').returncode == 0) + verify('unchanged invocation preserves exact dotfiles registrations', saved() == registrations and no_forwarded_saved()) + decisions = last_diagnostic()['hosts'] + verify('registration forwarding retained in diagnostics', decisions and all(h['source'] == 'forwarded' and h.get('account_preferences') and h.get('token_probe') for h in decisions) and decisions[1].get('same_user') and not decisions[2].get('same_user')) + verify('registration restoration retained in diagnostics', all(r['outcome'] == 'preferences_restored' for r in last_diagnostic()['writeback']['restorations'])) + r = gh('auth', 'logout', '--hostname', HOSTS[2], '--user', 'host-user') + verify('forwarded logout preserves hidden dotfiles host', r.returncode == 0 and saved() == registrations and api(HOSTS[2])) + r = gh('config', 'set', 'git_protocol', 'https', '--host', HOSTS[2]) + verify('host setting resave preserves registrations and user preference', r.returncode == 0 and saved() == registrations + ' host-user:\n git_protocol: https\n') + verify('forwarding preserves saved per-account settings', gh('config', 'get', 'git_protocol', '--host', HOSTS[2]).stdout.strip() == 'https') + r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) + verify('local login from tokenless preferences persists', r.returncode == 0 and api(HOSTS[0], 'local-user') and no_forwarded_saved()) + r = gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user') + verify('local logout from preferences restores forwarding', r.returncode == 0 and api(HOSTS[0])) + (original / 'hosts.yml').write_text(registrations) + ready_meta, release_meta = ROOT / 'metadata-ready', ROOT / 'metadata-release' + verify('metadata mutation alias saves', gh('alias', 'set', 'metadata-pending', '!gh config set git_protocol ssh --host enterprise.two.test; touch "$GH_TEST_READY"; while test ! -f "$GH_TEST_RELEASE"; do sleep 0.05; done').returncode == 0) + e = dict(os.environ, GH_TEST_READY=str(ready_meta), GH_TEST_RELEASE=str(release_meta)) + p = subprocess.Popen(['/bin/bash', '-lc', 'exec gh metadata-pending'], env=e, stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True) + try: + for _ in range(200): + if ready_meta.exists(): + break + time.sleep(0.05) + verify('tokenless metadata invocation reached mutation', ready_meta.exists()) + changed = registrations.replace('user: dotfiles-user', 'user: parallel-user') + 'unrelated.test:\n git_protocol: ssh\n' + (original / 'hosts.yml').write_text(changed) + release_meta.touch() + p.communicate(timeout=20) + verify('host-setting delta preserves parallel user and host changes', p.returncode == 0 and 'user: parallel-user' in saved() and 'unrelated.test:\n git_protocol: ssh' in saved() and no_forwarded_saved()) + verify('host-setting delta persists at host and account positions', 'enterprise.two.test:\n git_protocol: ssh' in saved() and ' host-user:\n git_protocol: ssh' in saved()) + finally: + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + verify('metadata alias removed', gh('alias', 'delete', 'metadata-pending').returncode == 0) + (original / 'hosts.yml').unlink() + verify('all hosts forwarded through real RPC and HTTP relay', all(api(h) for h in HOSTS)) verify('real auth status', gh('auth', 'status').returncode == 0) verify('forwarding never saved', no_forwarded_saved()) @@ -129,6 +178,22 @@ def last_diagnostic(): r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['local']) verify('keyring login succeeds without plaintext token', r.returncode == 0 and secrets[HOSTS[1]]['local'] not in saved() and api(HOSTS[1], 'local-user')) + verify('existing keyring credential excluded from forwarding', last_diagnostic()['hosts'][1]['source'] == 'keyring') + verify('unavailable keyring backend follows gh no-token result', api(HOSTS[1], env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/nonexistent'}) and last_diagnostic()['hosts'][1]['source'] == 'forwarded') + verify('unavailable backend leaves local keyring intact', api(HOSTS[1], 'local-user')) + revoked = ROOT / 'revoke-local' + revoked.touch() + try: + verify('expired keyring credential never falls back', not api(HOSTS[1]) and last_diagnostic()['hosts'][1]['source'] == 'keyring') + finally: + revoked.unlink() + forbidden = ROOT / 'forbid-local' + forbidden.touch() + try: + r = gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) + verify('insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr and last_diagnostic()['hosts'][1]['source'] == 'keyring') + finally: + forbidden.unlink() r = gh('auth', 'status', '--hostname', HOSTS[1]) verify('real auth status identifies keyring', r.returncode == 0 and 'keyring' in r.stdout + r.stderr) verify('keyring logout restores forwarding', gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'local-user').returncode == 0 and api(HOSTS[1])) diff --git a/pkg/ghcredentials/writeback.go b/pkg/ghcredentials/writeback.go index 9ffc224559..b48f44ec71 100644 --- a/pkg/ghcredentials/writeback.go +++ b/pkg/ghcredentials/writeback.go @@ -5,11 +5,17 @@ import "gopkg.in/yaml.v3" // Categories are a fixed allowlist: extension paths, aliases, account names, // arbitrary configuration keys and values must never enter diagnostics. type writebackSummary struct { + Restorations []hostRestoration `json:"restorations,omitempty"` UnchangedFiles int `json:"unchanged_files"` WrittenFiles int `json:"written_files"` Changes map[string]changeCount `json:"changes,omitempty"` } +type hostRestoration struct { + Host string `json:"host"` + Outcome string `json:"outcome"` +} + func (s *writebackSummary) unchanged() { if s != nil { s.UnchangedFiles++ diff --git a/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx b/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx index 32cfe5d7c2..7ac2240cc2 100644 --- a/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx +++ b/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx @@ -29,9 +29,9 @@ Disabling HTTPS credential injection does not disable SSH agent forwarding. ## GitHub CLI -With `gh` installed on the host and in the container, normal `gh` commands, aliases, and extensions can use the host's active account for each configured GitHub.com or GitHub Enterprise host. `gh` selects the target host. Explicit authentication environment variables and container-local authentication take precedence, even when invalid or expired. +With `gh` installed on the host and in the container, normal `gh` commands, aliases, and extensions can use the host's active account for each configured GitHub.com or GitHub Enterprise host. `gh` selects the target host. Explicit authentication environment variables and container-local authentication take precedence, even when invalid or expired. Account preferences in `hosts.yml` without credentials, such as dotfiles-managed `user` and `users` settings, do not prevent forwarding and are preserved. -Container logins and settings changes are saved locally. Logging out of a local account makes host authentication available again on the next invocation; it does not log out the host. Re-saving an unchanged forwarded token in the configuration file does not create independent local authentication. An explicit login saved in the container’s keyring is retained. If host authentication is unavailable, `gh` runs with its normal local configuration. +Container logins and settings changes are saved locally. Logging out of a local account makes host authentication available again on the next invocation; it does not log out the host. Re-saving an unchanged forwarded token in the configuration file does not create independent local authentication. An explicit login saved in the container’s keyring is retained. `gh auth token` does not distinguish a missing keyring entry from an inaccessible keyring backend; forwarding can apply in either case when no file or environment credential is available. If host authentication is unavailable, `gh` runs with its normal local configuration. To disable GitHub CLI forwarding independently of Git credentials: From 87883d133956210cf10b309092516d943dca1de9 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 09:55:11 +0900 Subject: [PATCH 03/20] fix(credentials): keep nested gh auth lookups local MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mark only authentication-lookup child processes as local-only so dispatcher re-entry cannot return forwarded credentials as local keyring authentication. Keep the mode inherited across config-directory changes without leaking it into ordinary commands, aliases or concurrent invocations. Record the safe lookup mode alongside existing authentication decisions. Regression coverage uses actual gh with fictional authentication and mock APIs, including nested config redirects and local keyring 401/403 precedence. Package race tests, scoped lint and the real-gh integration suite pass. ## Transcript - U1: > /home/me/orca/workspaces/orca-fc6b7a24-6e49-4e33-9205-b559634a6848/devenv-2 ここで実行したがまた401になった - A: Investigated the specified live workspace read-only. The updated mise wrapper and gh were present. A fresh normal invocation succeeded, while Devsy-first dispatch reproduced failure. Retained records showed forwarded credentials returned through a nested lookup being labeled keyring, then forwarding suppressed. Original hosts.yml remained unchanged; no live code/settings changes. - U2: > 今は通るようになった。起動直後はだめだった。タイミング問題があるか？ - A: Distinguished host relay readiness from dispatch timing: failed records already had host_status=available. Suggested a possible entry/PATH transition but did not establish the original startup cause. - U3: > mise wrapperから開始 - A: Accepted the correction that the user started through the mise wrapper; withdrew the claim that their original entry was Devsy-first. Compared initial lookup and installation timestamps; why that invocation selected a shim remained unknown. - U4: > それは進めて。 > > どちらの入口から呼ばれても正しく動くよう、内部の認証確認で転送を再発生させない修正が必要です。 > > これも確実性を高めるためにやったほうがいいがどう方式でやる？ - A: Ran actual mise 2026.10.8, gh 2.95.0 and installed Devsy in the same Linux VM with fully isolated HOME/config/data/cache/state/shims, fictional credentials and a loopback dummy relay. Single and two concurrent cold first calls succeeded; Devsy-first dispatch reliably reproduced the nested authentication defect. Unused lazy tools stayed uninstalled and original fixture preferences stayed unchanged. Removed all owned probe resources. Proposed a private lookup-child mode inherited through nested wrappers, independent of config-directory/session changes, with no mise-specific product code. - U5: > それで修正して - A: Implemented subprocess-scoped DEVSY_GH_AUTH_PROBE and wrapper bypass for local lookups, plus credential-free token_probe_mode diagnostics. Added regressions for nested dispatch, config-directory changes, local environment/file/keyring priority including 401/403, login/logout, aliases and child-env noninterference. The real-gh regression failed before the product change and passed afterward. Final package race tests passed, changed-scope lint reported 0 issues, and all isolated actual-gh checks passed in 61.59 seconds. Fresh read-only independent review accepted the exact candidate with no findings. No repo mise edits, live agent/workspace updates, push, release or cloud mutation. Initial live-startup timing cause remains unproven. Co-authored-by: Codex --- pkg/ghcredentials/diagnostics.go | 1 + pkg/ghcredentials/host.go | 4 ++- pkg/ghcredentials/session.go | 1 + pkg/ghcredentials/session_test.go | 22 ++++++++++++++ pkg/ghcredentials/testdata/fixture.py | 42 +++++++++++++++++++++++++++ pkg/ghcredentials/wrapper.go | 10 +++++-- 6 files changed, 77 insertions(+), 3 deletions(-) diff --git a/pkg/ghcredentials/diagnostics.go b/pkg/ghcredentials/diagnostics.go index c073cc7a2a..198b23e09d 100644 --- a/pkg/ghcredentials/diagnostics.go +++ b/pkg/ghcredentials/diagnostics.go @@ -16,6 +16,7 @@ type hostDecision struct { Preferences bool `json:"account_preferences,omitempty"` SameUser bool `json:"same_user,omitempty"` Probe string `json:"token_probe,omitempty"` + ProbeMode string `json:"token_probe_mode,omitempty"` } // Only these source-allowlisted fields may enter local diagnostics. Command diff --git a/pkg/ghcredentials/host.go b/pkg/ghcredentials/host.go index 223c9444ac..c80845a4c6 100644 --- a/pkg/ghcredentials/host.go +++ b/pkg/ghcredentials/host.go @@ -87,7 +87,9 @@ func resolveToken(ctx context.Context, binary, host string, env []string) (strin defer cancel() //nolint:gosec // binary is resolved from the user's PATH; host is one argument. cmd := exec.CommandContext(ctx, binary, "auth", "token", "--hostname", host) - cmd.Env = env + // Scope this to the lookup child: nested wrappers must not return forwarded + // credentials as evidence of local authentication. + cmd.Env = replaceEnv(env, authProbeEnv, "1") out, err := cmd.Output() if err != nil { if ctx.Err() != nil { diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index 0d8b4796c6..56551d4df0 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -465,6 +465,7 @@ func (a Account) inject( decision.Source = source return decision } + decision.ProbeMode = "local_only" token, err := resolveToken(ctx, binary, a.Host, env) decision.Probe = "empty" if err != nil { diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index ad11152e23..07b296207c 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -271,3 +271,25 @@ func TestTokenLookupTimeoutDoesNotAllowForwarding(t *testing.T) { require.Equal(t, "timeout", decision.Probe) require.Nil(t, get(hosts, a.Host)) } + +func TestTokenLookupIsLocalOnlyAndChildScoped(t *testing.T) { + root := t.TempDir() + t.Chdir(root) + t.Setenv("DEVSY_GH_AUTH_PROBE", "parent-value") + writeTestFile(t, filepath.Join(root, "auth"), `#!/bin/sh +test "$DEVSY_GH_AUTH_PROBE" = 1 || exit 42 +test "$DEVSY_GH_SESSION" = existing-session || exit 43 +printf '%s' "$GH_TOKEN" +`) + env := []string{ + testPathEnv, + "DEVSY_GH_AUTH_PROBE=0", + "DEVSY_GH_SESSION=existing-session", + "GH_TOKEN=dummy-local-value", + } + token, err := resolveToken(context.Background(), "/bin/sh", "github.example", env) + require.NoError(t, err) + require.True(t, "GH_TOKEN="+token == env[3], "local environment authentication changed") + require.Equal(t, "DEVSY_GH_AUTH_PROBE=0", env[1]) + require.Equal(t, "parent-value", os.Getenv("DEVSY_GH_AUTH_PROBE")) +} diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py index 2772d11062..07af3aad12 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/pkg/ghcredentials/testdata/fixture.py @@ -113,6 +113,44 @@ def last_diagnostic(): 'enterprise.two.test:\n git_protocol: https\n user: dotfiles-user\n oauth_token:\n users:\n dotfiles-user:\n oauth_token:\n other-user:\n' ) (original / 'hosts.yml').write_text(registrations) + + # A third-party dispatcher can return to Devsy after resolving the real gh. + # Exercise both entries without depending on any particular tool manager. + dispatcher = ROOT / 'dispatcher' + dispatcher.mkdir() + shim = dispatcher / 'gh' + shim.write_text('#!/bin/sh\nexport PATH=/usr/local/bin:/usr/bin:/bin\nif [ -n "$GH_TEST_CONFIG_OVERRIDE" ]; then export GH_CONFIG_DIR="$GH_TEST_CONFIG_OVERRIDE"; fi\nexec /usr/local/share/devsy/gh/bin/gh "$@"\n') + shim.chmod(0o755) + def nested_gh(*args, token=None, env=None): + e = dict(os.environ, PATH=str(dispatcher) + ':/usr/local/bin:/usr/bin:/bin') + if env: + e.update(env) + return subprocess.run(['/usr/local/share/devsy/gh/bin/gh', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + def nested_api(host, expected='host-user', **kw): + r = nested_gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) + return r.returncode == 0 and r.stdout.strip() == expected + verify('dispatcher re-entry cannot supply forwarded authentication to local lookup', all(nested_api(h) for h in HOSTS)) + verify('dispatcher re-entry preserves real auth status', nested_gh('auth', 'status').returncode == 0) + verify('dispatcher re-entry records local-only lookup', all(h['source'] == 'forwarded' and h['token_probe_mode'] == 'local_only' for h in last_diagnostic()['hosts'])) + redirected = ROOT / 'redirected-gh' + redirected.mkdir() + (redirected / 'hosts.yml').write_text(registrations) + verify('local-only lookup survives dispatcher config changes', nested_api(HOSTS[0], env={'GH_TEST_CONFIG_OVERRIDE': str(redirected)})) + verify('nested lookup leaves original preferences unchanged', saved() == registrations and no_forwarded_saved()) + verify('nested lookup retains explicit invalid environment authentication', not nested_api(HOSTS[0], env={'GH_TOKEN': secrets[HOSTS[0]]['invalid']})) + r = nested_gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) + verify('nested local login retains precedence', r.returncode == 0 and nested_api(HOSTS[0], 'local-user')) + local_nested = saved() + (original / 'hosts.yml').write_text(local_nested.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) + verify('nested invalid local token never falls back', not nested_api(HOSTS[0])) + (original / 'hosts.yml').write_text(local_nested) + verify('nested local logout restores forwarding', nested_gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user').returncode == 0 and nested_api(HOSTS[0])) + verify('nested alias preparation', gh('alias', 'set', 'probe-guard', '!test -z "$DEVSY_GH_AUTH_PROBE" && gh api user --jq .login').returncode == 0) + r = nested_gh('probe-guard') + verify('local-only marker does not leak into aliases or their children', r.returncode == 0 and r.stdout.strip() == 'host-user') + verify('nested alias removed', gh('alias', 'delete', 'probe-guard').returncode == 0) + (original / 'hosts.yml').write_text(registrations) + verify('tokenless registrations use host active accounts', all(api(h) for h in HOSTS)) verify('real auth status ignores hidden tokenless registrations', gh('auth', 'status').returncode == 0) verify('unchanged invocation preserves exact dotfiles registrations', saved() == registrations and no_forwarded_saved()) @@ -179,12 +217,14 @@ def last_diagnostic(): r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['local']) verify('keyring login succeeds without plaintext token', r.returncode == 0 and secrets[HOSTS[1]]['local'] not in saved() and api(HOSTS[1], 'local-user')) verify('existing keyring credential excluded from forwarding', last_diagnostic()['hosts'][1]['source'] == 'keyring') + verify('nested lookup preserves keyring authentication', nested_api(HOSTS[1], 'local-user') and last_diagnostic()['hosts'][1]['source'] == 'keyring') verify('unavailable keyring backend follows gh no-token result', api(HOSTS[1], env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/nonexistent'}) and last_diagnostic()['hosts'][1]['source'] == 'forwarded') verify('unavailable backend leaves local keyring intact', api(HOSTS[1], 'local-user')) revoked = ROOT / 'revoke-local' revoked.touch() try: verify('expired keyring credential never falls back', not api(HOSTS[1]) and last_diagnostic()['hosts'][1]['source'] == 'keyring') + verify('nested expired keyring credential never falls back', not nested_api(HOSTS[1]) and last_diagnostic()['hosts'][1]['source'] == 'keyring') finally: revoked.unlink() forbidden = ROOT / 'forbid-local' @@ -192,6 +232,8 @@ def last_diagnostic(): try: r = gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) verify('insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr and last_diagnostic()['hosts'][1]['source'] == 'keyring') + r = nested_gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) + verify('nested insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr and last_diagnostic()['hosts'][1]['source'] == 'keyring') finally: forbidden.unlink() r = gh('auth', 'status', '--hostname', HOSTS[1]) diff --git a/pkg/ghcredentials/wrapper.go b/pkg/ghcredentials/wrapper.go index 361aa9790d..e099fcdbd2 100644 --- a/pkg/ghcredentials/wrapper.go +++ b/pkg/ghcredentials/wrapper.go @@ -14,7 +14,10 @@ import ( "time" ) -const sessionEnv = "DEVSY_GH_SESSION" +const ( + sessionEnv = "DEVSY_GH_SESSION" + authProbeEnv = "DEVSY_GH_AUTH_PROBE" +) // Run is the wrapper entry point. args starts with the wrapper's path, followed // by untouched gh arguments. It bypasses Devsy's CLI/env/log initialization. @@ -27,7 +30,10 @@ func Run(args []string) int { return 127 } env := os.Environ() - if active := os.Getenv(sessionEnv); active != "" && active == os.Getenv("GH_CONFIG_DIR") { + active := os.Getenv(sessionEnv) + // A local lookup may re-enter through another dispatcher. Keep forwarding + // disabled throughout that process tree, even if it changes GH_CONFIG_DIR. + if os.Getenv(authProbeEnv) == "1" || (active != "" && active == os.Getenv("GH_CONFIG_DIR")) { return runGH(binary, args[1:], env, nil).Code } inv := &invocation{binary: binary, args: args[1:], env: env, diag: beginDiagnostic(binary)} From 4681196597f6e51ed3d9f126d4ca35c3e3aad69d Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 14:06:29 +0900 Subject: [PATCH 04/20] docs(credentials): remove redundant and inaccurate comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove the fixed-size diagnostic record claim and source-derived package and installer summaries. Keep runtime behavior unchanged and defer the inactive keyring-account behavior change pending discussion. Independent doc delta review accepted the six-line deletion; git diff --check passed. ## Transcript - U1: > 全部修正して - A: Inspected the current branch and the affected authentication path; no authentication code changes had been made when the user narrowed scope. - U2: > 2は修正。1はどういうこと？ - A: Explained that an inactive keyed account can be hidden by the forwarding overlay when active and legacy unkeyed credentials are absent, breaking logout despite preserving the keyring token. Deferred finding 1. Deleted only the three comment blocks from finding 2, checked the diff and obtained independent doc delta approval. Co-authored-by: Codex --- pkg/ghcredentials/config.go | 2 -- pkg/ghcredentials/diagnostics.go | 2 -- pkg/ghcredentials/install.go | 2 -- 3 files changed, 6 deletions(-) diff --git a/pkg/ghcredentials/config.go b/pkg/ghcredentials/config.go index cf6c6be77f..6cbde24e1f 100644 --- a/pkg/ghcredentials/config.go +++ b/pkg/ghcredentials/config.go @@ -1,5 +1,3 @@ -// Package ghcredentials supplies temporary GitHub CLI authentication without -// putting relayed credentials in the user's configuration. package ghcredentials import ( diff --git a/pkg/ghcredentials/diagnostics.go b/pkg/ghcredentials/diagnostics.go index 198b23e09d..9240be3127 100644 --- a/pkg/ghcredentials/diagnostics.go +++ b/pkg/ghcredentials/diagnostics.go @@ -113,8 +113,6 @@ func (d *diagnostic) finish() { d.Finished = time.Now() d.save(true) entries, _ := os.ReadDir(filepath.Dir(d.path)) - // Fixed-size records and a count bound cap local disk usage. Incomplete - // records identify killed runs; no remote diagnostic export exists. sort.Slice(entries, func(i, j int) bool { a, b := entries[i], entries[j] successA := diagnosticSuccess(filepath.Join(filepath.Dir(d.path), a.Name())) diff --git a/pkg/ghcredentials/install.go b/pkg/ghcredentials/install.go index ae37820894..61a2d1fdc3 100644 --- a/pkg/ghcredentials/install.go +++ b/pkg/ghcredentials/install.go @@ -12,8 +12,6 @@ import ( const WrapperDir = "/usr/local/share/devsy/gh/bin" -// Install owns only its private executable and a system profile fragment. The -// user's gh binaries, configuration, and shell dotfiles are never replaced. func Install(port int) error { binary, err := os.Executable() if err != nil { From 0fedae90eafc73e30bb63a7131a9e94ee1d30d40 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 14:46:21 +0900 Subject: [PATCH 05/20] fix(credentials): install gh relay wrapper on the standard PATH MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Install the POSIX wrapper at /usr/local/bin/gh and preserve an existing executable or symlink as gh.devsy-original in the same directory. Serialize installation, stage the wrapper before moving the original, and roll back a failed publication. Reinstallation retains the original backup; a replacement gh refreshes it without creating a wrapper chain. Resolve gh in PATH order, substituting the fixed backup at the wrapper's position and continuing the search when absent. Execute the selected absolute path. Remove private PATH/profile setup while preserving the existing internal entry point, authentication overlay, output and configuration writeback. The isolated real-gh fixture starts Devsy's container SSH server before wrapper installation, then uses non-PTY sessions for normal commands. It verifies the renamed real binary, earlier-PATH selection, local/keyring precedence, aliases, extensions, parallel updates and interruption recovery. Integration passed; related unit tests, race tests and standard cli:lint:ci passed. Independent code and doc reviews accepted the changes. Repo launcher changes are proposal-only; no live workspace, host authentication, push or release was changed. ## Transcript - U1: > # 実装依頼 > > ## 依頼 > `/Users/a13400/orca/workspaces/devsy/gh-relay` の現在のcheckoutで、GitHub CLI credentials relayのwrapperを `/usr/local/bin/gh` に配置する方式へ変更してください。 > > 既存ghは固定backupへ退避し、本体の選択は以下のPATH順の仕様に従ってください。認証転送・設定overlay・書き戻しの既存方式は維持してください。 > > ## 目的 > - 通常の対話・非対話実行で、専用ディレクトリへのPATH追加に依存せずホスト認証を利用できるようにする。 > - 既存ghと、repoがmiseで選ぶghの版・lazy導入を維持する。 > - wrapperの再設置で呼出しのネストを増やさない。 > > ## 成功条件 > - `/usr/local/bin/gh` に短いPOSIX shell wrapperを設置する。 > - 既存ghがある場合、同じディレクトリの固定backup先へ退避する。 > - 自分のwrapperを再度backupしない。何度設置しても `wrapper → gh本体` の構造を維持する。 > - gh更新によってwrapperが置き換わった場合、次回のcredentialsサービス起動時に、新しい本体で固定backupを更新してwrapperを再設置する。 > - Go側はPATHを順に探索し、wrapperより前に別のghがあれば、その本体を選ぶ。 > - 探索が自分のwrapperへ到達した場合、固定backupがあれば、その絶対パスを選ぶ。なければ後続のPATHを探索する。 > - 選択後のgh起動は絶対パスで行い、PATHによる再解決をしない。 > - `/usr/local/bin` に元のghがなくても、後続のPATHにあるghを利用できる。gh未導入だけを理由にDevsyの接続を失敗させない。 > - Devsyが起動した既存SSH serverの非PTYセッションでも、通常のPATHからwrapperが選ばれることを検証する。初回起動の設置タイミングも確認する。 > - repo launcherがmiseで用意したghのディレクトリをPATHの先頭に置けば、そのghが選ばれる。 > - 既存の認証優先順位、stdout・stderr・終了コード、設定保存・symlink保持・並行更新・異常終了後の回復を維持する。 > - 実ghと隔離したダミー認証で検証し、ローカル検証と未検証の実環境境界を区別して報告する。 > - repo側launcherは編集せず、必要な変更内容を報告する。 > > ## 決定事項 > - wrapperの公開入口を `/usr/local/bin/gh` にする。 > - 理由: 標準的な既存PATHを利用し、profileや起動後の環境更新への依存を減らすため。 > - 主な棄却案: 専用ディレクトリを各セッションのPATHへ追加する方式。shell・profile・起動経路ごとの反映確認が必要になるため。 > - 既存ghは、同じディレクトリの固定backup先1つへ退避する。 > - 理由: 相対symlinkの基準ディレクトリを維持し、世代別backupやwrapperのネストを避けるため。 > - 主な棄却案: 世代別backup、wrapper自身のbackup。 > - wrapperはPOSIX shellで、設置時のDevsyの絶対パスへ委譲する処理だけを持つ。 > - 理由: 通常のDevsy初期化を避ける既存入口とGo実装を活用し、shellへの処理重複を避けるため。 > - 主な棄却案: shell側で本体探索・認証処理・コマンド分類を行う方式。 > - 委譲先は既存の `devsy internal gh "$0" "$@"` 相当の専用入口とする。 > - 理由: repoがPATH上で別版のDevsyを選んでいても、relayを提供するDevsyを呼ぶため。 > - 主な棄却案: PATHからDevsyを探索する方式。 > - backupの有無は、Go側が実行時に固定backupファイルを確認して判断する。 > - 理由: 設置結果はファイル自体から判断でき、別のmetadataや本体指定引数が不要なため。 > - 主な棄却案: `--executable` の追加、backup情報を持つ別設定ファイル。 > - gh本体はPATH順を尊重し、自分のwrapperの位置でbackupを候補にする。 > - 理由: 通常入口ではbackupを使い、repo launcherではmiseが選んだghを使えるため。 > - 主な棄却案: backupの無条件優先。repoの版指定が効かなくなるため。 > - gh更新への追従は次回のcredentialsサービス起動時に行う。 > - 理由: 既存の設置処理で対応でき、常時監視の保守負担を避けるため。 > - 主な棄却案: 更新の常時監視と即時再設置。 > - 専用wrapperディレクトリへのPATH追加と、そのためのprofile設置は廃止する。 > - 理由: 新しい配置方式では不要なため。 > - 主な棄却案: 新旧方式の併存。 > - 対象は一般的なgh利用での自然な認証転送とする。 > - 理由: すべての特殊な認証状態を保証することより、通常利用の信頼性と保守負担を重視するため。 > - 主な棄却案: 特殊なkeyring状態まで網羅する追加処理。 > > ## 制約 > - ghの引数によるコマンド分類、対象ホストの推定、失敗したコマンド全体の自動再実行を行わない。 > - ghのstdout・stderr・終了コードを加工しない。 > - ホストの認証ストアを変更しない。転送した認証を元のコンテナ設定へ保存しない。 > - 明示された認証用環境変数と通常のローカル認証を優先し、期限切れ・権限不足を理由にホスト認証へ切り替えない。 > - Devsyにmise専用処理や依存を追加しない。 > - ghの自動インストールをDevsyへ追加しない。 > - 既存のghを識別せずに上書き・削除しない。backupを自分のwrapperで置き換えない。 > - 任意のprofile・ツール管理によるPATH変更に対する優先順位は保証しない。 > - repo側launcher、dotfiles、実workspaceは変更しない。 > - 旧wrapper配置との互換処理は追加しない。旧wrapperが残る環境への移行対応は不要。 > - トークンをログ・検証成果物へ含めない。 > - push、PR作成、release、VM再起動・再作成、agent更新は行わない。 > > ## 対象外 > - gh更新の常時監視。 > - 任意のツール管理ソフトによるPATH変更への自動対応。 > - active accountの認証と互換用tokenがなく、非アクティブなアカウントの個別keyring認証だけがある特殊状態への追加対応。 > - 同一トークンの再保存を独立した認証保存として必ず識別すること。 > - ホストでのghインストール・ログイン・アカウント変更。 > - repo側launcherの編集・適用。 > > ## 未検証の前提 > - 現在のcheckoutにおけるwrapper設置とSSH・IDE起動の順序、および初回コマンド実行との関係。 > - 固定backupとwrapperの安全な識別・再設置・更新方法。並行設置と失敗途中の復旧も確認すること。 > - 実ghを別名のbackupとして起動した場合の挙動。 > - 既存ghが通常ファイル・絶対symlink・相対symlinkの場合の退避動作。 > - 現在の隔離fixtureが、ログインシェルを使わない実行と本体選択を検証するために必要な変更箇所。 > - 対象実workspaceにおける非PTY経路での動作。過去の起動直後の失敗と今回のPATH問題が同一原因かは未確認。 > > ## 実装時の指示 > - 設計合意時のHEADは `468119659`、branchは `atty303/gh-relay`。開始時に現在の状態を確認する。 > - 主な変更対象: > - `pkg/ghcredentials/install.go` > - `pkg/ghcredentials/wrapper.go` の本体探索 > - 必要に応じて `main.go` の専用入口 > - 関連する設置・本体選択テストと実gh fixture > - shellからGoへの委譲では、現在と同様に設置時のDevsy絶対パスとrelayポートを埋め込む。 > - Goからghへは子プロセスとして実行し、終了後の書き戻しを維持する。 > - 固定backup名は、例として `/usr/local/bin/gh.devsy-original` がある。具体名は実装内で統一する。 > - 現在の実gh検証taskは `mise run cli:test:gh-relay`。fixtureは多くの呼出しで `bash -lc` を使っているため、その成功だけで非PTY経路の検証済みとしない。 > - 既知の `TestSnapshotWorkspacePolicyDoesNotFilterOtherMount` の失敗は、最新main単体の隔離ソースでも再現済み。今回の回帰と区別する。 > - 現在のrepo launcherは `/Users/a13400/src/CyberAgentAI/glaze/glaze/.local/bin/gh`。miseの `github-cli` をlazy導入し、そのbinディレクトリをPATHの先頭へ追加してから、`/usr/local/share/devsy/gh/bin/gh` へ委譲している。新方式に合わせた委譲先の変更案だけを提示する。 > - branch diffに対しての互換維持をしない - A: Implemented the standard-path wrapper and single fixed backup in the current checkout. Added safe staged installation, installation locking and backup-aware PATH selection with no private-path compatibility. Added executable/symlink, repeat/update, concurrency, interrupted move, failure preservation and backend-selection tests. Extended the real-gh fixture with an already-running Devsy SSH server and non-PTY execution. Fixed fixture HOME forwarding and test-only path normalization/lint findings. Verified integration, related packages, race checks and standard lint; independent code and doc reviewers accepted the final delta. Prepared and checked the repo launcher proposal without editing the repo, dotfiles or live workspaces. Co-authored-by: Codex --- pkg/ghcredentials/install.go | 118 ++++++++++--- pkg/ghcredentials/install_test.go | 159 ++++++++++++++++++ pkg/ghcredentials/testdata/fixture.py | 16 +- pkg/ghcredentials/testdata/ssh.go | 57 +++++++ pkg/ghcredentials/wrapper.go | 6 + pkg/ghcredentials/wrapper_integration_test.go | 16 +- 6 files changed, 343 insertions(+), 29 deletions(-) create mode 100644 pkg/ghcredentials/install_test.go create mode 100644 pkg/ghcredentials/testdata/ssh.go diff --git a/pkg/ghcredentials/install.go b/pkg/ghcredentials/install.go index 61a2d1fdc3..494e882a78 100644 --- a/pkg/ghcredentials/install.go +++ b/pkg/ghcredentials/install.go @@ -1,48 +1,122 @@ package ghcredentials import ( + "context" + "errors" "fmt" + "io" "os" "path/filepath" - "strings" + "time" "al.essio.dev/pkg/shellescape" - "github.com/devsy-org/devsy/pkg/envfile" ) -const WrapperDir = "/usr/local/share/devsy/gh/bin" +const ( + wrapperPath = "/usr/local/bin/gh" + backupSuffix = ".devsy-original" + wrapperHeader = "#!/bin/sh\n# devsy gh relay\n" +) func Install(port int) error { binary, err := os.Executable() if err != nil { return err } - // #nosec G301 -- Public executable path; it contains no authentication data. - if err := os.MkdirAll(WrapperDir, 0o755); err != nil { + return installWrapper(wrapperPath, binary, port) +} + +func installWrapper(path, binary string, port int) error { + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + dir := filepath.Dir(path) + // #nosec G301 -- Public executable directory without authentication data. + if err := os.MkdirAll(dir, 0o755); err != nil { return err } + l, err := lock(ctx, filepath.Join(dir, ".devsy-gh-install.lock")) + if err != nil { + return err + } + defer func() { _ = l.Close() }() + + staged, err := stageWrapper(dir, binary, port) + if err != nil { + return err + } + defer func() { _ = os.Remove(staged) }() + moved, err := backupOriginal(path) + if err != nil { + return err + } + if err := os.Rename(staged, path); err != nil { + if moved { + return errors.Join(err, os.Rename(path+backupSuffix, path)) + } + return err + } + return nil +} + +func stageWrapper(dir, binary string, port int) (name string, err error) { script := fmt.Sprintf( - "#!/bin/sh\n# devsy gh relay\nexport DEVSY_CREDENTIALS_SERVER_PORT=%d\nexec %s internal gh \"$0\" \"$@\"\n", + wrapperHeader+"export DEVSY_CREDENTIALS_SERVER_PORT=%d\nexec %s internal gh \"$0\" \"$@\"\n", port, shellescape.Quote(binary), ) - //nolint:gosec // This is the executable wrapper, containing no credentials. - if err := os.WriteFile(filepath.Join(WrapperDir, "gh"), []byte(script), 0o755); err != nil { - return err + f, err := os.CreateTemp(dir, ".devsy-gh-wrapper-*") + if err != nil { + return "", err } - // Devsy SSH/IDE sessions read envfile; login shells also need the wrapper. - path := os.Getenv("PATH") - if !strings.HasPrefix(path, WrapperDir+":") { - path = WrapperDir + ":" + path + name = f.Name() + defer func() { + _ = f.Close() + if err != nil { + _ = os.Remove(name) + } + }() + if _, err = f.WriteString(script); err != nil { + return name, err } - envfile.MergeAndApply(map[string]string{"PATH": path}) - // #nosec G301 -- System shell profiles must be readable by workspace users. - if err := os.MkdirAll("/etc/profile.d", 0o755); err != nil { - return err + if err = f.Chmod(0o755); err != nil { + return name, err + } + err = f.Close() + return name, err +} + +func backupOriginal(path string) (bool, error) { + if _, err := os.Lstat(path); errors.Is(err, os.ErrNotExist) { + return false, nil + } else if err != nil { + return false, err + } + own, err := isWrapper(path) + if err != nil || own { + return false, err + } + if _, err := executableInfo(path); err != nil { + return false, err + } + if err := os.Rename(path, path+backupSuffix); err != nil { + return false, err + } + return true, nil +} + +func isWrapper(path string) (bool, error) { + f, err := resolvedFile(path) + if err != nil { + return false, err + } + defer func() { _ = f.Close() }() + header := make([]byte, len(wrapperHeader)) + _, err = io.ReadFull(f, header) + if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) { + return false, nil + } + if err != nil { + return false, err } - profile := fmt.Sprintf(`# devsy gh relay -case ":$PATH:" in *:%s:*) ;; *) export PATH=%s:"$PATH" ;; esac -`, WrapperDir, WrapperDir) - // #nosec G306 -- Public PATH configuration, containing no credentials. - return os.WriteFile("/etc/profile.d/devsy-gh.sh", []byte(profile), 0o644) + return string(header) == wrapperHeader, nil } diff --git a/pkg/ghcredentials/install_test.go b/pkg/ghcredentials/install_test.go new file mode 100644 index 0000000000..c29866c28a --- /dev/null +++ b/pkg/ghcredentials/install_test.go @@ -0,0 +1,159 @@ +package ghcredentials + +import ( + "os" + "path/filepath" + "sync" + "testing" + + "github.com/stretchr/testify/require" +) + +const ( + testOriginalName = "real-gh" + testExitScript = "#!/bin/sh\nexit 0" + testRegularFile = "file" + testRelativeLink = "relative symlink" + testInstallBinary = "/devsy" +) + +func writeTestExecutable(t *testing.T, path string, data []byte) { + t.Helper() + dir, err := os.OpenRoot(filepath.Dir(path)) + require.NoError(t, err) + defer func() { _ = dir.Close() }() + require.NoError(t, dir.WriteFile(filepath.Base(path), data, 0o700)) +} + +func TestInstallWrapperPreservesOriginalAndUpdates(t *testing.T) { + for _, kind := range []string{testRegularFile, "absolute symlink", testRelativeLink} { + t.Run(kind, func(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "gh") + target := filepath.Join(dir, testOriginalName) + first := []byte("#!/bin/sh\nprintf first") + writeTestExecutable(t, target, first) + switch kind { + case testRegularFile: + writeTestExecutable(t, path, first) + case "absolute symlink": + require.NoError(t, os.Symlink(target, path)) + case testRelativeLink: + require.NoError(t, os.Symlink(testOriginalName, path)) + } + require.NoError(t, installWrapper(path, "/devsy first", 12049)) + backup := path + backupSuffix + original, err := readFile(backup) + require.NoError(t, err) + require.Equal(t, first, original) + if kind != testRegularFile { + link, err := os.Readlink(backup) + require.NoError(t, err) + if kind == testRelativeLink { + require.Equal(t, testOriginalName, link) + } else { + require.Equal(t, target, link) + } + } + require.NoError(t, installWrapper(path, "/devsy second", 12050)) + original, err = readFile(backup) + require.NoError(t, err) + require.Equal(t, first, original) + script, err := readFile(path) + require.NoError(t, err) + require.Contains(t, string(script), "12050") + require.Contains(t, string(script), "'/devsy second'") + require.NoError(t, os.Remove(path)) + second := []byte("#!/bin/sh\nprintf second") + writeTestExecutable(t, path, second) + require.NoError(t, installWrapper(path, testInstallBinary, 12049)) + original, err = readFile(backup) + require.NoError(t, err) + require.Equal(t, second, original) + }) + } +} + +func TestInstallWrapperWithoutOriginalAndInterruptedMove(t *testing.T) { + path := filepath.Join(t.TempDir(), "gh") + require.NoError(t, installWrapper(path, testInstallBinary, 12049)) + _, err := os.Lstat(path + backupSuffix) + require.ErrorIs(t, err, os.ErrNotExist) + require.NoError(t, os.Remove(path)) + original := []byte("#!/bin/sh\nprintf preserved") + writeTestExecutable(t, path+backupSuffix, original) + require.NoError(t, installWrapper(path, testInstallBinary, 12049)) + got, err := readFile(path + backupSuffix) + require.NoError(t, err) + require.Equal(t, original, got) +} + +func TestInstallWrapperConcurrent(t *testing.T) { + path := filepath.Join(t.TempDir(), "gh") + original := []byte("#!/bin/sh\nprintf original") + writeTestExecutable(t, path, original) + var wg sync.WaitGroup + errs := make(chan error, 8) + for range 8 { + wg.Go(func() { errs <- installWrapper(path, testInstallBinary, 12049) }) + } + wg.Wait() + close(errs) + for err := range errs { + require.NoError(t, err) + } + got, err := readFile(path + backupSuffix) + require.NoError(t, err) + require.Equal(t, original, got) +} + +func TestInstallWrapperFailurePreservesOriginal(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "gh") + original := []byte("#!/bin/sh\nprintf original") + writeTestExecutable(t, path, original) + require.NoError(t, os.Mkdir(path+backupSuffix, 0o700)) + require.Error(t, installWrapper(path, testInstallBinary, 12049)) + got, err := readFile(path) + require.NoError(t, err) + require.Equal(t, original, got) + entries, err := filepath.Glob(filepath.Join(dir, ".devsy-gh-wrapper-*")) + require.NoError(t, err) + require.Empty(t, entries) +} + +func TestRealBinaryRespectsPATHAndBackup(t *testing.T) { + root, err := filepath.EvalSymlinks(t.TempDir()) + require.NoError(t, err) + early := filepath.Join(root, "early") + standard := filepath.Join(root, "standard") + late := filepath.Join(root, "late") + for _, dir := range []string{early, standard, late} { + require.NoError(t, os.Mkdir(dir, 0o700)) + } + wrapper := filepath.Join(standard, "gh") + require.NoError(t, installWrapper(wrapper, testInstallBinary, 12049)) + later := filepath.Join(late, "gh") + writeTestExecutable(t, later, []byte(testExitScript)) + t.Setenv("PATH", standard+string(os.PathListSeparator)+late) + got, err := realBinary(wrapper) + require.NoError(t, err) + require.Equal(t, later, got) + backup := wrapper + backupSuffix + writeTestExecutable(t, backup, []byte(testExitScript)) + got, err = realBinary(wrapper) + require.NoError(t, err) + require.Equal(t, backup, got) + selected := filepath.Join(early, "gh") + writeTestExecutable(t, selected, []byte(testExitScript)) + t.Setenv("PATH", early+string(os.PathListSeparator)+standard+string(os.PathListSeparator)+late) + got, err = realBinary(wrapper) + require.NoError(t, err) + require.Equal(t, selected, got) + require.NoError(t, os.Remove(selected)) + require.NoError(t, os.Remove(backup)) + require.NoError(t, os.Link(wrapper, backup)) + got, err = realBinary(wrapper) + require.NoError(t, err) + require.Equal(t, later, got) +} diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py index b90d901d73..5b25e9c2eb 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/pkg/ghcredentials/testdata/fixture.py @@ -90,11 +90,15 @@ def gh(*args, token=None, env=None): e = dict(os.environ) if env: e.update(env) - return subprocess.run(['/bin/bash', '-lc', 'exec gh "$@"', '--', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + return subprocess.run(['/fixture/ssh-client', *args], input=token, text=True, capture_output=True, env=e, timeout=20) def api(host, expected='host-user', **kw): r = gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) ok = r.returncode == 0 and r.stdout.strip() == expected return ok + verify('SSH server predates wrapper installation', (ROOT / 'pre-install-gh').read_text().strip() == '/usr/local/bin/gh') + verify('standard wrapper installed', Path('/usr/local/bin/gh').read_text().startswith('#!/bin/sh\n# devsy gh relay\n')) + verify('real gh preserved in fixed backup', Path('/usr/local/bin/gh.devsy-original').read_bytes() == (ROOT / 'gh').read_bytes()) + verify('no dedicated profile installed', not Path('/etc/profile.d/devsy-gh.sh').exists()) host_before = (ROOT / 'host-gh' / 'hosts.yml').read_bytes() original = ROOT / 'guest-gh' original.mkdir(exist_ok=True) @@ -120,13 +124,13 @@ def last_diagnostic(): dispatcher = ROOT / 'dispatcher' dispatcher.mkdir() shim = dispatcher / 'gh' - shim.write_text('#!/bin/sh\nexport PATH=/usr/local/bin:/usr/bin:/bin\nif [ -n "$GH_TEST_CONFIG_OVERRIDE" ]; then export GH_CONFIG_DIR="$GH_TEST_CONFIG_OVERRIDE"; fi\nexec /usr/local/share/devsy/gh/bin/gh "$@"\n') + shim.write_text('#!/bin/sh\nexport PATH=/usr/bin:/usr/local/bin:/bin\nif [ -n "$GH_TEST_CONFIG_OVERRIDE" ]; then export GH_CONFIG_DIR="$GH_TEST_CONFIG_OVERRIDE"; fi\nexec /usr/local/bin/gh "$@"\n') shim.chmod(0o755) def nested_gh(*args, token=None, env=None): e = dict(os.environ, PATH=str(dispatcher) + ':/usr/local/bin:/usr/bin:/bin') if env: e.update(env) - return subprocess.run(['/usr/local/share/devsy/gh/bin/gh', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + return subprocess.run(['/usr/local/bin/gh', *args], input=token, text=True, capture_output=True, env=e, timeout=20) def nested_api(host, expected='host-user', **kw): r = nested_gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) return r.returncode == 0 and r.stdout.strip() == expected @@ -153,6 +157,12 @@ def nested_api(host, expected='host-user', **kw): (original / 'hosts.yml').write_text(registrations) verify('tokenless registrations use host active accounts', all(api(h) for h in HOSTS)) + verify('non-PTY SSH uses fixed real-gh backup', last_diagnostic()['binary'] == '/usr/local/bin/gh.devsy-original') + selected = ROOT / 'selected-gh' + selected.mkdir() + (selected / 'gh').symlink_to('/usr/bin/gh') + r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin'), text=True, capture_output=True, timeout=20) + verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and last_diagnostic()['binary'] == str(selected / 'gh')) verify('real auth status ignores hidden tokenless registrations', gh('auth', 'status').returncode == 0) verify('unchanged invocation preserves exact dotfiles registrations', saved() == registrations and no_forwarded_saved()) decisions = last_diagnostic()['hosts'] diff --git a/pkg/ghcredentials/testdata/ssh.go b/pkg/ghcredentials/testdata/ssh.go new file mode 100644 index 0000000000..0e146ac882 --- /dev/null +++ b/pkg/ghcredentials/testdata/ssh.go @@ -0,0 +1,57 @@ +package main + +import ( + "errors" + "fmt" + "os" + "time" + + "al.essio.dev/pkg/shellescape" + "golang.org/x/crypto/ssh" +) + +func main() { + if err := run(); err != nil { + var exit *ssh.ExitError + if errors.As(err, &exit) { + os.Exit(exit.ExitStatus()) + } + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} + +func run() error { + config := &ssh.ClientConfig{User: "root", HostKeyCallback: ssh.InsecureIgnoreHostKey(), Timeout: time.Second} //nolint:gosec // Network-disabled disposable fixture server. + var client *ssh.Client + var err error + for range 50 { + client, err = ssh.Dial("tcp", "127.0.0.1:12050", config) + if err == nil { + break + } + time.Sleep(100 * time.Millisecond) + } + if err != nil { + return err + } + defer func() { _ = client.Close() }() + session, err := client.NewSession() + if err != nil { + return err + } + defer func() { _ = session.Close() }() + for _, name := range []string{"HOME", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_PROMPT_DISABLED", "GH_BROWSER", "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "DBUS_SESSION_BUS_ADDRESS", "DEVSY_GH_DIAGNOSTICS"} { + if value, ok := os.LookupEnv(name); ok { + if err := session.Setenv(name, value); err != nil { + return err + } + } + } + session.Stdout, session.Stderr = os.Stdout, os.Stderr + if len(os.Args) == 2 && os.Args[1] == "--probe" { + return session.Run("command -v gh") + } + session.Stdin = os.Stdin + return session.Run(shellescape.QuoteCommand(append([]string{"gh"}, os.Args[1:]...))) +} diff --git a/pkg/ghcredentials/wrapper.go b/pkg/ghcredentials/wrapper.go index af3cc6e7d0..0d1a72051c 100644 --- a/pkg/ghcredentials/wrapper.go +++ b/pkg/ghcredentials/wrapper.go @@ -131,6 +131,7 @@ func (inv *invocation) withSession(s *session) int { } func realBinary(wrapper string) (string, error) { + installed, _ := filepath.EvalSymlinks(wrapper) self, _ := os.Stat(wrapper) for _, dir := range filepath.SplitList(os.Getenv("PATH")) { path := filepath.Join(dir, "gh") @@ -139,6 +140,11 @@ func realBinary(wrapper string) (string, error) { continue } if self != nil && os.SameFile(self, info) { + backup := installed + backupSuffix + original, err := executableInfo(backup) + if err == nil && !os.SameFile(self, original) { + return filepath.Abs(backup) + } continue } return filepath.Abs(path) diff --git a/pkg/ghcredentials/wrapper_integration_test.go b/pkg/ghcredentials/wrapper_integration_test.go index 7fa675717e..ac6afc9f7b 100644 --- a/pkg/ghcredentials/wrapper_integration_test.go +++ b/pkg/ghcredentials/wrapper_integration_test.go @@ -27,7 +27,7 @@ import ( "github.com/stretchr/testify/require" ) -//go:embed testdata/Dockerfile testdata/fixture.py +//go:embed testdata/Dockerfile testdata/fixture.py testdata/ssh.go var fixtureFiles embed.FS func TestRealGitHubCLI(t *testing.T) { @@ -116,7 +116,7 @@ func prepareFixture(t *testing.T, ctx context.Context, dir string) string { t.Helper() repo, err := filepath.Abs("../..") require.NoError(t, err) - for _, name := range []string{"Dockerfile", "fixture.py"} { + for _, name := range []string{"Dockerfile", "fixture.py", "ssh.go"} { b, err := fixtureFiles.ReadFile("testdata/" + name) require.NoError(t, err) root, err := os.OpenRoot(dir) @@ -143,6 +143,11 @@ func prepareFixture(t *testing.T, ctx context.Context, dir string) string { build.Env = append(os.Environ(), "GOOS=linux", "GOARCH="+runtime.GOARCH, "CGO_ENABLED=0") out, err := build.CombinedOutput() require.NoError(t, err, "build: %s", out) + sshBuild := exec.CommandContext(ctx, "go", "build", "-o", filepath.Join(dir, "ssh-client"), filepath.Join(dir, "ssh.go")) + sshBuild.Dir, sshBuild.Env = repo, build.Env + sshOut, err := sshBuild.CombinedOutput() + require.NoError(t, err, "build ssh fixture: %s", sshOut) + downloadGH(t, ctx, dir) return name } @@ -204,13 +209,16 @@ func startFixture( t.Helper() image := name + ":test" command := `cp /fixture/gh /usr/bin/gh +cp /fixture/gh /usr/local/bin/gh cp /fixture/devsy /usr/local/bin/devsy mkdir -p /root/.local/share/keyrings -export PATH=/fixture:$PATH +export PATH=/usr/local/bin:/usr/bin:/bin exec dbus-run-session -- sh -c ' printf "%s" "$DBUS_SESSION_BUS_ADDRESS" > /fixture/bus printf fixture-password | gnome-keyring-daemon --unlock >/dev/null 2>/tmp/keyring.log python3 /fixture/fixture.py server >/dev/null 2>/tmp/api.log & + /usr/local/bin/devsy internal agent container ssh-server --address 127.0.0.1:12050 >/tmp/ssh.log 2>&1 & + /fixture/ssh-client --probe > /fixture/pre-install-gh /usr/local/bin/devsy internal agent container credentials-server --user root --configure-github-helper sleep 600 '` @@ -264,7 +272,7 @@ func waitFixture(t *testing.T, ctx context.Context, name string, done <-chan err name, "sh", "-c", - "test -f /fixture/api-ready && test -f /usr/local/share/devsy/gh/bin/gh", + "test -f /fixture/api-ready && test -f /usr/local/bin/gh.devsy-original", ) if c.Run() == nil { ready = true From de8c8ce77bf93fd2ad55711640ac396d94f854a1 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 15:46:28 +0900 Subject: [PATCH 06/20] fix(credentials): preserve relay entry point in gh self references MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Default GH_PATH to /usr/local/bin/gh in container gh child environments, including fallback and nested execution, while preserving explicit values. Keep real executable selection unchanged. This lets gh auth setup-git persist the relay helper without Devsy classifying commands or rewriting Git config. Record the effective self path and its source in container diagnostics without inventing a container default for host authentication lookups. Extend isolated real-gh coverage to saved Git credential helpers, aliases, explicit and empty GH_PATH, and host/container diagnostic separation. No live workspace or host credentials are changed. ## Transcript - U1: > branchをPRを受け取るdevsyメンテナの気持ちでフレッシュレビューして > --- > **Title:** `feat(credentials): forward host GitHub CLI authentication` > > Allow ordinary `gh` commands, aliases, and extensions in a workspace to use the host's active GitHub account without a separate container login. This extends the existing credentials relay to GitHub.com and multiple GitHub Enterprise hosts, with host selection handled by `gh`. > > Forwarding is enabled by default and can be disabled independently with `SSH_INJECT_GITHUB_CREDENTIALS=false`. Explicit authentication environment variables and local credentials take precedence; tokenless `user`/`users` settings do not prevent forwarding. If forwarding is unavailable, the wrapper runs `gh` with its normal local configuration. > > Forwarded tokens stay in temporary configuration. Container login and configuration changes are persisted without saving relay-provided tokens or changing the host's configuration. The wrapper preserves `gh` output and exit status. > > Validation: > > - `mise run cli:lint:ci`: passed. > - Isolated integration with real gh 2.100.0, dummy credentials, mock GitHub/GHE endpoints, and Linux keyring: passed. Covers local-auth precedence, login/logout, aliases/extensions, concurrent changes, symlinks, and interruption recovery. > - Related package tests passed. `TestSnapshotWorkspacePolicyDoesNotFilterOtherMount` fails identically on unmodified `origin/main`. > - Public GitHub/GHE services and the macOS keychain were not exercised by the integration suite. - A: Reviewed the branch against the supplied PR contract. Identified that setup-git could persist a backing gh path and bypass the relay in later Git operations. No changes were made during review. - U2: > 他の指摘はないのか？ - A: Inspected further ordinary gh behavior and reproduced the persisted-helper authentication failure using isolated real gh and dummy credentials. - U3: > setup-gitに対応するとして、wrapper側でのgit configの再編集なしでは対応できないよな？ - A: Investigated official GH_PATH behavior in gh 2.95.0 and 2.100.0. An isolated experiment confirmed that gh itself can save a wrapper helper and later Git can retrieve forwarded credentials, without Devsy editing Git config. - U4: > setup-gitだけでなく共通にしても問題ないか？ - A: Confirmed common child-environment use preserves backend selection and explicit GH_PATH, while leaving host authentication lookup unchanged. - U5: > 対応して - A: Implemented common child GH_PATH defaulting and regression checks. Unit tests and isolated real-gh integration passed, including actual setup-git and subsequent git credential fill. Independent code review requested diagnostic separation for container defaults and host inputs; addressed and accepted. Final isolated integration passed in 69.44 seconds; cli:lint:ci passed with zero issues. No docs, launcher, live workspace, push or release changes. Co-authored-by: Codex --- pkg/ghcredentials/diagnostics.go | 10 +++++++++- pkg/ghcredentials/testdata/fixture.py | 22 ++++++++++++++++++++++ pkg/ghcredentials/testdata/ssh.go | 2 +- pkg/ghcredentials/wrapper.go | 3 +++ 4 files changed, 35 insertions(+), 2 deletions(-) diff --git a/pkg/ghcredentials/diagnostics.go b/pkg/ghcredentials/diagnostics.go index 9240be3127..84d1786055 100644 --- a/pkg/ghcredentials/diagnostics.go +++ b/pkg/ghcredentials/diagnostics.go @@ -68,6 +68,10 @@ func beginDiagnostic(binary string) *diagnostic { d.path = f.Name() _ = f.Close() d.Inputs = credentialInputs() + if _, explicit := os.LookupEnv("GH_PATH"); !explicit { + d.Inputs["GH_PATH"] = wrapperPath + d.Inputs["GH_PATH_SOURCE"] = "default" + } d.save(false) return d } @@ -162,7 +166,11 @@ func (d *diagnostic) observe(name string, started time.Time, err error) { } func credentialInputs() map[string]string { - inputs := map[string]string{} + inputs := map[string]string{"GH_PATH_SOURCE": "unset"} + if path, explicit := os.LookupEnv("GH_PATH"); explicit { + inputs["GH_PATH"] = path + inputs["GH_PATH_SOURCE"] = "environment" + } for _, name := range []string{ "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "HOME", "APPDATA", diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py index 5b25e9c2eb..dcdafc854b 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/pkg/ghcredentials/testdata/fixture.py @@ -163,6 +163,28 @@ def nested_api(host, expected='host-user', **kw): (selected / 'gh').symlink_to('/usr/bin/gh') r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin'), text=True, capture_output=True, timeout=20) verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and last_diagnostic()['binary'] == str(selected / 'gh')) + git_config = Path.home() / '.gitconfig' + previous_git_config = git_config.read_bytes() if git_config.exists() else None + try: + verify('setup-git accepts forwarded authentication', gh('auth', 'setup-git').returncode == 0) + helper = subprocess.run(['git', 'config', '--global', '--get-all', 'credential.https://github.com.helper'], text=True, capture_output=True, timeout=20) + verify('setup-git persists the relay wrapper', helper.returncode == 0 and '/usr/local/bin/gh auth git-credential' in helper.stdout and 'gh.devsy-original' not in helper.stdout) + credentials = subprocess.run(['git', 'credential', 'fill'], input='protocol=https\nhost=github.com\n\n', env=dict(os.environ, GIT_TERMINAL_PROMPT='0'), text=True, capture_output=True, timeout=20) + verify('persisted Git helper retrieves forwarded authentication', credentials.returncode == 0 and ('password=' + secrets[HOSTS[0]]['forwarded']) in credentials.stdout) + finally: + if previous_git_config is None: + git_config.unlink(missing_ok=True) + else: + git_config.write_bytes(previous_git_config) + verify('self-path alias saves', gh('alias', 'set', 'self-path', '!printf "%s" "$GH_PATH"').returncode == 0) + verify('aliases inherit relay self-path', gh('self-path').stdout == '/usr/local/bin/gh') + verify('default self-path retained in diagnostics', last_diagnostic()['inputs']['GH_PATH'] == '/usr/local/bin/gh' and last_diagnostic()['inputs']['GH_PATH_SOURCE'] == 'default') + verify('container self-path does not alter host inputs', last_diagnostic()['host_context']['inputs']['GH_PATH_SOURCE'] == 'unset' and 'GH_PATH' not in last_diagnostic()['host_context']['inputs']) + verify('explicit self-path is preserved', gh('self-path', env={'GH_PATH': '/fixture/custom-gh'}).stdout == '/fixture/custom-gh') + verify('explicit self-path retained in diagnostics', last_diagnostic()['inputs']['GH_PATH'] == '/fixture/custom-gh' and last_diagnostic()['inputs']['GH_PATH_SOURCE'] == 'environment') + verify('explicit empty self-path is preserved', gh('self-path', env={'GH_PATH': ''}).stdout == '') + verify('explicit empty self-path retained in diagnostics', last_diagnostic()['inputs']['GH_PATH'] == '' and last_diagnostic()['inputs']['GH_PATH_SOURCE'] == 'environment') + verify('self-path alias removed', gh('alias', 'delete', 'self-path').returncode == 0) verify('real auth status ignores hidden tokenless registrations', gh('auth', 'status').returncode == 0) verify('unchanged invocation preserves exact dotfiles registrations', saved() == registrations and no_forwarded_saved()) decisions = last_diagnostic()['hosts'] diff --git a/pkg/ghcredentials/testdata/ssh.go b/pkg/ghcredentials/testdata/ssh.go index 0e146ac882..ea8c54a046 100644 --- a/pkg/ghcredentials/testdata/ssh.go +++ b/pkg/ghcredentials/testdata/ssh.go @@ -41,7 +41,7 @@ func run() error { return err } defer func() { _ = session.Close() }() - for _, name := range []string{"HOME", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_PROMPT_DISABLED", "GH_BROWSER", "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "DBUS_SESSION_BUS_ADDRESS", "DEVSY_GH_DIAGNOSTICS"} { + for _, name := range []string{"HOME", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_PATH", "GH_PROMPT_DISABLED", "GH_BROWSER", "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "DBUS_SESSION_BUS_ADDRESS", "DEVSY_GH_DIAGNOSTICS"} { if value, ok := os.LookupEnv(name); ok { if err := session.Setenv(name, value); err != nil { return err diff --git a/pkg/ghcredentials/wrapper.go b/pkg/ghcredentials/wrapper.go index 0d1a72051c..aa75b75f36 100644 --- a/pkg/ghcredentials/wrapper.go +++ b/pkg/ghcredentials/wrapper.go @@ -28,6 +28,9 @@ func Run(args []string) int { return 127 } env := os.Environ() + if _, explicit := os.LookupEnv("GH_PATH"); !explicit { + env = replaceEnv(env, "GH_PATH", wrapperPath) + } active := os.Getenv(sessionEnv) // A local lookup may re-enter through another dispatcher. Keep forwarding // disabled throughout that process tree, even if it changes GH_CONFIG_DIR. From 15fb6687c45026c08b290503c61adc6e8ca74402 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 15:54:03 +0900 Subject: [PATCH 07/20] chore(credentials): remove redundant relay test comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove comments that repeat the adjacent test and fixture source. Keep runtime and test behavior unchanged. Python AST equality and whitespace checks passed; independent doc delta review accepted the changes. No gh-specific startup readiness handling is added: service ordering is shared with existing relays. ## Transcript - U1: > branchをPRを受け取るdevsyメンテナの気持ちでフレッシュレビューして > --- > **Title:** `feat(credentials): forward host GitHub CLI authentication` - A: Reviewed the full branch against origin/main. Related package tests, lint and isolated real-gh integration passed; mechanical merge had no conflicts. Reported initial service-readiness ordering and redundant comments; live workspace behavior was not tested. - U2: > P3は対応 - A: Removed only four redundant comment lines at the three reviewed locations. Python AST is unchanged; diff whitespace check passed. Independent doc delta review accepted the removal. No implementation or test behavior changed. - U3: > P2はdevsyの他のserviceも同じではないか？ それならghだけ特別対応しなくていい - A: Confirmed Git and Docker authentication share the asynchronous credentials-service lifecycle. Accepted the user scope decision to avoid gh-specific readiness handling and removed this common lifecycle issue from this PR merge-blocker assessment. Kept the code change limited to P3 comments. Co-authored-by: Codex --- pkg/ghcredentials/session_test.go | 1 - pkg/ghcredentials/testdata/fixture.py | 3 --- 2 files changed, 4 deletions(-) diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index c9af256928..3c86342ffa 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -165,7 +165,6 @@ func TestDanglingDotfilesSymlinkKeepsPendingChanges(t *testing.T) { func TestDisabledWritebackDiagnosticsDoNotParseConfiguration(t *testing.T) { var summary *writebackSummary - // Invalid YAML would fail diagnostic parsing if the disabled sink were entered. require.NoError(t, summary.inspect(configFile, []byte("["), []byte("["), []byte("["))) } diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py index dcdafc854b..ac9baf4a5b 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/pkg/ghcredentials/testdata/fixture.py @@ -111,7 +111,6 @@ def last_diagnostic(): return json.loads(max(folder.glob('*.json'), key=lambda f: f.stat().st_mtime_ns).read_text()) - # Dotfiles contain account preferences, never authentication. registrations = ( 'github.com:\n git_protocol: https\n user: local-user\n users:\n local-user:\n' 'enterprise.one.test:\n git_protocol: https\n user: host-user\n oauth_token: \"\"\n users:\n host-user:\n oauth_token: \"\"\n' @@ -119,8 +118,6 @@ def last_diagnostic(): ) (original / 'hosts.yml').write_text(registrations) - # A third-party dispatcher can return to Devsy after resolving the real gh. - # Exercise both entries without depending on any particular tool manager. dispatcher = ROOT / 'dispatcher' dispatcher.mkdir() shim = dispatcher / 'gh' From e21936eacb9ee9f4ca3f850df064ccba31032696 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 16:45:39 +0900 Subject: [PATCH 08/20] refactor(credentials): use standard logging and command dispatch for gh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Register gh as an internal Cobra passthrough command and restore the ordinary main entry point. Keep common CLI initialization while avoiding envfile.Apply and passing all gh arguments and exit status through unchanged. Configure the existing logger with DEVSY_LOG_LEVEL for stderr debug/trace, warnings for relay and overlay faults, and errors for pending configuration writeback. Normal absence of host authentication and local-auth precedence remain debug events. Remove the independent diagnostic recorder, retention, configuration switch, RPC context payload and writeback summaries. Preserve operational session metadata, token stripping, configuration merging, ownership and recovery. Remove comments describing the deleted recorder and bypass entry point. Validation: six related package suites and ghcredentials race tests passed; cli:lint:ci reported zero issues. The isolated real gh 2.100.0 integration passed in 62.73 seconds, covering Cobra flags/help, envfile isolation, logging levels, stdout/exit preservation, warning/error fallback, secret exclusion, existing authentication and configuration lifecycle behavior. Independent code and doc reviews accepted the fixed snapshot. Public GitHub/GHE, macOS Keychain and live workspace startup ordering were not exercised. No remote changes were made. ## Transcript - U1: > # 実装依頼 > > ## 依頼 > `/Users/a13400/orca/workspaces/devsy/gh-relay` の現在のcheckoutで、GitHub CLI credentials relayの独自diagnosticsを既存loggerへ置き換え、`main.go`の専用分岐を通常のCobra internalコマンドへ移してください。 > > ## 目的 > - gh認証転送を既存Devsyのログ・コマンド登録方式へ揃える。 > - 独自の診断保存機構と起動特例の保守負担を削減する。 > - 転送や設定保存の故障を通常実行でも利用者へ知らせる。 > > ## 成功条件 > - 独自のJSON診断ファイル、保持管理、`DEVSY_GH_DIAGNOSTICS`、診断専用の集計・収集処理を削除する。 > - wrapperのログは既存の`pkg/log`を使い、warning/errorを通常実行時にもstderrへ出す。 > - `DEVSY_LOG_LEVEL=debug`／`trace`で詳細ログを有効にできる。 > - ホストのgh未導入・未ログイン、明示環境変数やローカル認証による転送抑止はdebugとして扱う。 > - 転送対象があるのにrelay通信・overlay作成などが失敗した場合はwarning、設定の書き戻し失敗はerrorとして扱う。 > - gh自身の非ゼロ終了には追加のwarning/errorを重ねない。 > - トークン、認証内容を含む設定、relayレスポンス全体をログへ出さない。 > - `main.go`を通常の`cmd.Execute()`入口へ戻し、ghの委譲をCobraのinternalコマンドとして登録する。 > - ghの引数をDevsyのフラグとして解析せず、そのまま渡す。gh自身のhelp・alias・拡張も維持する。 > - envfileによって呼出元の認証環境を変更しない。 > - ghのstdoutと終了コードを維持する。stderrにはDevsyの診断ログが追加され得る。 > - 既存の認証優先順位、複数ホスト、PATH順の本体選択、固定backup、GH_PATH、設定overlay・書き戻し、symlink保持、並行更新、入れ子実行、異常終了後の回復を維持する。 > - 実ghと隔離したダミー認証で機能を検証し、ログレベルごとの出力・秘密情報の非出力を確認する。 > - ローカル検証と未検証の実環境境界を分けて報告する。 > > ## 決定事項 > - 独自diagnosticsを削除し、既存の`pkg/log`へ置き換える。 > - 理由: Devsy共通の操作方法と保守対象に揃えるため。 > - 主な棄却案: 常時JSON保存の維持、opt-inの専用ファイル出力、共通loggerへの新しい出力基盤追加。今回の目的に対して保守負担が大きいため。 > - stderrは通常の使いやすさの範囲で診断に使用する。 > - 理由: stdoutを維持すれば、warning/errorと明示的なdebugログを利用者へ届けられるため。 > - 主な棄却案: ghのstderrも完全に無加工とする方式。診断保存機構や別経路が必要になるため。 > - debugの有効化には既存の`DEVSY_LOG_LEVEL`を使う。 > - 理由: 専用設定を増やさず既存のログレベルに揃えるため。 > - 主な棄却案: `DEVSY_GH_DEBUG`などの専用環境変数。 > - 正常な転送抑止と、転送処理の故障を区別する。 > - 理由: 任意機能としての不在は静かに扱い、転送できるはずなのに失敗した場合は通知するため。 > - 主な棄却案: 転送不能をすべてwarningにする方式、故障もすべてdebugに隠す方式。 > - ghを通常のCobra internalコマンドとして登録する。 > - 理由: `main.go`のgh専用分岐をなくし、既存の委譲コマンドと同じ構造にするため。 > - 主な棄却案: `main.go`での先行dispatch維持、初期化を全面回避する汎用plumbing基盤の新設。 > - 共通処理による設定参照・logger設定・telemetry初期化は許容する。 > - 理由: 全面回避はghの透過的利用に必要な要件ではないため。 > - 主な棄却案: Devsyの共通初期化をすべて回避する方式。 > - 回復に必要なsession情報は維持する。 > - 理由: 独自の診断記録と、設定保存・回復に必要な機能上の状態は用途が異なるため。 > - 主な棄却案: 診断削除と一緒にsession metadataまで削除する方式。 > - gh専用の起動待機は追加しない。 > - 理由: credentialsサービスの非同期起動はGit・Dockerなどにも共通する既存のライフサイクルであり、このPRでghだけ特別扱いしないため。 > - 主な棄却案: wrapper設置とrelay受付をgh専用に待機する方式。 > > ## 制約 > - 現在のcheckoutで作業する。開始時のbranch・HEAD・未確定変更を確認する。 > - ホスト判定、gh引数によるコマンド分類、失敗したghコマンド全体の自動再実行を追加しない。 > - ghのstdoutと終了コードを変更しない。 > - ログのためにトークンや設定内容を表示・保存しない。既存redactionに加え、ログへ渡す値自体を安全な項目に限定する。 > - ホストの認証ストア、dotfiles、repo側launcher、実workspaceを変更しない。 > - mise専用処理・依存、ghの自動インストール、旧branch実装との互換処理を追加しない。 > - push、PR作成、release、VM再起動・再作成、agent更新を行わない。 > > ## 対象外 > - 共通loggerやtelemetry基盤の再設計。 > - machine daemon用diagnosticsへのgh記録の統合。 > - credentialsサービス全体の起動順序の変更。 > - 特殊な非アクティブアカウントだけのkeyring状態への追加対応。 > - 同一トークンの再保存を独立した認証保存として必ず識別すること。 > - repo側launcherの編集・適用。 > > ## 未検証の前提 > - Cobraの通常経路へ移した際の、gh引数・help・終了コード・入れ子実行の具体的な扱い。 > - 共通初期化と専用hookの組合せで、envfile適用を避けながら既存ログ設定を利用する方法。 > - 独自diagnosticsの削除に伴い、機能上必要な状態・エラー処理・回復処理から切り離すべき型と処理の範囲。 > - 現在のfixtureから診断JSONへの依存を除去し、実際の挙動とstderrで検証するための変更箇所。 > > ## 実装時の指示 > - 合意時のbranchは`atty303/gh-relay`、HEADは`15fb6687c`。開始時に再確認する。 > - 主な確認対象: > - `main.go` > - `cmd/root.go` > - `cmd/internal/internal.go` > - `cmd/internal/runtime_supervisor.go` > - `cmd/internal/agent.go` > - `pkg/log/logger.go` > - `pkg/ghcredentials/diagnostics.go` > - `pkg/ghcredentials/writeback.go` > - `pkg/ghcredentials/wrapper.go` > - `pkg/ghcredentials/host.go` > - `pkg/ghcredentials/result.go` > - `pkg/ghcredentials/session.go` > - 関連テストと`pkg/ghcredentials/testdata/fixture.py` > - 既存の`internal runtime-supervisor`は`DisableFlagParsing`と専用`PersistentPreRunE`を持つ。既存の構造を参照し、ghのための初期化全面回避機構は追加しない。 > - `agentPreRunE`は`envfile.Apply()`を呼ぶため、ghへそのまま適用しない。 > - `cmd.run()`はCobra実行前にも共通初期化を行う。これ自体を回避する必要はないが、ghの引数・終了結果への影響を確認する。 > - 独自diagnosticsにだけ必要なRPCレスポンス項目や型も、機能上の用途を確認して整理する。削除した形式との互換は維持しない。 > - fixtureの診断JSONの存在・内容を確認するassertionは、必要なものを機能の結果やログ出力の検証へ置き換える。不要になった診断機構専用テストは削除する。 > - 既存の実gh検証taskは`mise run cli:test:gh-relay`。標準lintは`mise run cli:lint:ci`。 > - 直前の検証では関連パッケージ・lint・実gh 2.100.0の隔離統合テストが成功している。実GitHub/GHE・macOS Keychain・実workspace起動順序の保証を意味しない。 > - `TestSnapshotWorkspacePolicyDoesNotFilterOtherMount`はbaseline `58162b7d2`でも再現した既知の失敗であり、今回の変更による回帰と区別する。 - A: Confirmed the clean requested checkout and fetched origin. Replaced the custom recorder with existing stderr logging and moved gh into Cobra's internal command tree. Kept operational session metadata and recovery; removed diagnostic-only data and assertions. Initial integration exposed missing DEVSY_LOG_LEVEL binding, then outdated combined-stream fallback assertions; fixed the hook and isolated stdout/stderr checks. Added real-gh envfile/argument/logging/privacy checks and command passthrough tests. Final package tests, race checks, lint and real-gh integration passed. Fresh code/doc reviews found no remaining issues. No live workspace, host credentials, dotfiles, launcher, push or release operations were performed. Co-authored-by: Codex --- cmd/internal/gh.go | 24 +++ cmd/internal/gh_test.go | 49 +++++ cmd/internal/internal.go | 1 + main.go | 12 +- pkg/ghcredentials/diagnostics.go | 201 ------------------ pkg/ghcredentials/host.go | 59 ++--- pkg/ghcredentials/process_unix.go | 17 +- pkg/ghcredentials/process_windows.go | 10 +- pkg/ghcredentials/result.go | 28 +-- pkg/ghcredentials/session.go | 143 +++++-------- pkg/ghcredentials/session_test.go | 15 +- pkg/ghcredentials/testdata/fixture.py | 117 +++++----- pkg/ghcredentials/testdata/ssh.go | 8 +- pkg/ghcredentials/wrapper.go | 76 +++---- pkg/ghcredentials/wrapper_integration_test.go | 15 +- pkg/ghcredentials/writeback.go | 131 ------------ 16 files changed, 303 insertions(+), 603 deletions(-) create mode 100644 cmd/internal/gh.go create mode 100644 cmd/internal/gh_test.go delete mode 100644 pkg/ghcredentials/diagnostics.go delete mode 100644 pkg/ghcredentials/writeback.go diff --git a/cmd/internal/gh.go b/cmd/internal/gh.go new file mode 100644 index 0000000000..f74d5e85d3 --- /dev/null +++ b/cmd/internal/gh.go @@ -0,0 +1,24 @@ +package cmdinternal + +import ( + "os" + + "github.com/devsy-org/devsy/pkg/config" + "github.com/devsy-org/devsy/pkg/ghcredentials" + "github.com/devsy-org/devsy/pkg/log" + "github.com/spf13/cobra" +) + +func NewGHCmd() *cobra.Command { + return &cobra.Command{ + Use: "gh", Hidden: true, + DisableFlagParsing: true, + PersistentPreRunE: func(*cobra.Command, []string) error { + log.Init( + log.Config{Level: os.Getenv(config.EnvLogLevel), DefaultLevel: log.DefaultLevel}, + ) + return nil + }, + Run: func(_ *cobra.Command, args []string) { os.Exit(ghcredentials.Run(args)) }, + } +} diff --git a/cmd/internal/gh_test.go b/cmd/internal/gh_test.go new file mode 100644 index 0000000000..e5d9b5c434 --- /dev/null +++ b/cmd/internal/gh_test.go @@ -0,0 +1,49 @@ +package cmdinternal + +import ( + "errors" + "testing" + + "github.com/devsy-org/devsy/cmd/flags" + "github.com/devsy-org/devsy/pkg/config" + "github.com/devsy-org/devsy/pkg/log" + "github.com/spf13/cobra" + "github.com/stretchr/testify/require" +) + +func TestGHCommandForwardsArgumentsAndConfiguresLogging(t *testing.T) { + for _, level := range []string{"", "debug", "trace"} { + t.Run(level, func(t *testing.T) { + t.Setenv(config.EnvLogLevel, level) + t.Cleanup(func() { log.Init(log.Config{DefaultLevel: log.DefaultLevel}) }) + parent := NewInternalCmd(&flags.GlobalFlags{}) + parent.PersistentPreRunE = func(*cobra.Command, []string) error { + return errors.New("parent initialization must not run") + } + args := []string{ + "/usr/local/bin/gh", + "api", + "--context", + "repo", + "--help", + "--log-level", + "invalid", + "--", + "one two", + } + command, _, err := parent.Find(append([]string{"gh"}, args...)) + require.NoError(t, err) + require.Equal(t, "gh", command.Name()) + var got []string + command.Run = func(_ *cobra.Command, args []string) { got = args } + parent.SetArgs(append([]string{"gh"}, args...)) + require.NoError(t, parent.Execute()) + require.Equal(t, args, got) + want := level + if want == "" { + want = log.DefaultLevel + } + require.Equal(t, want, log.LevelString()) + }) + } +} diff --git a/cmd/internal/internal.go b/cmd/internal/internal.go index 65c448396f..1ca6173231 100644 --- a/cmd/internal/internal.go +++ b/cmd/internal/internal.go @@ -16,6 +16,7 @@ func NewInternalCmd(globalFlags *flags.GlobalFlags) *cobra.Command { } cmd.AddCommand(NewAgentCmd(globalFlags)) cmd.AddCommand(NewRuntimeSupervisorCmd()) + cmd.AddCommand(NewGHCmd()) cmd.AddCommand(NewDaemonLocalCmd(globalFlags)) cmd.AddCommand(NewLogsDaemonCmd(globalFlags)) cmd.AddCommand(NewRunUserCommandsCmd(globalFlags)) diff --git a/main.go b/main.go index 574d4d89f0..c562ff0227 100644 --- a/main.go +++ b/main.go @@ -1,17 +1,7 @@ package main -import ( - "os" - - "github.com/devsy-org/devsy/cmd" - "github.com/devsy-org/devsy/pkg/ghcredentials" -) +import "github.com/devsy-org/devsy/cmd" func main() { - // gh must receive the caller's exact environment and streams, without - // Devsy's envfile loading, argument parsing, logging, or telemetry. - if len(os.Args) >= 4 && os.Args[1] == "internal" && os.Args[2] == "gh" { - os.Exit(ghcredentials.Run(os.Args[3:])) - } cmd.Execute() } diff --git a/pkg/ghcredentials/diagnostics.go b/pkg/ghcredentials/diagnostics.go deleted file mode 100644 index 84d1786055..0000000000 --- a/pkg/ghcredentials/diagnostics.go +++ /dev/null @@ -1,201 +0,0 @@ -package ghcredentials - -import ( - "encoding/json" - "os" - "path/filepath" - "sort" - "time" - - "github.com/devsy-org/devsy/pkg/version" -) - -type hostDecision struct { - Host string `json:"host"` - Source string `json:"source"` - Preferences bool `json:"account_preferences,omitempty"` - SameUser bool `json:"same_user,omitempty"` - Probe string `json:"token_probe,omitempty"` - ProbeMode string `json:"token_probe_mode,omitempty"` -} - -// Only these source-allowlisted fields may enter local diagnostics. Command -// arguments, config contents, accounts, error text, and token values are never -// recorded. gh owns target selection, so its arguments need no diagnostic copy. -type diagnostic struct { - Version string `json:"version"` - Started time.Time `json:"started"` - Finished time.Time `json:"finished,omitzero"` - Complete bool `json:"complete"` - Binary string `json:"binary"` - ConfigDir string `json:"config_dir,omitempty"` - Inputs map[string]string `json:"inputs"` - Hosts []hostDecision `json:"hosts,omitempty"` - Stage string `json:"stage"` - Code int `json:"exit_code"` - Signal int `json:"signal,omitempty"` - HostStatus string `json:"host_status,omitempty"` - HostContext HostContext `json:"host_context"` - HostDecisions []hostDecision `json:"host_decisions,omitempty"` - Operations []operation `json:"operations,omitempty"` - Writeback *writebackSummary `json:"writeback,omitempty"` - Recovery []recoveryDecision `json:"recovery,omitempty"` - path string -} - -func beginDiagnostic(binary string) *diagnostic { - d := &diagnostic{ - Version: version.GetVersion(), - Binary: binary, - Started: time.Now(), - Stage: "started", - } - if os.Getenv("DEVSY_GH_DIAGNOSTICS") == "false" { - return d - } - root, err := sessionRoot() - if err != nil { - return d - } - dir := filepath.Join(root, "diagnostics") - if os.MkdirAll(dir, 0o700) != nil { - return d - } - f, err := os.CreateTemp(dir, "run-*.json") - if err != nil { - return d - } - d.path = f.Name() - _ = f.Close() - d.Inputs = credentialInputs() - if _, explicit := os.LookupEnv("GH_PATH"); !explicit { - d.Inputs["GH_PATH"] = wrapperPath - d.Inputs["GH_PATH_SOURCE"] = "default" - } - d.save(false) - return d -} - -func (d *diagnostic) run(binary string, args, env []string, owner *os.File) int { - started := time.Now() - result := runGH(binary, args, env, owner) - d.Code = result.Code - d.Signal = result.Signal - if d.path != "" { - op := operation{ - Name: "gh.execute", - Status: "ok", - DurationMS: time.Since(started).Milliseconds(), - ErrorType: result.ErrorType, - } - if result.Code != 0 { - op.Status = "error" - } - if result.Signal != 0 { - op.Status = "cancelled" - } - d.Operations = append(d.Operations, op) - } - return d.Code -} - -func (d *diagnostic) save(complete bool) { - if d.path == "" { - return - } - d.Complete = complete - b, err := json.Marshal(d) - if err == nil { - _ = os.WriteFile(d.path, b, 0o600) - } -} - -func (d *diagnostic) finish() { - if d.path == "" { - return - } - d.Finished = time.Now() - d.save(true) - entries, _ := os.ReadDir(filepath.Dir(d.path)) - sort.Slice(entries, func(i, j int) bool { - a, b := entries[i], entries[j] - successA := diagnosticSuccess(filepath.Join(filepath.Dir(d.path), a.Name())) - successB := diagnosticSuccess(filepath.Join(filepath.Dir(d.path), b.Name())) - if successA != successB { - return successA - } - ai, _ := a.Info() - bi, _ := b.Info() - return ai != nil && bi != nil && ai.ModTime().Before(bi.ModTime()) - }) - for len(entries) > 64 { - _ = os.Remove(filepath.Join(filepath.Dir(d.path), entries[0].Name())) - entries = entries[1:] - } -} - -func diagnosticSuccess(path string) bool { - b, err := readFile(path) - if err != nil { - return false - } - var d diagnostic - if json.Unmarshal(b, &d) != nil { - return false - } - return d.Complete && d.Code == 0 && d.Stage != "writeback_pending" -} - -type operation struct { - Name string `json:"name"` - Status string `json:"status"` - ErrorType string `json:"error_type,omitempty"` - DurationMS int64 `json:"duration_ms"` -} - -func (d *diagnostic) observe(name string, started time.Time, err error) { - if d.path == "" { - return - } - op := operation{Name: name, Status: "ok", DurationMS: time.Since(started).Milliseconds()} - if err != nil { - op.Status = "error" - op.Name, op.ErrorType = describeError(name, err) - } - d.Operations = append(d.Operations, op) -} - -func credentialInputs() map[string]string { - inputs := map[string]string{"GH_PATH_SOURCE": "unset"} - if path, explicit := os.LookupEnv("GH_PATH"); explicit { - inputs["GH_PATH"] = path - inputs["GH_PATH_SOURCE"] = "environment" - } - for _, name := range []string{ - "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", - "GH_HOST", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "HOME", "APPDATA", - } { - value, present := os.LookupEnv(name) - state := "unset" - if present { - state = "empty" - if value != "" { - state = "set" - } - } - inputs[name] = state - } - return inputs -} - -func (d *diagnostic) hostResult(result HostResult) { - if d.path == "" || result.Status == "" { - return - } - op := operation{Name: "host.resolve", Status: "ok", DurationMS: result.DurationMS} - if result.Status != "available" { - op.Status = "unavailable" - op.ErrorType = result.Status - } - d.Operations = append(d.Operations, op) -} diff --git a/pkg/ghcredentials/host.go b/pkg/ghcredentials/host.go index 2d9e9f0fe4..08d36500fb 100644 --- a/pkg/ghcredentials/host.go +++ b/pkg/ghcredentials/host.go @@ -8,29 +8,27 @@ import ( "strings" "time" + "github.com/devsy-org/devsy/pkg/log" "gopkg.in/yaml.v3" ) +const hostNoActiveAccounts = "no_active_accounts" + type Account struct { Host string `json:"host"` User string `json:"user"` Token string `json:"token"` } -func HostAccounts(ctx context.Context) (result HostResult) { - started := time.Now() - info := HostContext{Inputs: credentialInputs()} - defer func() { result.Context = info; result.DurationMS = time.Since(started).Milliseconds() }() +func HostAccounts(ctx context.Context) HostResult { binary, err := exec.LookPath("gh") if err != nil { return HostResult{Status: "gh_not_found"} } - info.Binary = binary dir, err := ConfigDir() if err != nil { return HostResult{Status: "config_path_failed"} } - info.ConfigDir = dir probe, err := os.MkdirTemp("", "devsy-gh-host-") if err != nil { return HostResult{Status: "config_snapshot_" + errorKind(err)} @@ -56,38 +54,51 @@ func collectHostAccounts( hosts *yaml.Node, env []string, ) HostResult { - result := HostResult{Status: "no_active_accounts"} + result := HostResult{Status: hostNoActiveAccounts} for i := 0; i+1 < len(hosts.Content); i += 2 { host := hosts.Content[i].Value u := get(hosts.Content[i+1], "user") if u == nil || u.Value == "" { - result.Decisions = append( - result.Decisions, - hostDecision{Host: host, Source: "no_active_account"}, - ) + log.Debugw("gh relay: host account unavailable", "host", host) continue } - token, err := resolveToken(ctx, binary, host, env) + account, err := (Account{Host: host, User: u.Value}).resolveHostToken(ctx, binary, env) if err != nil { - result.Decisions = append( - result.Decisions, - hostDecision{Host: host, Source: "token_" + errorKind(err)}, - ) - continue - } - if token != "" { - result.Accounts = append( - result.Accounts, - Account{Host: host, User: u.Value, Token: token}, - ) + result.Status = "token_lookup_failed" + } else if account.valid() { + result.Accounts = append(result.Accounts, account) } } - if len(result.Accounts) > 0 { + if len(result.Accounts) > 0 && result.Status == hostNoActiveAccounts { result.Status = "available" } return result } +func (a Account) resolveHostToken( + ctx context.Context, + binary string, + env []string, +) (Account, error) { + token, err := resolveToken(ctx, binary, a.Host, env) + if err != nil { + if noTokenResult(err) { + log.Debugw("gh relay: host account unavailable", "host", a.Host) + return Account{}, nil + } + log.Warnw( + "gh relay: host token lookup failed", + "host", + a.Host, + "error_type", + errorKind(err), + ) + return Account{}, err + } + a.Token = token + return a, nil +} + func resolveToken(ctx context.Context, binary, host string, env []string) (string, error) { ctx, cancel := context.WithTimeout(ctx, 3*time.Second) defer cancel() diff --git a/pkg/ghcredentials/process_unix.go b/pkg/ghcredentials/process_unix.go index af9ba73ff6..a369413168 100644 --- a/pkg/ghcredentials/process_unix.go +++ b/pkg/ghcredentials/process_unix.go @@ -7,9 +7,11 @@ import ( "os/exec" "os/signal" "syscall" + + "github.com/devsy-org/devsy/pkg/log" ) -func runGH(binary string, args, env []string, owner *os.File) processResult { +func runGH(binary string, args, env []string, owner *os.File) int { //nolint:gosec // The user chose this executable and these unchanged arguments. cmd := exec.Command(binary, args...) cmd.Env, cmd.Stdin, cmd.Stdout, cmd.Stderr = env, os.Stdin, os.Stdout, os.Stderr @@ -20,22 +22,19 @@ func runGH(binary string, args, env []string, owner *os.File) processResult { signal.Notify(signals, os.Interrupt, syscall.SIGTERM, syscall.SIGHUP) defer signal.Stop(signals) if err := cmd.Start(); err != nil { - return processResult{Code: 126, ErrorType: errorKind(err)} + log.Errorw("gh relay: executable unavailable", "error_type", errorKind(err)) + return 126 } done := make(chan struct{}) defer close(done) go forwardSignals(cmd.Process, signals, done) if err := cmd.Wait(); err != nil { if status, ok := cmd.ProcessState.Sys().(syscall.WaitStatus); ok && status.Signaled() { - return processResult{ - Code: 128 + int(status.Signal()), - Signal: int(status.Signal()), - ErrorType: "signal", - } + return 128 + int(status.Signal()) } - return processResult{Code: cmd.ProcessState.ExitCode(), ErrorType: "exit_nonzero"} + return cmd.ProcessState.ExitCode() } - return processResult{} + return 0 } func forwardSignals(process *os.Process, signals <-chan os.Signal, done <-chan struct{}) { diff --git a/pkg/ghcredentials/process_windows.go b/pkg/ghcredentials/process_windows.go index 8ae0263bc8..6c6b076340 100644 --- a/pkg/ghcredentials/process_windows.go +++ b/pkg/ghcredentials/process_windows.go @@ -4,6 +4,7 @@ import ( "os" "os/exec" + "github.com/devsy-org/devsy/pkg/log" "github.com/gofrs/flock" ) @@ -24,15 +25,16 @@ func tryOwner(path string) (*ownerLock, bool, error) { func (l *ownerLock) Close() error { return l.lock.Close() } -func runGH(binary string, args, env []string, _ *os.File) processResult { +func runGH(binary string, args, env []string, _ *os.File) int { //nolint:gosec // The user chose this executable and these unchanged arguments. cmd := exec.Command(binary, args...) cmd.Env, cmd.Stdin, cmd.Stdout, cmd.Stderr = env, os.Stdin, os.Stdout, os.Stderr if err := cmd.Run(); err != nil { if cmd.ProcessState != nil { - return processResult{Code: cmd.ProcessState.ExitCode(), ErrorType: "exit_nonzero"} + return cmd.ProcessState.ExitCode() } - return processResult{Code: 126, ErrorType: errorKind(err)} + log.Errorw("gh relay: executable unavailable", "error_type", errorKind(err)) + return 126 } - return processResult{} + return 0 } diff --git a/pkg/ghcredentials/result.go b/pkg/ghcredentials/result.go index 99786e3630..91ba9baa59 100644 --- a/pkg/ghcredentials/result.go +++ b/pkg/ghcredentials/result.go @@ -6,23 +6,13 @@ import ( "os" "os/exec" "syscall" + + "github.com/devsy-org/devsy/pkg/log" ) -// HostResult separates an unavailable authentication source from a broken -// transport. Only Accounts contains secrets; Status and Decisions are safe for -// the local diagnostic recorder. type HostResult struct { - Accounts []Account `json:"accounts"` - Status string `json:"status"` - Decisions []hostDecision `json:"decisions,omitempty"` - Context HostContext `json:"context"` - DurationMS int64 `json:"duration_ms"` -} - -type HostContext struct { - Binary string `json:"binary,omitempty"` - ConfigDir string `json:"config_dir,omitempty"` - Inputs map[string]string `json:"inputs,omitempty"` + Accounts []Account `json:"accounts"` + Status string `json:"status"` } type boundaryError struct { @@ -83,7 +73,11 @@ func describeError(name string, err error) (string, string) { return name, errorKind(err) } -type processResult struct { - Code, Signal int - ErrorType string +func logFailure(message, operation string, err error, saving bool) { + operation, kind := describeError(operation, err) + if saving { + log.Errorw(message, "operation", operation, "error_type", kind) + } else { + log.Warnw(message, "operation", operation, "error_type", kind) + } } diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index 26256bd5bc..d71a1c853a 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -14,17 +14,16 @@ import ( "strings" "time" + "github.com/devsy-org/devsy/pkg/log" "github.com/gofrs/flock" "gopkg.in/yaml.v3" ) type session struct { - Original string `json:"original"` - Accounts []Account `json:"accounts"` - dir string - lock *ownerLock - decisions []hostDecision - writeback *writebackSummary + Original string `json:"original"` + Accounts []Account `json:"accounts"` + dir string + lock *ownerLock } func sessionRoot() (string, error) { @@ -61,7 +60,7 @@ func startSession(ctx context.Context, original string, accounts []Account) (*se return nil, boundary("session.catalog_lock", err) } defer func() { _ = l.Close() }() - recoverSessions(ctx, root, false) + recoverSessions(ctx, root) return newSession(ctx, root, original, accounts) } @@ -155,60 +154,45 @@ func copyEntry(ctx context.Context, src, dst string, depth int) error { return os.WriteFile(dst, data, mode) } -type recoveryDecision struct { - Session string `json:"session"` - ConfigDir string `json:"config_dir,omitempty"` - Outcome string `json:"outcome"` - Operation string `json:"operation,omitempty"` - ErrorType string `json:"error_type,omitempty"` - Writeback *writebackSummary `json:"writeback,omitempty"` -} - -func recoverSessions(ctx context.Context, root string, recording bool) []recoveryDecision { - var results []recoveryDecision - entries, _ := os.ReadDir(root) +func recoverSessions(ctx context.Context, root string) { + entries, err := os.ReadDir(root) + if err != nil { + logFailure("gh relay: recovery pending", "recovery.list", err, true) + return + } for _, e := range entries { if e.IsDir() && strings.HasPrefix(e.Name(), "session-") { - results = append(results, recoverSession(ctx, filepath.Join(root, e.Name()), recording)) + dir := filepath.Join(root, e.Name()) + if err := recoverSession(ctx, dir); err != nil { + logFailure("gh relay: recovery pending", "recovery.writeback", err, true) + } } } - return results } -func recoverSession(ctx context.Context, dir string, recording bool) recoveryDecision { - result := recoveryDecision{Session: filepath.Base(dir), Outcome: "active"} +func recoverSession(ctx context.Context, dir string) error { owner, ok, err := tryOwner(filepath.Join(dir, "owner.lock")) if err != nil { - return recoveryFailure(result, "recovery.owner_lock", err) + return boundary("recovery.owner_lock", err) } if !ok { - return result + return nil } defer func() { _ = owner.Close() }() - return recoverUnlocked(ctx, dir, recording) -} - -func recoverUnlocked(ctx context.Context, dir string, recording bool) recoveryDecision { - result := recoveryDecision{Session: filepath.Base(dir), Outcome: "recovered"} old, err := loadSession(dir) if err != nil { - return recoveryFailure(result, "recovery.metadata", err) + return boundary("recovery.metadata", err) } - if old == nil { - _ = os.RemoveAll(dir) - return result + if old != nil { + if err := old.sync(ctx); err != nil { + return err + } } - result.ConfigDir = old.Original - if recording { - old.writeback = &writebackSummary{} + if err := os.RemoveAll(dir); err != nil { + return boundary("recovery.cleanup", err) } - err = old.sync(ctx) - result.Writeback = old.writeback - if err != nil { - return recoveryFailure(result, "recovery.writeback", err) - } - _ = os.RemoveAll(dir) - return result + log.Debug("gh relay: configuration recovered") + return nil } func loadSession(dir string) (*session, error) { @@ -227,12 +211,6 @@ func loadSession(dir string) (*session, error) { return &old, nil } -func recoveryFailure(result recoveryDecision, name string, err error) recoveryDecision { - result.Outcome = "pending" - result.Operation, result.ErrorType = describeError(name, err) - return result -} - func (s *session) save() error { b, err := json.Marshal(s) if err != nil { @@ -308,10 +286,10 @@ func (s *session) overlay( if err != nil { return boundary("config.probe_snapshot", err) } - decision := a.inject(ctx, binary, replaceEnv(env, "GH_CONFIG_DIR", probe), hosts) + source := a.inject(ctx, binary, replaceEnv(env, "GH_CONFIG_DIR", probe), hosts) _ = os.RemoveAll(probe) - s.decisions = append(s.decisions, decision) - if decision.Source == "forwarded" { + log.Debugw("gh relay: authentication source", "host", a.Host, "source", source) + if source == "forwarded" { injected = append(injected, a) } } @@ -356,13 +334,7 @@ func (s *session) strip(before, data []byte) ([]byte, error) { return nil, err } for _, a := range s.Accounts { - outcome := stripAccount(hosts, get(original, a.Host), a) - if s.writeback != nil { - s.writeback.Restorations = append( - s.writeback.Restorations, - hostRestoration{Host: a.Host, Outcome: outcome}, - ) - } + stripAccount(hosts, get(original, a.Host), a) } return yaml.Marshal(hosts) } @@ -456,38 +428,32 @@ func (a Account) inject( binary string, env []string, hosts *yaml.Node, -) hostDecision { +) string { h := get(hosts, a.Host) - decision := hostDecision{Host: a.Host, Source: "invalid_account"} if !a.valid() { - return decision - } - decision.Preferences = get(h, "user") != nil || get(h, "users") != nil - if user := get(h, "user"); user != nil { - decision.SameUser = user.Value == a.User + return "invalid_account" } if source := localAuthSource(h); source != "" { - decision.Source = source - return decision + return source } - decision.ProbeMode = "local_only" token, err := resolveToken(ctx, binary, a.Host, env) - decision.Probe = "empty" - if err != nil { - decision.Probe = errorKind(err) - if !noTokenResult(err) { - decision.Source = "local_lookup_failed" - return decision - } + if err != nil && !noTokenResult(err) { + log.Warnw( + "gh relay: forwarding unavailable", + "operation", + "local.token_lookup", + "host", + a.Host, + "error_type", + errorKind(err), + ) + return "local_lookup_failed" } if token != "" { - decision.Probe = "token_returned" - decision.Source = localTokenSource(token, env) - return decision + return localTokenSource(token, env) } a.overlay(hosts) - decision.Source = "forwarded" - return decision + return "forwarded" } // gh auth token reports an absent token as exit 1. It also collapses keyring @@ -514,11 +480,11 @@ func (a Account) overlay(hosts *yaml.Node) { put(h, "users", users) } -func stripAccount(hosts, original *yaml.Node, a Account) string { +func stripAccount(hosts, original *yaml.Node, a Account) { h := get(hosts, a.Host) if h == nil { put(hosts, a.Host, original) - return "original_host_restored" + return } u := get(h, "users") account := get(u, a.User) @@ -542,10 +508,6 @@ func stripAccount(hosts, original *yaml.Node, a Account) string { if len(h.Content) == 0 { put(hosts, a.Host, nil) } - if keyringLogin { - return "keyring_registration" - } - return "preferences_restored" } func removeForwardedToken(n, original *yaml.Node, token string) { @@ -603,7 +565,6 @@ func (s *session) syncFile(name string) error { } if bytes.Equal(before, after) { - s.writeback.unchanged() return nil } path := filepath.Join(s.Original, name) @@ -619,14 +580,10 @@ func (s *session) syncFile(name string) error { return nil } err = s.saveOriginal(name, after) - s.writeback.written(err) return boundary("config.write", err) } func (s *session) mergeFile(name string, before, after, latest []byte) ([]byte, error) { - if err := s.writeback.inspect(name, before, after, latest); err != nil { - return nil, err - } if name == hostsFile || name == configFile { return mergeYAML(before, after, latest) } diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index 3c86342ffa..f9a12ef0d0 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -35,7 +35,6 @@ func TestSessionMergePreservesDotfilesAndConcurrentChanges(t *testing.T) { s, err := startSession(context.Background(), original, nil) require.NoError(t, err) writeTestFile(t, filepath.Join(s.work(), configFile), "editor: nano\npager: less\n") - s.writeback = &writebackSummary{} writeTestFile(t, target, "editor: emacs\npager: cat\nprompt: disabled\n") require.NoError(t, s.finish(context.Background())) link, err := os.Readlink(filepath.Join(original, configFile)) @@ -48,9 +47,6 @@ func TestSessionMergePreservesDotfilesAndConcurrentChanges(t *testing.T) { require.Equal(t, "nano", get(n, "editor").Value) require.Equal(t, "cat", get(n, "pager").Value) require.Equal(t, "disabled", get(n, "prompt").Value) - require.Equal(t, 1, s.writeback.WrittenFiles) - require.Equal(t, changeCount{Modified: 1, Conflicts: 1}, s.writeback.Changes["config.editor"]) - require.NotContains(t, s.writeback.Changes, "config.pager") } func TestSessionConcurrentInvocationsAndRecovery(t *testing.T) { @@ -163,11 +159,6 @@ func TestDanglingDotfilesSymlinkKeepsPendingChanges(t *testing.T) { require.NoError(t, c.finish(context.Background())) } -func TestDisabledWritebackDiagnosticsDoNotParseConfiguration(t *testing.T) { - var summary *writebackSummary - require.NoError(t, summary.inspect(configFile, []byte("["), []byte("["), []byte("["))) -} - func TestSessionForwardsTokenlessRegistrations(t *testing.T) { for _, tc := range []struct { user, tokenField, userBody string @@ -191,9 +182,6 @@ func TestSessionForwardsTokenlessRegistrations(t *testing.T) { t.Cleanup(func() { _ = s.lock.Close() }) env := []string{"HOME=" + root, "GH_CONFIG_DIR=" + original, testPathEnv} require.NoError(t, s.inject(context.Background(), "/usr/bin/false", env)) - require.Equal(t, "forwarded", s.decisions[0].Source) - require.True(t, s.decisions[0].Preferences) - require.Equal(t, tc.user == a.User, s.decisions[0].SameUser) data, err := readFile(filepath.Join(s.work(), hostsFile)) require.NoError(t, err) n, err := decode(data) @@ -269,8 +257,7 @@ func TestTokenLookupTimeoutDoesNotAllowForwarding(t *testing.T) { a := Account{Host: testHost, User: testUser, Token: testForwardedToken} hosts := mapping() decision := a.inject(ctx, "/usr/bin/false", []string{testPathEnv}, hosts) - require.Equal(t, "local_lookup_failed", decision.Source) - require.Equal(t, "timeout", decision.Probe) + require.Equal(t, "local_lookup_failed", decision) require.Nil(t, get(hosts, a.Host)) } diff --git a/pkg/ghcredentials/testdata/fixture.py b/pkg/ghcredentials/testdata/fixture.py index ac9baf4a5b..3c33c28159 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/pkg/ghcredentials/testdata/fixture.py @@ -81,6 +81,7 @@ def check(): 'DBUS_SESSION_BUS_ADDRESS': (ROOT / 'bus').read_text().strip(), }) results = [] + stderr_outputs = [] def verify(name, ok): results.append({'check': name, 'ok': bool(ok)}) if not ok: @@ -90,7 +91,9 @@ def gh(*args, token=None, env=None): e = dict(os.environ) if env: e.update(env) - return subprocess.run(['/fixture/ssh-client', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + result = subprocess.run(['/fixture/ssh-client', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + stderr_outputs.append(result.stderr) + return result def api(host, expected='host-user', **kw): r = gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) ok = r.returncode == 0 and r.stdout.strip() == expected @@ -106,11 +109,6 @@ def saved(): return (original / 'hosts.yml').read_text() if (original / 'hosts.yml').exists() else '' def no_forwarded_saved(): return all(v['forwarded'] not in saved() for v in secrets.values()) - def last_diagnostic(): - folder = ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics' - return json.loads(max(folder.glob('*.json'), key=lambda f: f.stat().st_mtime_ns).read_text()) - - registrations = ( 'github.com:\n git_protocol: https\n user: local-user\n users:\n local-user:\n' 'enterprise.one.test:\n git_protocol: https\n user: host-user\n oauth_token: \"\"\n users:\n host-user:\n oauth_token: \"\"\n' @@ -127,13 +125,14 @@ def nested_gh(*args, token=None, env=None): e = dict(os.environ, PATH=str(dispatcher) + ':/usr/local/bin:/usr/bin:/bin') if env: e.update(env) - return subprocess.run(['/usr/local/bin/gh', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + result = subprocess.run(['/usr/local/bin/gh', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + stderr_outputs.append(result.stderr) + return result def nested_api(host, expected='host-user', **kw): r = nested_gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) return r.returncode == 0 and r.stdout.strip() == expected verify('dispatcher re-entry cannot supply forwarded authentication to local lookup', all(nested_api(h) for h in HOSTS)) verify('dispatcher re-entry preserves real auth status', nested_gh('auth', 'status').returncode == 0) - verify('dispatcher re-entry records local-only lookup', all(h['source'] == 'forwarded' and h['token_probe_mode'] == 'local_only' for h in last_diagnostic()['hosts'])) redirected = ROOT / 'redirected-gh' redirected.mkdir() (redirected / 'hosts.yml').write_text(registrations) @@ -153,13 +152,44 @@ def nested_api(host, expected='host-user', **kw): verify('nested alias removed', gh('alias', 'delete', 'probe-guard').returncode == 0) (original / 'hosts.yml').write_text(registrations) + envfile = Path('/etc/envfile.json') + previous_envfile = envfile.read_bytes() if envfile.exists() else None + try: + envfile.write_text(json.dumps({'env': {'GH_TOKEN': secrets[HOSTS[0]]['invalid']}})) + verify('Devsy envfile does not replace forwarded authentication', api(HOSTS[0])) + verify('Devsy envfile does not replace explicit authentication', api(HOSTS[0], 'local-user', env={'GH_TOKEN': secrets[HOSTS[0]]['local']})) + finally: + if previous_envfile is None: + envfile.unlink(missing_ok=True) + else: + envfile.write_bytes(previous_envfile) + + for args in [('--help',), ('--version',), ('api', '--devsy-unknown-fixture-flag'), ('api', '--context', 'sample'), ('api', '--log-level', 'invalid'), ('api', '--log-output', 'json')]: + expected = subprocess.run(['/usr/local/bin/gh.devsy-original', *args], text=True, capture_output=True, timeout=20) + r = gh(*args) + verify('real gh arguments pass through Cobra: ' + ' '.join(args), (r.returncode, r.stdout, r.stderr) == (expected.returncode, expected.stdout, expected.stderr)) + for level in ['debug', 'trace']: + r = gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': level}) + verify(level + ' enables relay logs without changing stdout', r.returncode == 0 and r.stdout == 'host-user\n' and 'gh relay: authentication source' in r.stderr and 'forwarded' in r.stderr) + r = gh('api', 'user', '--jq', '.login') + verify('normal successful forwarding is silent', r.returncode == 0 and r.stdout == 'host-user\n' and r.stderr == '') verify('tokenless registrations use host active accounts', all(api(h) for h in HOSTS)) - verify('non-PTY SSH uses fixed real-gh backup', last_diagnostic()['binary'] == '/usr/local/bin/gh.devsy-original') + try: + (original / 'hosts.yml').write_text('[invalid yaml') + expected = subprocess.run(['/usr/local/bin/gh.devsy-original', '--version'], text=True, capture_output=True, timeout=20) + r = gh('--version') + verify('overlay failure preserves gh stdout and exit', (r.returncode, r.stdout) == (expected.returncode, expected.stdout)) + verify('overlay failure warns without config content', 'gh relay: forwarding unavailable' in r.stderr and 'schema_invalid' in r.stderr) + finally: + (original / 'hosts.yml').write_text(registrations) + r = gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) + verify('non-PTY SSH uses fixed real-gh backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and '/usr/local/bin/gh.devsy-original' in r.stderr) selected = ROOT / 'selected-gh' selected.mkdir() (selected / 'gh').symlink_to('/usr/bin/gh') - r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin'), text=True, capture_output=True, timeout=20) - verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and last_diagnostic()['binary'] == str(selected / 'gh')) + r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin', DEVSY_LOG_LEVEL='debug'), text=True, capture_output=True, timeout=20) + stderr_outputs.append(r.stderr) + verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and str(selected / 'gh') in r.stderr) git_config = Path.home() / '.gitconfig' previous_git_config = git_config.read_bytes() if git_config.exists() else None try: @@ -175,18 +205,11 @@ def nested_api(host, expected='host-user', **kw): git_config.write_bytes(previous_git_config) verify('self-path alias saves', gh('alias', 'set', 'self-path', '!printf "%s" "$GH_PATH"').returncode == 0) verify('aliases inherit relay self-path', gh('self-path').stdout == '/usr/local/bin/gh') - verify('default self-path retained in diagnostics', last_diagnostic()['inputs']['GH_PATH'] == '/usr/local/bin/gh' and last_diagnostic()['inputs']['GH_PATH_SOURCE'] == 'default') - verify('container self-path does not alter host inputs', last_diagnostic()['host_context']['inputs']['GH_PATH_SOURCE'] == 'unset' and 'GH_PATH' not in last_diagnostic()['host_context']['inputs']) verify('explicit self-path is preserved', gh('self-path', env={'GH_PATH': '/fixture/custom-gh'}).stdout == '/fixture/custom-gh') - verify('explicit self-path retained in diagnostics', last_diagnostic()['inputs']['GH_PATH'] == '/fixture/custom-gh' and last_diagnostic()['inputs']['GH_PATH_SOURCE'] == 'environment') verify('explicit empty self-path is preserved', gh('self-path', env={'GH_PATH': ''}).stdout == '') - verify('explicit empty self-path retained in diagnostics', last_diagnostic()['inputs']['GH_PATH'] == '' and last_diagnostic()['inputs']['GH_PATH_SOURCE'] == 'environment') verify('self-path alias removed', gh('alias', 'delete', 'self-path').returncode == 0) verify('real auth status ignores hidden tokenless registrations', gh('auth', 'status').returncode == 0) verify('unchanged invocation preserves exact dotfiles registrations', saved() == registrations and no_forwarded_saved()) - decisions = last_diagnostic()['hosts'] - verify('registration forwarding retained in diagnostics', decisions and all(h['source'] == 'forwarded' and h.get('account_preferences') and h.get('token_probe') for h in decisions) and decisions[1].get('same_user') and not decisions[2].get('same_user')) - verify('registration restoration retained in diagnostics', all(r['outcome'] == 'preferences_restored' for r in last_diagnostic()['writeback']['restorations'])) r = gh('auth', 'logout', '--hostname', HOSTS[2], '--user', 'host-user') verify('forwarded logout preserves hidden dotfiles host', r.returncode == 0 and saved() == registrations and api(HOSTS[2])) r = gh('config', 'set', 'git_protocol', 'https', '--host', HOSTS[2]) @@ -223,10 +246,10 @@ def nested_api(host, expected='host-user', **kw): verify('all hosts forwarded through real RPC and HTTP relay', all(api(h) for h in HOSTS)) verify('real auth status', gh('auth', 'status').returncode == 0) verify('forwarding never saved', no_forwarded_saved()) - verify('unchanged authentication run recorded without user delta', not last_diagnostic()['writeback'].get('changes')) r = gh('api', 'user', '--jq', '.login', env={'GH_HOST': HOSTS[2]}) verify('gh chooses GH_HOST', r.returncode == 0 and r.stdout.strip() == 'host-user') - verify('explicit invalid GH_TOKEN never falls back', gh('api', 'user', env={'GH_TOKEN': secrets[HOSTS[0]]['invalid']}).returncode != 0) + r = gh('api', 'user', env={'GH_TOKEN': secrets[HOSTS[0]]['invalid'], 'DEVSY_LOG_LEVEL': 'debug'}) + verify('explicit invalid GH_TOKEN never falls back', r.returncode != 0 and 'gh relay: authentication source' in r.stderr and 'environment' in r.stderr) verify('explicit invalid GITHUB_TOKEN never falls back', gh('api', 'user', env={'GITHUB_TOKEN': secrets[HOSTS[0]]['invalid']}).returncode != 0) verify('explicit enterprise environment never falls back', gh('api', 'user', '--hostname', HOSTS[1], env={'GH_ENTERPRISE_TOKEN': secrets[HOSTS[1]]['invalid']}).returncode != 0) r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['forwarded']) @@ -237,6 +260,8 @@ def nested_api(host, expected='host-user', **kw): verify('logout refreshed credential', gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and api(HOSTS[0])) r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) verify('different token local login persists', r.returncode == 0 and secrets[HOSTS[0]]['local'] in saved() and api(HOSTS[0], 'local-user')) + r = gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) + verify('local authentication preference explained at debug level', r.returncode == 0 and r.stdout == 'local-user\n' and 'gh relay: authentication source' in r.stderr and 'local_config' in r.stderr) local_text = saved() (original / 'hosts.yml').write_text(local_text.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) verify('invalid local credential never falls back', not api(HOSTS[0])) @@ -246,24 +271,25 @@ def nested_api(host, expected='host-user', **kw): r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['local']) verify('keyring login succeeds without plaintext token', r.returncode == 0 and secrets[HOSTS[1]]['local'] not in saved() and api(HOSTS[1], 'local-user')) - verify('existing keyring credential excluded from forwarding', last_diagnostic()['hosts'][1]['source'] == 'keyring') - verify('nested lookup preserves keyring authentication', nested_api(HOSTS[1], 'local-user') and last_diagnostic()['hosts'][1]['source'] == 'keyring') - verify('unavailable keyring backend follows gh no-token result', api(HOSTS[1], env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/nonexistent'}) and last_diagnostic()['hosts'][1]['source'] == 'forwarded') + r = gh('api', 'user', '--hostname', HOSTS[1], '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) + verify('existing keyring credential excluded from forwarding', r.returncode == 0 and r.stdout.strip() == 'local-user' and 'gh relay: authentication source' in r.stderr and 'keyring' in r.stderr) + verify('nested lookup preserves keyring authentication', nested_api(HOSTS[1], 'local-user')) + verify('unavailable keyring backend follows gh no-token result', api(HOSTS[1], env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/nonexistent'})) verify('unavailable backend leaves local keyring intact', api(HOSTS[1], 'local-user')) revoked = ROOT / 'revoke-local' revoked.touch() try: - verify('expired keyring credential never falls back', not api(HOSTS[1]) and last_diagnostic()['hosts'][1]['source'] == 'keyring') - verify('nested expired keyring credential never falls back', not nested_api(HOSTS[1]) and last_diagnostic()['hosts'][1]['source'] == 'keyring') + verify('expired keyring credential never falls back', not api(HOSTS[1])) + verify('nested expired keyring credential never falls back', not nested_api(HOSTS[1])) finally: revoked.unlink() forbidden = ROOT / 'forbid-local' forbidden.touch() try: r = gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) - verify('insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr and last_diagnostic()['hosts'][1]['source'] == 'keyring') + verify('insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr) r = nested_gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) - verify('nested insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr and last_diagnostic()['hosts'][1]['source'] == 'keyring') + verify('nested insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr) finally: forbidden.unlink() r = gh('auth', 'status', '--hostname', HOSTS[1]) @@ -306,8 +332,6 @@ def wait_ready(): target.write_text(target.read_text().replace('editor: nano', 'editor: emacs') + '\nprompt: disabled\n') release.touch() p.communicate(timeout=20) - changes = last_diagnostic()['writeback']['changes'] - verify('successful current-wins merge recorded', changes.get('config.editor', {}).get('current_wins') == 1 and last_diagnostic()['writeback']['written_files'] > 0) verify('parallel and dotfiles changes survive', p.returncode == 0 and gh('config', 'get', 'editor').stdout.strip() == 'pending' and gh('config', 'get', 'pager').stdout.strip() == 'less' and gh('config', 'get', 'prompt').stdout.strip() == 'disabled' and config.is_symlink()) finally: if p.poll() is None: @@ -334,17 +358,18 @@ def wait_ready(): latest = target.read_text() target.write_text('[broken yaml') release.touch() - p.communicate(timeout=20) + _, writeback_stderr = p.communicate(timeout=20) + stderr_outputs.append(writeback_stderr.decode()) verify('writeback failure does not change real gh exit', p.returncode == 0) - diagnostic = last_diagnostic() - verify('writeback schema failure retained in diagnostics', diagnostic['stage'] == 'writeback_pending' and any(op['name'] == 'config.merge' and op.get('error_type') == 'schema_invalid' for op in diagnostic['operations'])) + verify('writeback failure emits actionable error', 'gh relay: configuration writeback pending' in writeback_stderr.decode() and 'schema_invalid' in writeback_stderr.decode()) finally: if latest is not None: target.write_text(latest) if p.poll() is None: os.killpg(p.pid, signal.SIGKILL) p.communicate() - verify('pending writeback recovered after original repaired', gh('config', 'get', 'editor').stdout.strip() == 'pending') + r = gh('config', 'get', 'editor', env={'DEVSY_LOG_LEVEL': 'debug'}) + verify('pending writeback recovered after original repaired', r.returncode == 0 and r.stdout.strip() == 'pending' and 'gh relay: configuration recovered' in r.stderr) ready.unlink() @@ -372,11 +397,12 @@ def wait_ready(): try: hp.write_text('{}\n') r = gh('config', 'get', 'editor') - first = last_diagnostic() + verify('no host accounts fall back without warning', r.returncode == 0 and 'gh relay:' not in r.stderr) + debug = gh('config', 'get', 'editor', env={'DEVSY_LOG_LEVEL': 'debug'}) + verify('no host accounts explained at debug level', debug.returncode == 0 and 'gh relay: host authentication' in debug.stderr and 'no_active_accounts' in debug.stderr) hp.write_text('[broken yaml') other = gh('config', 'get', 'editor') - second = last_diagnostic() - verify('same exit distinguishes host authentication failures', r.returncode == other.returncode == 0 and first['host_status'] == 'no_active_accounts' and second['host_status'] == 'config_decode_failed') + verify('host authentication failure warns without changing gh exit', other.returncode == 0 and 'gh relay: forwarding unavailable' in other.stderr and 'config_decode_failed' in other.stderr) finally: hp.write_text(host_before) text = hp.read_text().replace('host-user', 'second-user') @@ -390,23 +416,8 @@ def wait_ready(): verify('local config dir override', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '/fixture/custom-gh'}).returncode == 0 and (ROOT / 'custom-gh' / 'config.yml').exists()) verify('XDG config location', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '/fixture/custom-xdg'}).returncode == 0 and (ROOT / 'custom-xdg' / 'gh' / 'config.yml').exists()) verify('HOME config location', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '', 'HOME': '/fixture/custom-home'}).returncode == 0 and (ROOT / 'custom-home' / '.config' / 'gh' / 'config.yml').exists()) - records = list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics').glob('*.json')) - verify('bounded local diagnostics', 0 < len(records) <= 64) - verify('diagnostics contain no credential values', all(all(secret not in f.read_text() for values in secrets.values() for secret in values.values()) for f in records)) - before = {f.name for f in records} - r = gh('config', 'set', 'browser', '/bin/true', env={'DEVSY_GH_DIAGNOSTICS': 'false'}) - verify('disabled diagnostics preserve configuration writes', r.returncode == 0 and gh('config', 'get', 'browser', env={'DEVSY_GH_DIAGNOSTICS': 'false'}).stdout.strip() == '/bin/true') - verify('diagnostics opt out', api(HOSTS[2], 'second-user', env={'DEVSY_GH_DIAGNOSTICS': 'false'}) and before == {f.name for f in records[0].parent.glob('*.json')}) - - diagnostic_dir = ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics' - held = diagnostic_dir.with_name('held-diagnostics') - diagnostic_dir.rename(held) - diagnostic_dir.symlink_to('/proc/1') - try: - verify('recording failure does not change authentication', api(HOSTS[2], 'second-user')) - finally: - diagnostic_dir.unlink() - held.rename(diagnostic_dir) + verify('no diagnostic files created', not (ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics').exists()) + verify('stderr logs contain no credential values', all(secret not in output for output in stderr_outputs for values in secrets.values() for secret in values.values())) verify('normal sessions cleaned', not list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*'))) ready.unlink() diff --git a/pkg/ghcredentials/testdata/ssh.go b/pkg/ghcredentials/testdata/ssh.go index ea8c54a046..6813aba2b8 100644 --- a/pkg/ghcredentials/testdata/ssh.go +++ b/pkg/ghcredentials/testdata/ssh.go @@ -22,7 +22,11 @@ func main() { } func run() error { - config := &ssh.ClientConfig{User: "root", HostKeyCallback: ssh.InsecureIgnoreHostKey(), Timeout: time.Second} //nolint:gosec // Network-disabled disposable fixture server. + config := &ssh.ClientConfig{ + User: "root", + HostKeyCallback: ssh.InsecureIgnoreHostKey(), + Timeout: time.Second, + } //nolint:gosec // Network-disabled disposable fixture server. var client *ssh.Client var err error for range 50 { @@ -41,7 +45,7 @@ func run() error { return err } defer func() { _ = session.Close() }() - for _, name := range []string{"HOME", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_PATH", "GH_PROMPT_DISABLED", "GH_BROWSER", "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "DBUS_SESSION_BUS_ADDRESS", "DEVSY_GH_DIAGNOSTICS"} { + for _, name := range []string{"HOME", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_PATH", "GH_PROMPT_DISABLED", "GH_BROWSER", "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "DBUS_SESSION_BUS_ADDRESS", "DEVSY_LOG_LEVEL"} { if value, ok := os.LookupEnv(name); ok { if err := session.Setenv(name, value); err != nil { return err diff --git a/pkg/ghcredentials/wrapper.go b/pkg/ghcredentials/wrapper.go index aa75b75f36..20341017dc 100644 --- a/pkg/ghcredentials/wrapper.go +++ b/pkg/ghcredentials/wrapper.go @@ -12,6 +12,8 @@ import ( "strconv" "strings" "time" + + "github.com/devsy-org/devsy/pkg/log" ) const ( @@ -21,12 +23,15 @@ const ( func Run(args []string) int { if len(args) == 0 { + log.Error("gh relay: wrapper path missing") return 127 } binary, err := realBinary(args[0]) if err != nil { + log.Errorw("gh relay: executable unavailable", "error_type", errorKind(err)) return 127 } + log.Debugw("gh relay: executable selected", "binary", binary) env := os.Environ() if _, explicit := os.LookupEnv("GH_PATH"); !explicit { env = replaceEnv(env, "GH_PATH", wrapperPath) @@ -35,16 +40,15 @@ func Run(args []string) int { // A local lookup may re-enter through another dispatcher. Keep forwarding // disabled throughout that process tree, even if it changes GH_CONFIG_DIR. if os.Getenv(authProbeEnv) == "1" || (active != "" && active == os.Getenv("GH_CONFIG_DIR")) { - return runGH(binary, args[1:], env, nil).Code + return runGH(binary, args[1:], env, nil) } - inv := &invocation{binary: binary, args: args[1:], env: env, diag: beginDiagnostic(binary)} - defer inv.diag.finish() + inv := &invocation{binary: binary, args: args[1:], env: env} ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() inv.recover(ctx) session := inv.prepare(ctx) if session == nil { - return inv.diag.run(binary, inv.args, env, nil) + return runGH(binary, inv.args, env, nil) } return inv.withSession(session) } @@ -52,83 +56,71 @@ func Run(args []string) int { type invocation struct { binary string args, env []string - diag *diagnostic } func (inv *invocation) recover(ctx context.Context) { - started := time.Now() root, err := sessionRoot() if err != nil { - inv.diag.observe("session.recover", started, err) + logFailure("gh relay: recovery unavailable", "session.recover", err, false) return } l, err := lock(ctx, filepath.Join(root, "sessions.lock")) if err != nil { - inv.diag.observe("session.recover_lock", started, err) + logFailure("gh relay: recovery unavailable", "session.recover_lock", err, false) return } defer func() { _ = l.Close() }() - inv.diag.Recovery = recoverSessions(ctx, root, inv.diag.path != "") - inv.diag.observe("session.recover", started, nil) + recoverSessions(ctx, root) } func (inv *invocation) prepare(ctx context.Context) *session { - started := time.Now() result, err := fetch(ctx) - inv.diag.observe("relay.fetch", started, err) - inv.diag.HostStatus = result.Status - inv.diag.HostContext = result.Context - inv.diag.hostResult(result) - inv.diag.HostDecisions = result.Decisions - if err != nil || len(result.Accounts) == 0 { - inv.diag.Stage = "relay_unavailable" + if err != nil { + logFailure("gh relay: forwarding unavailable", "relay.fetch", err, false) + return nil + } + log.Debugw("gh relay: host authentication", "status", result.Status) + switch result.Status { + case "available", "gh_not_found", hostNoActiveAccounts, "rpc_denied": + default: + log.Warnw( + "gh relay: forwarding unavailable", + "operation", + "host.resolve", + "status", + result.Status, + ) + } + if len(result.Accounts) == 0 { return nil } original, err := ConfigDir() if err != nil { - inv.diag.Stage = "config_unavailable" + logFailure("gh relay: forwarding unavailable", "config.path", err, false) return nil } - inv.diag.ConfigDir = original - started = time.Now() s, err := startSession(ctx, original, result.Accounts) - inv.diag.observe("session.snapshot", started, err) if err != nil { - inv.diag.Stage = "snapshot_failed" + logFailure("gh relay: forwarding unavailable", "session.snapshot", err, false) return nil } - started = time.Now() if err := s.inject(ctx, inv.binary, inv.env); err != nil { - inv.diag.observe("config.overlay", started, err) + logFailure("gh relay: forwarding unavailable", "config.overlay", err, false) _ = os.RemoveAll(s.dir) _ = s.lock.Close() - inv.diag.Stage = "overlay_failed" return nil } - inv.diag.observe("config.overlay", started, nil) - inv.diag.Hosts = s.decisions return s } func (inv *invocation) withSession(s *session) int { - inv.diag.Stage = "overlay_ready" - inv.diag.save(false) env := replaceEnv(inv.env, "GH_CONFIG_DIR", s.work()) env = replaceEnv(env, sessionEnv, s.work()) - code := inv.diag.run(inv.binary, inv.args, env, s.lock.file) + code := runGH(inv.binary, inv.args, env, s.lock.file) ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() - started := time.Now() - if inv.diag.path != "" { - s.writeback = &writebackSummary{} - } - err := s.finish(ctx) - inv.diag.Writeback = s.writeback - inv.diag.observe("config.writeback", started, err) - if err != nil { - inv.diag.Stage = "writeback_pending" - } else { - inv.diag.Stage = "written_back" + if err := s.finish(ctx); err != nil { + logFailure("gh relay: configuration writeback pending", "config.writeback", err, true) } return code } diff --git a/pkg/ghcredentials/wrapper_integration_test.go b/pkg/ghcredentials/wrapper_integration_test.go index ac6afc9f7b..422f469c21 100644 --- a/pkg/ghcredentials/wrapper_integration_test.go +++ b/pkg/ghcredentials/wrapper_integration_test.go @@ -143,7 +143,14 @@ func prepareFixture(t *testing.T, ctx context.Context, dir string) string { build.Env = append(os.Environ(), "GOOS=linux", "GOARCH="+runtime.GOARCH, "CGO_ENABLED=0") out, err := build.CombinedOutput() require.NoError(t, err, "build: %s", out) - sshBuild := exec.CommandContext(ctx, "go", "build", "-o", filepath.Join(dir, "ssh-client"), filepath.Join(dir, "ssh.go")) + sshBuild := exec.CommandContext( + ctx, + "go", + "build", + "-o", + filepath.Join(dir, "ssh-client"), + filepath.Join(dir, "ssh.go"), + ) sshBuild.Dir, sshBuild.Env = repo, build.Env sshOut, err := sshBuild.CombinedOutput() require.NoError(t, err, "build ssh fixture: %s", sshOut) @@ -307,7 +314,11 @@ func verifyRelayFallback(t *testing.T, ctx context.Context, name string, stop co "GH_CONFIG_DIR=/fixture/fallback-gh gh config set editor nano && "+ "GH_CONFIG_DIR=/fixture/fallback-gh gh config get editor", ) - plain, err := fallback.CombinedOutput() + var stderr bytes.Buffer + fallback.Stderr = &stderr + plain, err := fallback.Output() require.NoError(t, err, "fallback: %s", plain) require.Equal(t, "pending\nnano\n", string(plain)) + require.Contains(t, stderr.String(), "gh relay: forwarding unavailable") + require.Contains(t, stderr.String(), "connection_refused") } diff --git a/pkg/ghcredentials/writeback.go b/pkg/ghcredentials/writeback.go deleted file mode 100644 index b48f44ec71..0000000000 --- a/pkg/ghcredentials/writeback.go +++ /dev/null @@ -1,131 +0,0 @@ -package ghcredentials - -import "gopkg.in/yaml.v3" - -// Categories are a fixed allowlist: extension paths, aliases, account names, -// arbitrary configuration keys and values must never enter diagnostics. -type writebackSummary struct { - Restorations []hostRestoration `json:"restorations,omitempty"` - UnchangedFiles int `json:"unchanged_files"` - WrittenFiles int `json:"written_files"` - Changes map[string]changeCount `json:"changes,omitempty"` -} - -type hostRestoration struct { - Host string `json:"host"` - Outcome string `json:"outcome"` -} - -func (s *writebackSummary) unchanged() { - if s != nil { - s.UnchangedFiles++ - } -} - -func (s *writebackSummary) written(err error) { - if s != nil && err == nil { - s.WrittenFiles++ - } -} - -type changeCount struct { - Added int `json:"added"` - Modified int `json:"modified"` - Deleted int `json:"deleted"` - Conflicts int `json:"current_wins"` - AlreadyApplied int `json:"already_applied"` -} - -func (s *writebackSummary) inspect(name string, before, after, latest []byte) error { - if s == nil { - return nil - } - if name != configFile && name != hostsFile { - s.record("other_files", scalarBytes(before), scalarBytes(after), scalarBytes(latest)) - return nil - } - nodes, err := decodeConfigs(before, after, latest) - if err != nil { - return err - } - b, a, l := nodes[0], nodes[1], nodes[2] - if equal(b, a) { - s.UnchangedFiles++ - return nil - } - for key := range mappingKeys(b, a) { - s.inspectNodes(changeCategory(name, key), get(b, key), get(a, key), get(l, key)) - } - return nil -} - -func decodeConfigs(configs ...[]byte) ([]*yaml.Node, error) { - nodes := make([]*yaml.Node, 0, len(configs)) - for _, data := range configs { - n, err := decode(data) - if err != nil { - return nil, err - } - nodes = append(nodes, n) - } - return nodes, nil -} - -func changeCategory(name, key string) string { - if name == hostsFile { - return "hosts" - } - switch key { - case "editor", "pager", "aliases", "git_protocol", "prompt", "browser": - return "config." + key - default: - return "config.other" - } -} - -func scalarBytes(b []byte) *yaml.Node { - if b == nil { - return nil - } - return scalar(string(b)) -} - -func (s *writebackSummary) inspectNodes(category string, before, after, latest *yaml.Node) { - if equal(before, after) { - return - } - if mappingCompatible(before, after) { - if before == nil { - before = mapping() - } - for key := range mappingKeys(before, after) { - s.inspectNodes(category, get(before, key), get(after, key), get(latest, key)) - } - return - } - s.record(category, before, after, latest) -} - -func (s *writebackSummary) record(category string, before, after, latest *yaml.Node) { - if equal(before, after) { - return - } - if s.Changes == nil { - s.Changes = map[string]changeCount{} - } - c := s.Changes[category] - switch { - case before == nil: - c.Added++ - case after == nil: - c.Deleted++ - default: - c.Modified++ - } - if equal(after, latest) { - c.AlreadyApplied++ - } else if !equal(before, latest) { - c.Conflicts++ - } - s.Changes[category] = c -} From f2c6c3e21b8bb49f29b155fbde48ff6c5eccdc31 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 17:13:36 +0900 Subject: [PATCH 09/20] test(credentials): integrate gh relay coverage into the ssh suite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Run the isolated real-gh authentication checks through the existing ssh Ginkgo suite and CI label instead of a separate opt-in Go test task. Keep dummy credentials, mock GitHub/GHE endpoints and Linux keyring checks. Reuse prebuilt agent artifacts and the fixture image's OpenSSH client, removing runtime Go compilation and its extra helper. Copy all Linux agent artifacts in the existing local E2E build task so native architecture execution is available alongside the existing amd64 fallback. Validation: related package tests and cli:lint:ci passed. The existing ssh label's focused real-gh2.100.0 case passed in465.719 seconds. A Linux E2E artifact also selected the new case in dry-run. Independent code and doc reviews accepted the exact final patch. The full ssh suite and hosted CI were not run; public GitHub/GHE, macOS keychain and live workspace startup remain outside this local isolated validation. ## Transcript - U1: > cli:test:gh-relayの追加は妥当なのか？ 既存のcli:e2e suiteに入れなくていいのか？ - A: Audited the standalone task and build tag against existing E2E task and CI wiring. Found that the dedicated task was not selected by the existing CI suite; proposed integrating the real-gh coverage. - U2: > gh-relayをe2eで独立testするのは妥当なのか？ 既存のsuiteへの追加が適しているのでは？ - A: Read the existing SSH suite and credentials forwarding cases. Agreed to add the case to the existing ssh Describe and label without a new suite, label or CI job. - U3: > 修正して - A: Confirmed branch atty303/gh-relay at e21936eacb9ee9f4ca3f850df064ccba31032696 with a clean checkout and fetched origin. Moved the integration harness and fixtures into the existing SSH suite, removed the standalone task/build tag, reused built agent artifacts and replaced the runtime-built SSH helper with isolated OpenSSH. Independent code and doc reviews accepted the final snapshot. The first runtime snapshot exposed a copied executable mode error; corrected mode0755 and verified the final snapshot via the existing ssh label. Final real-gh2.100.0 isolated integration passed in465.719 seconds, related package tests and lint passed, and a Linux CI-style E2E artifact selected the case in dry-run. Did not execute the whole SSH suite or hosted CI, public GitHub/GHE, macOS keychain or live workspaces. No product implementation, host credentials, dotfiles, launcher, remote refs or cloud resources changed. Co-authored-by: Codex --- .../tests/ssh/gh_credentials.go | 113 ++++++++++-------- e2e/tests/ssh/ssh.go | 6 + .../tests/ssh/testdata/gh-relay}/Dockerfile | 0 .../tests/ssh/testdata/gh-relay}/fixture.py | 32 ++++- mise.toml | 6 +- pkg/ghcredentials/testdata/ssh.go | 61 ---------- 6 files changed, 101 insertions(+), 117 deletions(-) rename pkg/ghcredentials/wrapper_integration_test.go => e2e/tests/ssh/gh_credentials.go (78%) rename {pkg/ghcredentials/testdata => e2e/tests/ssh/testdata/gh-relay}/Dockerfile (100%) rename {pkg/ghcredentials/testdata => e2e/tests/ssh/testdata/gh-relay}/fixture.py (95%) delete mode 100644 pkg/ghcredentials/testdata/ssh.go diff --git a/pkg/ghcredentials/wrapper_integration_test.go b/e2e/tests/ssh/gh_credentials.go similarity index 78% rename from pkg/ghcredentials/wrapper_integration_test.go rename to e2e/tests/ssh/gh_credentials.go index 422f469c21..3684829d26 100644 --- a/pkg/ghcredentials/wrapper_integration_test.go +++ b/e2e/tests/ssh/gh_credentials.go @@ -1,6 +1,4 @@ -//go:build ghrelay - -package ghcredentials_test +package ssh import ( "archive/tar" @@ -20,25 +18,24 @@ import ( "path/filepath" "runtime" "strings" - "testing" "time" "github.com/devsy-org/devsy/pkg/agent/tunnelserver" + "github.com/onsi/ginkgo/v2" "github.com/stretchr/testify/require" ) -//go:embed testdata/Dockerfile testdata/fixture.py testdata/ssh.go +//go:embed testdata/gh-relay/Dockerfile testdata/gh-relay/fixture.py var fixtureFiles embed.FS -func TestRealGitHubCLI(t *testing.T) { - ctx, cancel := context.WithTimeout(context.Background(), 8*time.Minute) - defer cancel() +func testGitHubCLIForwarding(ctx context.Context) { + t := ginkgo.GinkgoT() dir := t.TempDir() - name := prepareFixture(t, ctx, dir) + fixture := prepareFixture(t, ctx, dir) prepareHostAuthentication(t, dir) - stop, done := startFixture(t, ctx, dir, name) + stop, done := startFixture(t, ctx, dir, fixture) defer stop() - waitFixture(t, ctx, name, done) + waitFixture(t, ctx, fixture.name, done) // stdout contains only boolean test results. Raw gh streams and credentials // stay inside the disposable fixture and are never attached to failures. //nolint:gosec // name is the generated test container. @@ -46,7 +43,7 @@ func TestRealGitHubCLI(t *testing.T) { ctx, "docker", "exec", - name, + fixture.name, "python3", "/fixture/fixture.py", "check", @@ -55,13 +52,13 @@ func TestRealGitHubCLI(t *testing.T) { require.NoError(t, err, "real gh checks: %s", report) verifyRealGHReport(t, report) - verifyRelayFallback(t, ctx, name, stop) + verifyRelayFallback(t, ctx, fixture.name, stop) } -func downloadGH(t *testing.T, ctx context.Context, dir string) { +func downloadGH(t ginkgo.GinkgoTInterface, ctx context.Context, dir, arch string) { t.Helper() const version = "2.100.0" - filename := "gh_" + version + "_linux_" + runtime.GOARCH + ".tar.gz" + filename := "gh_" + version + "_linux_" + arch + ".tar.gz" get := func(name string) []byte { req, err := http.NewRequestWithContext( ctx, @@ -99,7 +96,7 @@ func downloadGH(t *testing.T, ctx context.Context, dir string) { } } -func verifyRealGHReport(t *testing.T, report []byte) { +func verifyRealGHReport(t ginkgo.GinkgoTInterface, report []byte) { t.Helper() var results []struct { Check string `json:"check"` @@ -112,17 +109,32 @@ func verifyRealGHReport(t *testing.T, report []byte) { } } -func prepareFixture(t *testing.T, ctx context.Context, dir string) string { +type ghRelayFixture struct { + name string + arch string +} + +func prepareFixture(t ginkgo.GinkgoTInterface, ctx context.Context, dir string) ghRelayFixture { t.Helper() - repo, err := filepath.Abs("../..") + agents, err := os.OpenRoot("bin") require.NoError(t, err) - for _, name := range []string{"Dockerfile", "fixture.py", "ssh.go"} { - b, err := fixtureFiles.ReadFile("testdata/" + name) - require.NoError(t, err) - root, err := os.OpenRoot(dir) + defer func() { _ = agents.Close() }() + arch := runtime.GOARCH + agent := "devsy-linux-" + arch + if _, err := agents.Stat(agent); os.IsNotExist(err) { + arch = "amd64" + agent = "devsy-linux-" + arch + } + data, err := agents.ReadFile(agent) + require.NoError(t, err) + root, err := os.OpenRoot(dir) + require.NoError(t, err) + defer func() { _ = root.Close() }() + require.NoError(t, root.WriteFile("devsy", data, 0o755)) + for _, name := range []string{"Dockerfile", "fixture.py"} { + b, err := fixtureFiles.ReadFile("testdata/gh-relay/" + name) require.NoError(t, err) require.NoError(t, root.WriteFile(name, b, 0o600)) - require.NoError(t, root.Close()) } random := make([]byte, 12) _, err = rand.Read(random) @@ -130,36 +142,28 @@ func prepareFixture(t *testing.T, ctx context.Context, dir string) string { name := "devsy-gh-test-" + hex.EncodeToString(random) image := name + ":test" //nolint:gosec // image and directory are generated fixture resources. - buildImage := exec.CommandContext(ctx, "docker", "build", "-q", "-t", image, dir) + buildImage := exec.CommandContext( + ctx, + "docker", + "build", + "--platform", + "linux/"+arch, + "-q", + "-t", + image, + dir, + ) imageOut, err := buildImage.CombinedOutput() require.NoError(t, err, "fixture image: %s", imageOut) t.Cleanup(func() { //nolint:gosec // image is the random, test-owned tag. _ = exec.Command("docker", "image", "rm", image).Run() }) - //nolint:gosec // The output is inside t.TempDir; the source is this repository. - build := exec.CommandContext(ctx, "go", "build", "-o", filepath.Join(dir, "devsy"), ".") - build.Dir = repo - build.Env = append(os.Environ(), "GOOS=linux", "GOARCH="+runtime.GOARCH, "CGO_ENABLED=0") - out, err := build.CombinedOutput() - require.NoError(t, err, "build: %s", out) - sshBuild := exec.CommandContext( - ctx, - "go", - "build", - "-o", - filepath.Join(dir, "ssh-client"), - filepath.Join(dir, "ssh.go"), - ) - sshBuild.Dir, sshBuild.Env = repo, build.Env - sshOut, err := sshBuild.CombinedOutput() - require.NoError(t, err, "build ssh fixture: %s", sshOut) - - downloadGH(t, ctx, dir) - return name + downloadGH(t, ctx, dir, arch) + return ghRelayFixture{name: name, arch: arch} } -func prepareHostAuthentication(t *testing.T, dir string) { +func prepareHostAuthentication(t ginkgo.GinkgoTInterface, dir string) { t.Helper() type tokens struct { Forwarded string `json:"forwarded"` @@ -209,11 +213,13 @@ func prepareHostAuthentication(t *testing.T, dir string) { } func startFixture( - t *testing.T, + t ginkgo.GinkgoTInterface, ctx context.Context, - dir, name string, + dir string, + fixture ghRelayFixture, ) (context.CancelFunc, <-chan error) { t.Helper() + name := fixture.name image := name + ":test" command := `cp /fixture/gh /usr/bin/gh cp /fixture/gh /usr/local/bin/gh @@ -225,7 +231,7 @@ exec dbus-run-session -- sh -c ' printf fixture-password | gnome-keyring-daemon --unlock >/dev/null 2>/tmp/keyring.log python3 /fixture/fixture.py server >/dev/null 2>/tmp/api.log & /usr/local/bin/devsy internal agent container ssh-server --address 127.0.0.1:12050 >/tmp/ssh.log 2>&1 & - /fixture/ssh-client --probe > /fixture/pre-install-gh + python3 /fixture/fixture.py probe /usr/local/bin/devsy internal agent container credentials-server --user root --configure-github-helper sleep 600 '` @@ -235,6 +241,8 @@ exec dbus-run-session -- sh -c ' "docker", "run", "--rm", + "--platform", + "linux/"+fixture.arch, "--name", name, "--network", @@ -267,7 +275,7 @@ exec dbus-run-session -- sh -c ' return tunnelCancel, done } -func waitFixture(t *testing.T, ctx context.Context, name string, done <-chan error) { +func waitFixture(t ginkgo.GinkgoTInterface, ctx context.Context, name string, done <-chan error) { t.Helper() ready := false for range 150 { @@ -294,7 +302,12 @@ func waitFixture(t *testing.T, ctx context.Context, name string, done <-chan err require.True(t, ready, "fixture was not ready") } -func verifyRelayFallback(t *testing.T, ctx context.Context, name string, stop context.CancelFunc) { +func verifyRelayFallback( + t ginkgo.GinkgoTInterface, + ctx context.Context, + name string, + stop context.CancelFunc, +) { t.Helper() // A transport failure must execute normal gh once, without relayed auth. stop() diff --git a/e2e/tests/ssh/ssh.go b/e2e/tests/ssh/ssh.go index 3ae113c9fc..173682f401 100644 --- a/e2e/tests/ssh/ssh.go +++ b/e2e/tests/ssh/ssh.go @@ -40,6 +40,12 @@ var _ = ginkgo.Describe("devsy ssh test suite", ginkgo.Label("ssh"), ginkgo.Orde framework.ExpectNoError(err) }) + ginkgo.It( + "should forward host GitHub CLI authentication without a container login", + ginkgo.SpecTimeout(framework.TimeoutLong()), + testGitHubCLIForwarding, + ) + ginkgo.It( "should start a new workspace with a docker provider (default) and SSH into it", ginkgo.SpecTimeout(framework.TimeoutShort()), diff --git a/pkg/ghcredentials/testdata/Dockerfile b/e2e/tests/ssh/testdata/gh-relay/Dockerfile similarity index 100% rename from pkg/ghcredentials/testdata/Dockerfile rename to e2e/tests/ssh/testdata/gh-relay/Dockerfile diff --git a/pkg/ghcredentials/testdata/fixture.py b/e2e/tests/ssh/testdata/gh-relay/fixture.py similarity index 95% rename from pkg/ghcredentials/testdata/fixture.py rename to e2e/tests/ssh/testdata/gh-relay/fixture.py index 3c33c28159..e2490a5384 100644 --- a/pkg/ghcredentials/testdata/fixture.py +++ b/e2e/tests/ssh/testdata/gh-relay/fixture.py @@ -4,6 +4,7 @@ import os from pathlib import Path import signal +import shlex import ssl import subprocess import sys @@ -13,6 +14,33 @@ HOSTS = ['github.com', 'enterprise.one.test', 'enterprise.two.test'] secrets = json.loads((ROOT / 'accounts.json').read_text()) +SSH_ENV = [ + 'HOME', 'GH_CONFIG_DIR', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'XDG_DATA_HOME', + 'GH_TOKEN', 'GITHUB_TOKEN', 'GH_ENTERPRISE_TOKEN', 'GITHUB_ENTERPRISE_TOKEN', + 'GH_HOST', 'GH_PATH', 'GH_PROMPT_DISABLED', 'GH_BROWSER', 'HTTP_PROXY', + 'HTTPS_PROXY', 'NO_PROXY', 'SSL_CERT_FILE', 'DBUS_SESSION_BUS_ADDRESS', + 'DEVSY_LOG_LEVEL', +] + +def ssh_command(command): + return [ + 'ssh', '-F', '/dev/null', '-T', '-p', '12050', '-oStrictHostKeyChecking=no', + '-oUserKnownHostsFile=/dev/null', '-oLogLevel=ERROR', '-oBatchMode=yes', + '-oSendEnv=' + ' '.join(SSH_ENV), 'root@127.0.0.1', command, + ] + +def probe(): + for _ in range(50): + try: + result = subprocess.run(ssh_command('command -v gh'), stdin=subprocess.DEVNULL, text=True, capture_output=True, timeout=1) + except subprocess.TimeoutExpired: + result = None + if result is not None and result.returncode == 0: + (ROOT / 'pre-install-gh').write_text(result.stdout) + return + time.sleep(0.1) + raise SystemExit(1) + def server(): subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-keyout', '/tmp/fixture-key.pem', '-out', '/tmp/fixture-cert.pem', '-days', '1', '-subj', '/CN=github.com', '-addext', 'subjectAltName=DNS:github.com,DNS:api.github.com,DNS:enterprise.one.test,DNS:enterprise.two.test'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) tls = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) @@ -91,7 +119,7 @@ def gh(*args, token=None, env=None): e = dict(os.environ) if env: e.update(env) - result = subprocess.run(['/fixture/ssh-client', *args], input=token, text=True, capture_output=True, env=e, timeout=20) + result = subprocess.run(ssh_command(shlex.join(['gh', *args])), input=token, text=True, capture_output=True, env=e, timeout=20) stderr_outputs.append(result.stderr) return result def api(host, expected='host-user', **kw): @@ -433,5 +461,7 @@ def wait_ready(): if sys.argv[1] == 'server': server() +elif sys.argv[1] == 'probe': + probe() else: check() diff --git a/mise.toml b/mise.toml index 7a6d1d535a..dc6e409a04 100644 --- a/mise.toml +++ b/mise.toml @@ -111,10 +111,6 @@ export PATH="$(go env GOPATH)/bin:$PATH" protoc -I . tunnel.proto --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative ''' -[tasks."cli:test:gh-relay"] -description = "verify GitHub CLI relay with isolated real gh and Linux keyring (requires Docker)" -run = 'go test -tags=ghrelay ./pkg/ghcredentials -run "^TestRealGitHubCLI$" -v -count=1 -timeout=10m' - [tasks."cli:test"] description = "run devsy unit tests" shell = "bash -c" @@ -129,7 +125,7 @@ description = "build devsy for e2e tests" run = [ { task = "cli:build:dev" }, "mkdir -p e2e/bin", - "cp dist/devsy-dev_linux_amd64_v1/devsy-linux-amd64 e2e/bin/devsy-linux-amd64", + 'for binary in dist/devsy-dev_linux_*/devsy-linux-*; do cp "$binary" e2e/bin/; done', ''' host_os="$(go env GOOS)" host_arch="$(go env GOARCH)" diff --git a/pkg/ghcredentials/testdata/ssh.go b/pkg/ghcredentials/testdata/ssh.go deleted file mode 100644 index 6813aba2b8..0000000000 --- a/pkg/ghcredentials/testdata/ssh.go +++ /dev/null @@ -1,61 +0,0 @@ -package main - -import ( - "errors" - "fmt" - "os" - "time" - - "al.essio.dev/pkg/shellescape" - "golang.org/x/crypto/ssh" -) - -func main() { - if err := run(); err != nil { - var exit *ssh.ExitError - if errors.As(err, &exit) { - os.Exit(exit.ExitStatus()) - } - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } -} - -func run() error { - config := &ssh.ClientConfig{ - User: "root", - HostKeyCallback: ssh.InsecureIgnoreHostKey(), - Timeout: time.Second, - } //nolint:gosec // Network-disabled disposable fixture server. - var client *ssh.Client - var err error - for range 50 { - client, err = ssh.Dial("tcp", "127.0.0.1:12050", config) - if err == nil { - break - } - time.Sleep(100 * time.Millisecond) - } - if err != nil { - return err - } - defer func() { _ = client.Close() }() - session, err := client.NewSession() - if err != nil { - return err - } - defer func() { _ = session.Close() }() - for _, name := range []string{"HOME", "GH_CONFIG_DIR", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "GH_TOKEN", "GITHUB_TOKEN", "GH_ENTERPRISE_TOKEN", "GITHUB_ENTERPRISE_TOKEN", "GH_HOST", "GH_PATH", "GH_PROMPT_DISABLED", "GH_BROWSER", "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "SSL_CERT_FILE", "DBUS_SESSION_BUS_ADDRESS", "DEVSY_LOG_LEVEL"} { - if value, ok := os.LookupEnv(name); ok { - if err := session.Setenv(name, value); err != nil { - return err - } - } - } - session.Stdout, session.Stderr = os.Stdout, os.Stderr - if len(os.Args) == 2 && os.Args[1] == "--probe" { - return session.Run("command -v gh") - } - session.Stdin = os.Stdin - return session.Run(shellescape.QuoteCommand(append([]string{"gh"}, os.Args[1:]...))) -} From 75060043fa384cff807027a7712d272942048de3 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 19:27:09 +0900 Subject: [PATCH 10/20] test(credentials): fix relay CI compatibility and fixture findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Check executable versions when asserting Compose feature isolation. The gh relay intentionally provides an entry point before an optional backend is installed, so command lookup alone no longer proves the GitHub CLI feature is present. Keep the production wrapper and lazy installation contract. Split the real-gh fixture into scenario methods without changing assertion order. Generate its TLS key in a private temporary directory, keep only the public certificate at a shared container path, and use a non-root image default with an explicit root override for the privileged test setup. These changes address the seven CodeFactor findings in the new fixture. Validation: cli:lint:ci and related package tests passed. The focused Compose case that failed CI passed locally in106.889 seconds. The final real-gh2.100.0 ssh-label case passed with isolated dummy authentication, mock endpoints and Linux keyring in73.817 seconds. Independent review verified assertion ordering and accepted the exact final snapshot. Hosted CI for the updated commit has not run at this checkpoint. ## Transcript - U1: > CIの失敗でPRとして対応しなければならないところは修正して。 - A: Inspected PR1543 checks and Actions run39112178821. CI Success was an aggregate of the Compose feature-isolation failure; CodeFactor reported seven fixture findings. Confirmed the production wrapper intentionally exists before lazy installation and changed the test oracle to execute --version rather than removing that behavior. Addressed fixture complexity, fixed temporary TLS key paths and default root image user without adding dependencies or changing production code. Related tests, lint and both focused runtime cases passed; a superseded SSH test snapshot was deliberately interrupted before running the final snapshot. Independent code review accepted the final diff and confirmed existing statement and assertion order. Prepared a signed local test commit for non-force push to the existing PR branch. No host credentials, dotfiles, repo launcher, real workspace or cloud resource changes were requested or made; runtime checks used test-owned containers. CI refresh and remote result verification follow this frozen checkpoint. Co-authored-by: Codex --- e2e/tests/ssh/gh_credentials.go | 2 + e2e/tests/ssh/testdata/gh-relay/Dockerfile | 1 + e2e/tests/ssh/testdata/gh-relay/fixture.py | 837 ++++++++++-------- .../up-docker-compose/up_docker_compose.go | 10 +- 4 files changed, 453 insertions(+), 397 deletions(-) diff --git a/e2e/tests/ssh/gh_credentials.go b/e2e/tests/ssh/gh_credentials.go index 3684829d26..f24fe60493 100644 --- a/e2e/tests/ssh/gh_credentials.go +++ b/e2e/tests/ssh/gh_credentials.go @@ -241,6 +241,8 @@ exec dbus-run-session -- sh -c ' "docker", "run", "--rm", + "--user", + "root", "--platform", "linux/"+fixture.arch, "--name", diff --git a/e2e/tests/ssh/testdata/gh-relay/Dockerfile b/e2e/tests/ssh/testdata/gh-relay/Dockerfile index 9229a5a1b1..ba300de087 100644 --- a/e2e/tests/ssh/testdata/gh-relay/Dockerfile +++ b/e2e/tests/ssh/testdata/gh-relay/Dockerfile @@ -1,3 +1,4 @@ FROM mcr.microsoft.com/devcontainers/base:ubuntu USER root RUN apt-get update && apt-get install -y --no-install-recommends python3 dbus gnome-keyring openssl ca-certificates && rm -rf /var/lib/apt/lists/* +USER vscode diff --git a/e2e/tests/ssh/testdata/gh-relay/fixture.py b/e2e/tests/ssh/testdata/gh-relay/fixture.py index e2490a5384..f0fe80dc2d 100644 --- a/e2e/tests/ssh/testdata/gh-relay/fixture.py +++ b/e2e/tests/ssh/testdata/gh-relay/fixture.py @@ -8,10 +8,12 @@ import ssl import subprocess import sys +import tempfile import time ROOT = Path('/fixture') HOSTS = ['github.com', 'enterprise.one.test', 'enterprise.two.test'] +CERTIFICATE = Path('/usr/local/share/ca-certificates/devsy-gh-fixture.crt') secrets = json.loads((ROOT / 'accounts.json').read_text()) SSH_ENV = [ @@ -42,426 +44,477 @@ def probe(): raise SystemExit(1) def server(): - subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-keyout', '/tmp/fixture-key.pem', '-out', '/tmp/fixture-cert.pem', '-days', '1', '-subj', '/CN=github.com', '-addext', 'subjectAltName=DNS:github.com,DNS:api.github.com,DNS:enterprise.one.test,DNS:enterprise.two.test'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) - tls = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) - tls.load_cert_chain('/tmp/fixture-cert.pem', '/tmp/fixture-key.pem') - tokens = {} - for record in secrets.values(): - tokens[record['forwarded']] = 'host-user' - tokens[record['local']] = 'local-user' - tokens[record['second']] = 'second-user' - tokens[record['refresh']] = 'host-user' - - class API(http.server.BaseHTTPRequestHandler): - def log_message(self, *args): - pass - def handle_api(self): - if self.command == 'POST': - self.rfile.read(int(self.headers.get('Content-Length', 0))) - token = self.headers.get('Authorization', '').split(' ')[-1] - user = tokens.get(token) - if user == 'local-user' and (ROOT / 'revoke-local').exists(): - user = None - forbidden = user == 'local-user' and (ROOT / 'forbid-local').exists() and self.path.startswith(('/repos/', '/api/v3/repos/')) - body = {'login': user, 'id': 123, 'data': {'viewer': {'login': user}}} if user else {'message': 'Bad credentials'} - if self.path == '/login/device/code': - user = 'host-user' - body = {'device_code': 'fixture-device', 'user_code': 'fixture-code', 'verification_uri': 'https://github.com/login/device', 'expires_in': 600, 'interval': 1} - elif self.path == '/login/oauth/access_token': - user = 'host-user' - body = {'access_token': secrets[HOSTS[0]]['refresh'], 'token_type': 'bearer', 'scope': 'repo,read:org,gist'} - data = json.dumps(body).encode() - self.send_response(403 if forbidden else 200 if user else 401) - self.send_header('X-OAuth-Scopes', 'repo, read:org, gist') - self.send_header('Content-Type', 'application/json') - self.send_header('Content-Length', str(len(data))) - self.end_headers() - self.wfile.write(data) - do_GET = handle_api - do_POST = handle_api - - class Proxy(http.server.BaseHTTPRequestHandler): - def log_message(self, *args): - pass - def do_CONNECT(self): - if self.path not in [h + ':443' for h in HOSTS + ['api.github.com']]: - self.send_error(403) - return - self.send_response(200) - self.end_headers() - self.wfile.flush() - try: - with tls.wrap_socket(self.connection, server_side=True) as conn: - API(conn, self.client_address, self.server) - except (ssl.SSLError, ConnectionError): + with tempfile.TemporaryDirectory(prefix='devsy-gh-tls-') as directory: + key = Path(directory) / 'key.pem' + subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-keyout', str(key), '-out', str(CERTIFICATE), '-days', '1', '-subj', '/CN=github.com', '-addext', 'subjectAltName=DNS:github.com,DNS:api.github.com,DNS:enterprise.one.test,DNS:enterprise.two.test'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) + tls = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + tls.load_cert_chain(str(CERTIFICATE), str(key)) + tokens = {} + for record in secrets.values(): + tokens[record['forwarded']] = 'host-user' + tokens[record['local']] = 'local-user' + tokens[record['second']] = 'second-user' + tokens[record['refresh']] = 'host-user' + + class API(http.server.BaseHTTPRequestHandler): + def log_message(self, *args): + pass + def handle_api(self): + if self.command == 'POST': + self.rfile.read(int(self.headers.get('Content-Length', 0))) + token = self.headers.get('Authorization', '').split(' ')[-1] + user = tokens.get(token) + if user == 'local-user' and (ROOT / 'revoke-local').exists(): + user = None + forbidden = user == 'local-user' and (ROOT / 'forbid-local').exists() and self.path.startswith(('/repos/', '/api/v3/repos/')) + body = {'login': user, 'id': 123, 'data': {'viewer': {'login': user}}} if user else {'message': 'Bad credentials'} + if self.path == '/login/device/code': + user = 'host-user' + body = {'device_code': 'fixture-device', 'user_code': 'fixture-code', 'verification_uri': 'https://github.com/login/device', 'expires_in': 600, 'interval': 1} + elif self.path == '/login/oauth/access_token': + user = 'host-user' + body = {'access_token': secrets[HOSTS[0]]['refresh'], 'token_type': 'bearer', 'scope': 'repo,read:org,gist'} + data = json.dumps(body).encode() + self.send_response(403 if forbidden else 200 if user else 401) + self.send_header('X-OAuth-Scopes', 'repo, read:org, gist') + self.send_header('Content-Type', 'application/json') + self.send_header('Content-Length', str(len(data))) + self.end_headers() + self.wfile.write(data) + do_GET = handle_api + do_POST = handle_api + + class Proxy(http.server.BaseHTTPRequestHandler): + def log_message(self, *args): pass - srv = http.server.ThreadingHTTPServer(('127.0.0.1', 8123), Proxy) - (ROOT / 'api-ready').touch() - srv.serve_forever() - - -def check(): - os.environ.update({ - 'HTTPS_PROXY': 'http://127.0.0.1:8123', 'HTTP_PROXY': 'http://127.0.0.1:8123', - 'NO_PROXY': 'localhost,127.0.0.1', 'SSL_CERT_FILE': '/tmp/fixture-cert.pem', - 'GH_CONFIG_DIR': '/fixture/guest-gh', 'XDG_DATA_HOME': '/fixture/guest-data', - 'XDG_CACHE_HOME': '/fixture/guest-cache', 'GH_PROMPT_DISABLED': '1', - 'DBUS_SESSION_BUS_ADDRESS': (ROOT / 'bus').read_text().strip(), - }) - results = [] - stderr_outputs = [] - def verify(name, ok): - results.append({'check': name, 'ok': bool(ok)}) + def do_CONNECT(self): + if self.path not in [h + ':443' for h in HOSTS + ['api.github.com']]: + self.send_error(403) + return + self.send_response(200) + self.end_headers() + self.wfile.flush() + try: + with tls.wrap_socket(self.connection, server_side=True) as conn: + API(conn, self.client_address, self.server) + except (ssl.SSLError, ConnectionError): + pass + srv = http.server.ThreadingHTTPServer(('127.0.0.1', 8123), Proxy) + (ROOT / 'api-ready').touch() + srv.serve_forever() + +class RelayChecks: + def verify(self, name, ok): + self.results.append({'check': name, 'ok': bool(ok)}) if not ok: - print(json.dumps(results), flush=True) + print(json.dumps(self.results), flush=True) raise SystemExit(1) - def gh(*args, token=None, env=None): + + def gh(self, *args, token=None, env=None): e = dict(os.environ) if env: e.update(env) result = subprocess.run(ssh_command(shlex.join(['gh', *args])), input=token, text=True, capture_output=True, env=e, timeout=20) - stderr_outputs.append(result.stderr) + self.stderr_outputs.append(result.stderr) return result - def api(host, expected='host-user', **kw): - r = gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) + + def api(self, host, expected='host-user', **kw): + r = self.gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) ok = r.returncode == 0 and r.stdout.strip() == expected return ok - verify('SSH server predates wrapper installation', (ROOT / 'pre-install-gh').read_text().strip() == '/usr/local/bin/gh') - verify('standard wrapper installed', Path('/usr/local/bin/gh').read_text().startswith('#!/bin/sh\n# devsy gh relay\n')) - verify('real gh preserved in fixed backup', Path('/usr/local/bin/gh.devsy-original').read_bytes() == (ROOT / 'gh').read_bytes()) - verify('no dedicated profile installed', not Path('/etc/profile.d/devsy-gh.sh').exists()) - host_before = (ROOT / 'host-gh' / 'hosts.yml').read_bytes() - original = ROOT / 'guest-gh' - original.mkdir(exist_ok=True) - def saved(): - return (original / 'hosts.yml').read_text() if (original / 'hosts.yml').exists() else '' - def no_forwarded_saved(): - return all(v['forwarded'] not in saved() for v in secrets.values()) - registrations = ( - 'github.com:\n git_protocol: https\n user: local-user\n users:\n local-user:\n' - 'enterprise.one.test:\n git_protocol: https\n user: host-user\n oauth_token: \"\"\n users:\n host-user:\n oauth_token: \"\"\n' - 'enterprise.two.test:\n git_protocol: https\n user: dotfiles-user\n oauth_token:\n users:\n dotfiles-user:\n oauth_token:\n other-user:\n' - ) - (original / 'hosts.yml').write_text(registrations) - - dispatcher = ROOT / 'dispatcher' - dispatcher.mkdir() - shim = dispatcher / 'gh' - shim.write_text('#!/bin/sh\nexport PATH=/usr/bin:/usr/local/bin:/bin\nif [ -n "$GH_TEST_CONFIG_OVERRIDE" ]; then export GH_CONFIG_DIR="$GH_TEST_CONFIG_OVERRIDE"; fi\nexec /usr/local/bin/gh "$@"\n') - shim.chmod(0o755) - def nested_gh(*args, token=None, env=None): - e = dict(os.environ, PATH=str(dispatcher) + ':/usr/local/bin:/usr/bin:/bin') + + def saved(self): + return (self.original / 'hosts.yml').read_text() if (self.original / 'hosts.yml').exists() else '' + + def no_forwarded_saved(self): + return all(v['forwarded'] not in self.saved() for v in secrets.values()) + + def nested_gh(self, *args, token=None, env=None): + e = dict(os.environ, PATH=str(self.dispatcher) + ':/usr/local/bin:/usr/bin:/bin') if env: e.update(env) result = subprocess.run(['/usr/local/bin/gh', *args], input=token, text=True, capture_output=True, env=e, timeout=20) - stderr_outputs.append(result.stderr) + self.stderr_outputs.append(result.stderr) return result - def nested_api(host, expected='host-user', **kw): - r = nested_gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) + + def nested_api(self, host, expected='host-user', **kw): + r = self.nested_gh('api', 'user', '--hostname', host, '--jq', '.login', **kw) return r.returncode == 0 and r.stdout.strip() == expected - verify('dispatcher re-entry cannot supply forwarded authentication to local lookup', all(nested_api(h) for h in HOSTS)) - verify('dispatcher re-entry preserves real auth status', nested_gh('auth', 'status').returncode == 0) - redirected = ROOT / 'redirected-gh' - redirected.mkdir() - (redirected / 'hosts.yml').write_text(registrations) - verify('local-only lookup survives dispatcher config changes', nested_api(HOSTS[0], env={'GH_TEST_CONFIG_OVERRIDE': str(redirected)})) - verify('nested lookup leaves original preferences unchanged', saved() == registrations and no_forwarded_saved()) - verify('nested lookup retains explicit invalid environment authentication', not nested_api(HOSTS[0], env={'GH_TOKEN': secrets[HOSTS[0]]['invalid']})) - r = nested_gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) - verify('nested local login retains precedence', r.returncode == 0 and nested_api(HOSTS[0], 'local-user')) - local_nested = saved() - (original / 'hosts.yml').write_text(local_nested.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) - verify('nested invalid local token never falls back', not nested_api(HOSTS[0])) - (original / 'hosts.yml').write_text(local_nested) - verify('nested local logout restores forwarding', nested_gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user').returncode == 0 and nested_api(HOSTS[0])) - verify('nested alias preparation', gh('alias', 'set', 'probe-guard', '!test -z "$DEVSY_GH_AUTH_PROBE" && gh api user --jq .login').returncode == 0) - r = nested_gh('probe-guard') - verify('local-only marker does not leak into aliases or their children', r.returncode == 0 and r.stdout.strip() == 'host-user') - verify('nested alias removed', gh('alias', 'delete', 'probe-guard').returncode == 0) - (original / 'hosts.yml').write_text(registrations) - - envfile = Path('/etc/envfile.json') - previous_envfile = envfile.read_bytes() if envfile.exists() else None - try: - envfile.write_text(json.dumps({'env': {'GH_TOKEN': secrets[HOSTS[0]]['invalid']}})) - verify('Devsy envfile does not replace forwarded authentication', api(HOSTS[0])) - verify('Devsy envfile does not replace explicit authentication', api(HOSTS[0], 'local-user', env={'GH_TOKEN': secrets[HOSTS[0]]['local']})) - finally: - if previous_envfile is None: - envfile.unlink(missing_ok=True) - else: - envfile.write_bytes(previous_envfile) - - for args in [('--help',), ('--version',), ('api', '--devsy-unknown-fixture-flag'), ('api', '--context', 'sample'), ('api', '--log-level', 'invalid'), ('api', '--log-output', 'json')]: - expected = subprocess.run(['/usr/local/bin/gh.devsy-original', *args], text=True, capture_output=True, timeout=20) - r = gh(*args) - verify('real gh arguments pass through Cobra: ' + ' '.join(args), (r.returncode, r.stdout, r.stderr) == (expected.returncode, expected.stdout, expected.stderr)) - for level in ['debug', 'trace']: - r = gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': level}) - verify(level + ' enables relay logs without changing stdout', r.returncode == 0 and r.stdout == 'host-user\n' and 'gh relay: authentication source' in r.stderr and 'forwarded' in r.stderr) - r = gh('api', 'user', '--jq', '.login') - verify('normal successful forwarding is silent', r.returncode == 0 and r.stdout == 'host-user\n' and r.stderr == '') - verify('tokenless registrations use host active accounts', all(api(h) for h in HOSTS)) - try: - (original / 'hosts.yml').write_text('[invalid yaml') - expected = subprocess.run(['/usr/local/bin/gh.devsy-original', '--version'], text=True, capture_output=True, timeout=20) - r = gh('--version') - verify('overlay failure preserves gh stdout and exit', (r.returncode, r.stdout) == (expected.returncode, expected.stdout)) - verify('overlay failure warns without config content', 'gh relay: forwarding unavailable' in r.stderr and 'schema_invalid' in r.stderr) - finally: - (original / 'hosts.yml').write_text(registrations) - r = gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) - verify('non-PTY SSH uses fixed real-gh backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and '/usr/local/bin/gh.devsy-original' in r.stderr) - selected = ROOT / 'selected-gh' - selected.mkdir() - (selected / 'gh').symlink_to('/usr/bin/gh') - r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin', DEVSY_LOG_LEVEL='debug'), text=True, capture_output=True, timeout=20) - stderr_outputs.append(r.stderr) - verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and str(selected / 'gh') in r.stderr) - git_config = Path.home() / '.gitconfig' - previous_git_config = git_config.read_bytes() if git_config.exists() else None - try: - verify('setup-git accepts forwarded authentication', gh('auth', 'setup-git').returncode == 0) - helper = subprocess.run(['git', 'config', '--global', '--get-all', 'credential.https://github.com.helper'], text=True, capture_output=True, timeout=20) - verify('setup-git persists the relay wrapper', helper.returncode == 0 and '/usr/local/bin/gh auth git-credential' in helper.stdout and 'gh.devsy-original' not in helper.stdout) - credentials = subprocess.run(['git', 'credential', 'fill'], input='protocol=https\nhost=github.com\n\n', env=dict(os.environ, GIT_TERMINAL_PROMPT='0'), text=True, capture_output=True, timeout=20) - verify('persisted Git helper retrieves forwarded authentication', credentials.returncode == 0 and ('password=' + secrets[HOSTS[0]]['forwarded']) in credentials.stdout) - finally: - if previous_git_config is None: - git_config.unlink(missing_ok=True) - else: - git_config.write_bytes(previous_git_config) - verify('self-path alias saves', gh('alias', 'set', 'self-path', '!printf "%s" "$GH_PATH"').returncode == 0) - verify('aliases inherit relay self-path', gh('self-path').stdout == '/usr/local/bin/gh') - verify('explicit self-path is preserved', gh('self-path', env={'GH_PATH': '/fixture/custom-gh'}).stdout == '/fixture/custom-gh') - verify('explicit empty self-path is preserved', gh('self-path', env={'GH_PATH': ''}).stdout == '') - verify('self-path alias removed', gh('alias', 'delete', 'self-path').returncode == 0) - verify('real auth status ignores hidden tokenless registrations', gh('auth', 'status').returncode == 0) - verify('unchanged invocation preserves exact dotfiles registrations', saved() == registrations and no_forwarded_saved()) - r = gh('auth', 'logout', '--hostname', HOSTS[2], '--user', 'host-user') - verify('forwarded logout preserves hidden dotfiles host', r.returncode == 0 and saved() == registrations and api(HOSTS[2])) - r = gh('config', 'set', 'git_protocol', 'https', '--host', HOSTS[2]) - verify('host setting resave preserves registrations and user preference', r.returncode == 0 and saved() == registrations + ' host-user:\n git_protocol: https\n') - verify('forwarding preserves saved per-account settings', gh('config', 'get', 'git_protocol', '--host', HOSTS[2]).stdout.strip() == 'https') - r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) - verify('local login from tokenless preferences persists', r.returncode == 0 and api(HOSTS[0], 'local-user') and no_forwarded_saved()) - r = gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user') - verify('local logout from preferences restores forwarding', r.returncode == 0 and api(HOSTS[0])) - (original / 'hosts.yml').write_text(registrations) - ready_meta, release_meta = ROOT / 'metadata-ready', ROOT / 'metadata-release' - verify('metadata mutation alias saves', gh('alias', 'set', 'metadata-pending', '!gh config set git_protocol ssh --host enterprise.two.test; touch "$GH_TEST_READY"; while test ! -f "$GH_TEST_RELEASE"; do sleep 0.05; done').returncode == 0) - e = dict(os.environ, GH_TEST_READY=str(ready_meta), GH_TEST_RELEASE=str(release_meta)) - p = subprocess.Popen(['/bin/bash', '-lc', 'exec gh metadata-pending'], env=e, stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True) - try: - for _ in range(200): - if ready_meta.exists(): - break - time.sleep(0.05) - verify('tokenless metadata invocation reached mutation', ready_meta.exists()) - changed = registrations.replace('user: dotfiles-user', 'user: parallel-user') + 'unrelated.test:\n git_protocol: ssh\n' - (original / 'hosts.yml').write_text(changed) - release_meta.touch() - p.communicate(timeout=20) - verify('host-setting delta preserves parallel user and host changes', p.returncode == 0 and 'user: parallel-user' in saved() and 'unrelated.test:\n git_protocol: ssh' in saved() and no_forwarded_saved()) - verify('host-setting delta persists at host and account positions', 'enterprise.two.test:\n git_protocol: ssh' in saved() and ' host-user:\n git_protocol: ssh' in saved()) - finally: - if p.poll() is None: - os.killpg(p.pid, signal.SIGKILL) - p.communicate() - verify('metadata alias removed', gh('alias', 'delete', 'metadata-pending').returncode == 0) - (original / 'hosts.yml').unlink() - - verify('all hosts forwarded through real RPC and HTTP relay', all(api(h) for h in HOSTS)) - verify('real auth status', gh('auth', 'status').returncode == 0) - verify('forwarding never saved', no_forwarded_saved()) - r = gh('api', 'user', '--jq', '.login', env={'GH_HOST': HOSTS[2]}) - verify('gh chooses GH_HOST', r.returncode == 0 and r.stdout.strip() == 'host-user') - r = gh('api', 'user', env={'GH_TOKEN': secrets[HOSTS[0]]['invalid'], 'DEVSY_LOG_LEVEL': 'debug'}) - verify('explicit invalid GH_TOKEN never falls back', r.returncode != 0 and 'gh relay: authentication source' in r.stderr and 'environment' in r.stderr) - verify('explicit invalid GITHUB_TOKEN never falls back', gh('api', 'user', env={'GITHUB_TOKEN': secrets[HOSTS[0]]['invalid']}).returncode != 0) - verify('explicit enterprise environment never falls back', gh('api', 'user', '--hostname', HOSTS[1], env={'GH_ENTERPRISE_TOKEN': secrets[HOSTS[1]]['invalid']}).returncode != 0) - r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['forwarded']) - verify('same token login stays forwarded', r.returncode == 0 and no_forwarded_saved() and api(HOSTS[0])) - verify('auth switch uses forwarded account', gh('auth', 'switch', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and no_forwarded_saved()) - r = gh('auth', 'refresh', '--hostname', HOSTS[0], '--insecure-storage', env={'GH_BROWSER': '/bin/true'}) - verify('refresh saves a newly issued credential', r.returncode == 0 and secrets[HOSTS[0]]['refresh'] in saved() and api(HOSTS[0])) - verify('logout refreshed credential', gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and api(HOSTS[0])) - r = gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) - verify('different token local login persists', r.returncode == 0 and secrets[HOSTS[0]]['local'] in saved() and api(HOSTS[0], 'local-user')) - r = gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) - verify('local authentication preference explained at debug level', r.returncode == 0 and r.stdout == 'local-user\n' and 'gh relay: authentication source' in r.stderr and 'local_config' in r.stderr) - local_text = saved() - (original / 'hosts.yml').write_text(local_text.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) - verify('invalid local credential never falls back', not api(HOSTS[0])) - (original / 'hosts.yml').write_text(local_text) - r = gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user') - verify('logout restores forwarding on next invocation', r.returncode == 0 and api(HOSTS[0]) and no_forwarded_saved()) - - r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['local']) - verify('keyring login succeeds without plaintext token', r.returncode == 0 and secrets[HOSTS[1]]['local'] not in saved() and api(HOSTS[1], 'local-user')) - r = gh('api', 'user', '--hostname', HOSTS[1], '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) - verify('existing keyring credential excluded from forwarding', r.returncode == 0 and r.stdout.strip() == 'local-user' and 'gh relay: authentication source' in r.stderr and 'keyring' in r.stderr) - verify('nested lookup preserves keyring authentication', nested_api(HOSTS[1], 'local-user')) - verify('unavailable keyring backend follows gh no-token result', api(HOSTS[1], env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/nonexistent'})) - verify('unavailable backend leaves local keyring intact', api(HOSTS[1], 'local-user')) - revoked = ROOT / 'revoke-local' - revoked.touch() - try: - verify('expired keyring credential never falls back', not api(HOSTS[1])) - verify('nested expired keyring credential never falls back', not nested_api(HOSTS[1])) - finally: - revoked.unlink() - forbidden = ROOT / 'forbid-local' - forbidden.touch() - try: - r = gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) - verify('insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr) - r = nested_gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) - verify('nested insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr) - finally: - forbidden.unlink() - r = gh('auth', 'status', '--hostname', HOSTS[1]) - verify('real auth status identifies keyring', r.returncode == 0 and 'keyring' in r.stdout + r.stderr) - verify('keyring logout restores forwarding', gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'local-user').returncode == 0 and api(HOSTS[1])) - - r = gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['forwarded']) - verify('same token explicit keyring login retained', r.returncode == 0 and 'host-user' in saved() and no_forwarded_saved() and api(HOSTS[1])) - verify('same token keyring logout restores forwarding', gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'host-user').returncode == 0 and api(HOSTS[1])) - - verify('alias saves', gh('alias', 'set', 'save-editor', '!gh config set editor nano').returncode == 0) - verify('alias nested configuration persists', gh('save-editor').returncode == 0 and gh('config', 'get', 'editor').stdout.strip() == 'nano') - ext = ROOT / 'guest-data' / 'gh' / 'extensions' / 'gh-probe' - ext.mkdir(parents=True) - (ext / 'gh-probe').write_text('#!/bin/sh\ngh config set pager cat\ngh api user --hostname enterprise.two.test --jq .login\nprintf "extension-out\\n"\nprintf "extension-err\\n" >&2\nexit 7\n') - (ext / 'gh-probe').chmod(0o755) - r = gh('probe') - verify('extension streams and failure exit pass unchanged', r.returncode == 7 and r.stdout == 'host-user\nextension-out\n' and r.stderr == 'extension-err\n') - verify('extension change persists after failure', gh('config', 'get', 'pager').stdout.strip() == 'cat') - - config = original / 'config.yml' - target = ROOT / 'dotfiles-config.yml' - config.rename(target) - config.symlink_to(target) - ready, release = ROOT / 'command-ready', ROOT / 'command-release' - verify('concurrent alias saves', gh('alias', 'set', 'pending-editor', '!gh config set editor pending; touch "$GH_TEST_READY"; while test ! -f "$GH_TEST_RELEASE"; do sleep 0.05; done').returncode == 0) - def pending(): - e = dict(os.environ, GH_TEST_READY=str(ready), GH_TEST_RELEASE=str(release)) + + def pending(self): + e = dict(os.environ, GH_TEST_READY=str(self.ready), GH_TEST_RELEASE=str(self.release)) return subprocess.Popen(['/bin/bash', '-lc', 'exec gh pending-editor'], env=e, stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True) - def wait_ready(): + + def wait_ready(self): for _ in range(200): - if ready.exists(): + if self.ready.exists(): return True time.sleep(0.05) return False - p = pending() - try: - verify('concurrent invocation reached user mutation', wait_ready()) - verify('parallel gh saves unrelated pager', gh('config', 'set', 'pager', 'less').returncode == 0) - target.write_text(target.read_text().replace('editor: nano', 'editor: emacs') + '\nprompt: disabled\n') - release.touch() - p.communicate(timeout=20) - verify('parallel and dotfiles changes survive', p.returncode == 0 and gh('config', 'get', 'editor').stdout.strip() == 'pending' and gh('config', 'get', 'pager').stdout.strip() == 'less' and gh('config', 'get', 'prompt').stdout.strip() == 'disabled' and config.is_symlink()) - finally: - if p.poll() is None: - os.killpg(p.pid, signal.SIGKILL) - p.communicate() - ready.unlink() - release.unlink() - verify('prepare interrupted mutation', gh('config', 'set', 'editor', 'vi').returncode == 0) - p = pending() - try: - verify('interrupted invocation reached mutation', wait_ready()) - finally: - os.killpg(p.pid, signal.SIGKILL) - p.communicate() - verify('next invocation recovers killed wrapper mutation', gh('config', 'get', 'editor').stdout.strip() == 'pending') - verify('recovered session contains no forwarded credentials', no_forwarded_saved()) - - ready.unlink() - verify('prepare pending writeback', gh('config', 'set', 'editor', 'vi').returncode == 0) - p = pending() - latest = None - try: - verify('writeback failure reached user mutation', wait_ready()) - latest = target.read_text() - target.write_text('[broken yaml') - release.touch() - _, writeback_stderr = p.communicate(timeout=20) - stderr_outputs.append(writeback_stderr.decode()) - verify('writeback failure does not change real gh exit', p.returncode == 0) - verify('writeback failure emits actionable error', 'gh relay: configuration writeback pending' in writeback_stderr.decode() and 'schema_invalid' in writeback_stderr.decode()) - finally: - if latest is not None: - target.write_text(latest) - if p.poll() is None: + + def __init__(self): + os.environ.update({ + 'HTTPS_PROXY': 'http://127.0.0.1:8123', 'HTTP_PROXY': 'http://127.0.0.1:8123', + 'NO_PROXY': 'localhost,127.0.0.1', 'SSL_CERT_FILE': str(CERTIFICATE), + 'GH_CONFIG_DIR': '/fixture/guest-gh', 'XDG_DATA_HOME': '/fixture/guest-data', + 'XDG_CACHE_HOME': '/fixture/guest-cache', 'GH_PROMPT_DISABLED': '1', + 'DBUS_SESSION_BUS_ADDRESS': (ROOT / 'bus').read_text().strip(), + }) + self.results = [] + self.stderr_outputs = [] + + def check_installation(self): + self.verify('SSH server predates wrapper installation', (ROOT / 'pre-install-gh').read_text().strip() == '/usr/local/bin/gh') + self.verify('standard wrapper installed', Path('/usr/local/bin/gh').read_text().startswith('#!/bin/sh\n# devsy gh relay\n')) + self.verify('real gh preserved in fixed backup', Path('/usr/local/bin/gh.devsy-original').read_bytes() == (ROOT / 'gh').read_bytes()) + self.verify('no dedicated profile installed', not Path('/etc/profile.d/devsy-gh.sh').exists()) + self.host_before = (ROOT / 'host-gh' / 'hosts.yml').read_bytes() + self.original = ROOT / 'guest-gh' + self.original.mkdir(exist_ok=True) + self.registrations = ( + 'github.com:\n git_protocol: https\n user: local-user\n users:\n local-user:\n' + 'enterprise.one.test:\n git_protocol: https\n user: host-user\n oauth_token: \"\"\n users:\n host-user:\n oauth_token: \"\"\n' + 'enterprise.two.test:\n git_protocol: https\n user: dotfiles-user\n oauth_token:\n users:\n dotfiles-user:\n oauth_token:\n other-user:\n' + ) + (self.original / 'hosts.yml').write_text(self.registrations) + + def check_dispatcher(self): + self.dispatcher = ROOT / 'dispatcher' + self.dispatcher.mkdir() + shim = self.dispatcher / 'gh' + shim.write_text('#!/bin/sh\nexport PATH=/usr/bin:/usr/local/bin:/bin\nif [ -n "$GH_TEST_CONFIG_OVERRIDE" ]; then export GH_CONFIG_DIR="$GH_TEST_CONFIG_OVERRIDE"; fi\nexec /usr/local/bin/gh "$@"\n') + shim.chmod(0o755) + self.verify('dispatcher re-entry cannot supply forwarded authentication to local lookup', all(self.nested_api(h) for h in HOSTS)) + self.verify('dispatcher re-entry preserves real auth status', self.nested_gh('auth', 'status').returncode == 0) + redirected = ROOT / 'redirected-gh' + redirected.mkdir() + (redirected / 'hosts.yml').write_text(self.registrations) + self.verify('local-only lookup survives dispatcher config changes', self.nested_api(HOSTS[0], env={'GH_TEST_CONFIG_OVERRIDE': str(redirected)})) + self.verify('nested lookup leaves original preferences unchanged', self.saved() == self.registrations and self.no_forwarded_saved()) + self.verify('nested lookup retains explicit invalid environment authentication', not self.nested_api(HOSTS[0], env={'GH_TOKEN': secrets[HOSTS[0]]['invalid']})) + r = self.nested_gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) + self.verify('nested local login retains precedence', r.returncode == 0 and self.nested_api(HOSTS[0], 'local-user')) + local_nested = self.saved() + (self.original / 'hosts.yml').write_text(local_nested.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) + self.verify('nested invalid local token never falls back', not self.nested_api(HOSTS[0])) + (self.original / 'hosts.yml').write_text(local_nested) + self.verify('nested local logout restores forwarding', self.nested_gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user').returncode == 0 and self.nested_api(HOSTS[0])) + self.verify('nested alias preparation', self.gh('alias', 'set', 'probe-guard', '!test -z "$DEVSY_GH_AUTH_PROBE" && gh api user --jq .login').returncode == 0) + r = self.nested_gh('probe-guard') + self.verify('local-only marker does not leak into aliases or their children', r.returncode == 0 and r.stdout.strip() == 'host-user') + self.verify('nested alias removed', self.gh('alias', 'delete', 'probe-guard').returncode == 0) + (self.original / 'hosts.yml').write_text(self.registrations) + + def check_envfile(self): + envfile = Path('/etc/envfile.json') + previous_envfile = envfile.read_bytes() if envfile.exists() else None + try: + envfile.write_text(json.dumps({'env': {'GH_TOKEN': secrets[HOSTS[0]]['invalid']}})) + self.verify('Devsy envfile does not replace forwarded authentication', self.api(HOSTS[0])) + self.verify('Devsy envfile does not replace explicit authentication', self.api(HOSTS[0], 'local-user', env={'GH_TOKEN': secrets[HOSTS[0]]['local']})) + finally: + if previous_envfile is None: + envfile.unlink(missing_ok=True) + else: + envfile.write_bytes(previous_envfile) + + def check_dispatch(self): + for args in [('--help',), ('--version',), ('api', '--devsy-unknown-fixture-flag'), ('api', '--context', 'sample'), ('api', '--log-level', 'invalid'), ('api', '--log-output', 'json')]: + expected = subprocess.run(['/usr/local/bin/gh.devsy-original', *args], text=True, capture_output=True, timeout=20) + r = self.gh(*args) + self.verify('real gh arguments pass through Cobra: ' + ' '.join(args), (r.returncode, r.stdout, r.stderr) == (expected.returncode, expected.stdout, expected.stderr)) + for level in ['debug', 'trace']: + r = self.gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': level}) + self.verify(level + ' enables relay logs without changing stdout', r.returncode == 0 and r.stdout == 'host-user\n' and 'gh relay: authentication source' in r.stderr and 'forwarded' in r.stderr) + r = self.gh('api', 'user', '--jq', '.login') + self.verify('normal successful forwarding is silent', r.returncode == 0 and r.stdout == 'host-user\n' and r.stderr == '') + self.verify('tokenless registrations use host active accounts', all(self.api(h) for h in HOSTS)) + + def check_overlay_and_selection(self): + try: + (self.original / 'hosts.yml').write_text('[invalid yaml') + expected = subprocess.run(['/usr/local/bin/gh.devsy-original', '--version'], text=True, capture_output=True, timeout=20) + r = self.gh('--version') + self.verify('overlay failure preserves gh stdout and exit', (r.returncode, r.stdout) == (expected.returncode, expected.stdout)) + self.verify('overlay failure warns without config content', 'gh relay: forwarding unavailable' in r.stderr and 'schema_invalid' in r.stderr) + finally: + (self.original / 'hosts.yml').write_text(self.registrations) + r = self.gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) + self.verify('non-PTY SSH uses fixed real-gh backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and '/usr/local/bin/gh.devsy-original' in r.stderr) + selected = ROOT / 'selected-gh' + selected.mkdir() + (selected / 'gh').symlink_to('/usr/bin/gh') + r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin', DEVSY_LOG_LEVEL='debug'), text=True, capture_output=True, timeout=20) + self.stderr_outputs.append(r.stderr) + self.verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and str(selected / 'gh') in r.stderr) + + def check_self_references(self): + git_config = Path.home() / '.gitconfig' + previous_git_config = git_config.read_bytes() if git_config.exists() else None + try: + self.verify('setup-git accepts forwarded authentication', self.gh('auth', 'setup-git').returncode == 0) + helper = subprocess.run(['git', 'config', '--global', '--get-all', 'credential.https://github.com.helper'], text=True, capture_output=True, timeout=20) + self.verify('setup-git persists the relay wrapper', helper.returncode == 0 and '/usr/local/bin/gh auth git-credential' in helper.stdout and 'gh.devsy-original' not in helper.stdout) + credentials = subprocess.run(['git', 'credential', 'fill'], input='protocol=https\nhost=github.com\n\n', env=dict(os.environ, GIT_TERMINAL_PROMPT='0'), text=True, capture_output=True, timeout=20) + self.verify('persisted Git helper retrieves forwarded authentication', credentials.returncode == 0 and ('password=' + secrets[HOSTS[0]]['forwarded']) in credentials.stdout) + finally: + if previous_git_config is None: + git_config.unlink(missing_ok=True) + else: + git_config.write_bytes(previous_git_config) + self.verify('self-path alias saves', self.gh('alias', 'set', 'self-path', '!printf "%s" "$GH_PATH"').returncode == 0) + self.verify('aliases inherit relay self-path', self.gh('self-path').stdout == '/usr/local/bin/gh') + self.verify('explicit self-path is preserved', self.gh('self-path', env={'GH_PATH': '/fixture/custom-gh'}).stdout == '/fixture/custom-gh') + self.verify('explicit empty self-path is preserved', self.gh('self-path', env={'GH_PATH': ''}).stdout == '') + self.verify('self-path alias removed', self.gh('alias', 'delete', 'self-path').returncode == 0) + + def check_preferences(self): + self.verify('real auth status ignores hidden tokenless registrations', self.gh('auth', 'status').returncode == 0) + self.verify('unchanged invocation preserves exact dotfiles registrations', self.saved() == self.registrations and self.no_forwarded_saved()) + r = self.gh('auth', 'logout', '--hostname', HOSTS[2], '--user', 'host-user') + self.verify('forwarded logout preserves hidden dotfiles host', r.returncode == 0 and self.saved() == self.registrations and self.api(HOSTS[2])) + r = self.gh('config', 'set', 'git_protocol', 'https', '--host', HOSTS[2]) + self.verify('host setting resave preserves registrations and user preference', r.returncode == 0 and self.saved() == self.registrations + ' host-user:\n git_protocol: https\n') + self.verify('forwarding preserves saved per-account settings', self.gh('config', 'get', 'git_protocol', '--host', HOSTS[2]).stdout.strip() == 'https') + r = self.gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) + self.verify('local login from tokenless preferences persists', r.returncode == 0 and self.api(HOSTS[0], 'local-user') and self.no_forwarded_saved()) + r = self.gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user') + self.verify('local logout from preferences restores forwarding', r.returncode == 0 and self.api(HOSTS[0])) + (self.original / 'hosts.yml').write_text(self.registrations) + ready_meta, release_meta = ROOT / 'metadata-ready', ROOT / 'metadata-release' + self.verify('metadata mutation alias saves', self.gh('alias', 'set', 'metadata-pending', '!gh config set git_protocol ssh --host enterprise.two.test; touch "$GH_TEST_READY"; while test ! -f "$GH_TEST_RELEASE"; do sleep 0.05; done').returncode == 0) + e = dict(os.environ, GH_TEST_READY=str(ready_meta), GH_TEST_RELEASE=str(release_meta)) + p = subprocess.Popen(['/bin/bash', '-lc', 'exec gh metadata-pending'], env=e, stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True) + try: + for _ in range(200): + if ready_meta.exists(): + break + time.sleep(0.05) + self.verify('tokenless metadata invocation reached mutation', ready_meta.exists()) + changed = self.registrations.replace('user: dotfiles-user', 'user: parallel-user') + 'unrelated.test:\n git_protocol: ssh\n' + (self.original / 'hosts.yml').write_text(changed) + release_meta.touch() + p.communicate(timeout=20) + self.verify('host-setting delta preserves parallel user and host changes', p.returncode == 0 and 'user: parallel-user' in self.saved() and 'unrelated.test:\n git_protocol: ssh' in self.saved() and self.no_forwarded_saved()) + self.verify('host-setting delta persists at host and account positions', 'enterprise.two.test:\n git_protocol: ssh' in self.saved() and ' host-user:\n git_protocol: ssh' in self.saved()) + finally: + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + self.verify('metadata alias removed', self.gh('alias', 'delete', 'metadata-pending').returncode == 0) + (self.original / 'hosts.yml').unlink() + + def check_authentication(self): + self.verify('all hosts forwarded through real RPC and HTTP relay', all(self.api(h) for h in HOSTS)) + self.verify('real auth status', self.gh('auth', 'status').returncode == 0) + self.verify('forwarding never saved', self.no_forwarded_saved()) + r = self.gh('api', 'user', '--jq', '.login', env={'GH_HOST': HOSTS[2]}) + self.verify('gh chooses GH_HOST', r.returncode == 0 and r.stdout.strip() == 'host-user') + r = self.gh('api', 'user', env={'GH_TOKEN': secrets[HOSTS[0]]['invalid'], 'DEVSY_LOG_LEVEL': 'debug'}) + self.verify('explicit invalid GH_TOKEN never falls back', r.returncode != 0 and 'gh relay: authentication source' in r.stderr and 'environment' in r.stderr) + self.verify('explicit invalid GITHUB_TOKEN never falls back', self.gh('api', 'user', env={'GITHUB_TOKEN': secrets[HOSTS[0]]['invalid']}).returncode != 0) + self.verify('explicit enterprise environment never falls back', self.gh('api', 'user', '--hostname', HOSTS[1], env={'GH_ENTERPRISE_TOKEN': secrets[HOSTS[1]]['invalid']}).returncode != 0) + + def check_local_authentication(self): + r = self.gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['forwarded']) + self.verify('same token login stays forwarded', r.returncode == 0 and self.no_forwarded_saved() and self.api(HOSTS[0])) + self.verify('auth switch uses forwarded account', self.gh('auth', 'switch', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and self.no_forwarded_saved()) + r = self.gh('auth', 'refresh', '--hostname', HOSTS[0], '--insecure-storage', env={'GH_BROWSER': '/bin/true'}) + self.verify('refresh saves a newly issued credential', r.returncode == 0 and secrets[HOSTS[0]]['refresh'] in self.saved() and self.api(HOSTS[0])) + self.verify('logout refreshed credential', self.gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'host-user').returncode == 0 and self.api(HOSTS[0])) + r = self.gh('auth', 'login', '--hostname', HOSTS[0], '--with-token', '--insecure-storage', token=secrets[HOSTS[0]]['local']) + self.verify('different token local login persists', r.returncode == 0 and secrets[HOSTS[0]]['local'] in self.saved() and self.api(HOSTS[0], 'local-user')) + r = self.gh('api', 'user', '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) + self.verify('local authentication preference explained at debug level', r.returncode == 0 and r.stdout == 'local-user\n' and 'gh relay: authentication source' in r.stderr and 'local_config' in r.stderr) + local_text = self.saved() + (self.original / 'hosts.yml').write_text(local_text.replace(secrets[HOSTS[0]]['local'], secrets[HOSTS[0]]['invalid'])) + self.verify('invalid local credential never falls back', not self.api(HOSTS[0])) + (self.original / 'hosts.yml').write_text(local_text) + r = self.gh('auth', 'logout', '--hostname', HOSTS[0], '--user', 'local-user') + self.verify('logout restores forwarding on next invocation', r.returncode == 0 and self.api(HOSTS[0]) and self.no_forwarded_saved()) + + def check_keyring(self): + r = self.gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['local']) + self.verify('keyring login succeeds without plaintext token', r.returncode == 0 and secrets[HOSTS[1]]['local'] not in self.saved() and self.api(HOSTS[1], 'local-user')) + r = self.gh('api', 'user', '--hostname', HOSTS[1], '--jq', '.login', env={'DEVSY_LOG_LEVEL': 'debug'}) + self.verify('existing keyring credential excluded from forwarding', r.returncode == 0 and r.stdout.strip() == 'local-user' and 'gh relay: authentication source' in r.stderr and 'keyring' in r.stderr) + self.verify('nested lookup preserves keyring authentication', self.nested_api(HOSTS[1], 'local-user')) + self.verify('unavailable keyring backend follows gh no-token result', self.api(HOSTS[1], env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/nonexistent'})) + self.verify('unavailable backend leaves local keyring intact', self.api(HOSTS[1], 'local-user')) + revoked = ROOT / 'revoke-local' + revoked.touch() + try: + self.verify('expired keyring credential never falls back', not self.api(HOSTS[1])) + self.verify('nested expired keyring credential never falls back', not self.nested_api(HOSTS[1])) + finally: + revoked.unlink() + forbidden = ROOT / 'forbid-local' + forbidden.touch() + try: + r = self.gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) + self.verify('insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr) + r = self.nested_gh('api', 'repos/fixture/private', '--hostname', HOSTS[1]) + self.verify('nested insufficient keyring permission never falls back', r.returncode != 0 and '403' in r.stderr) + finally: + forbidden.unlink() + r = self.gh('auth', 'status', '--hostname', HOSTS[1]) + self.verify('real auth status identifies keyring', r.returncode == 0 and 'keyring' in r.stdout + r.stderr) + self.verify('keyring logout restores forwarding', self.gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'local-user').returncode == 0 and self.api(HOSTS[1])) + + r = self.gh('auth', 'login', '--hostname', HOSTS[1], '--with-token', token=secrets[HOSTS[1]]['forwarded']) + self.verify('same token explicit keyring login retained', r.returncode == 0 and 'host-user' in self.saved() and self.no_forwarded_saved() and self.api(HOSTS[1])) + self.verify('same token keyring logout restores forwarding', self.gh('auth', 'logout', '--hostname', HOSTS[1], '--user', 'host-user').returncode == 0 and self.api(HOSTS[1])) + + def check_extensions(self): + self.verify('alias saves', self.gh('alias', 'set', 'save-editor', '!gh config set editor nano').returncode == 0) + self.verify('alias nested configuration persists', self.gh('save-editor').returncode == 0 and self.gh('config', 'get', 'editor').stdout.strip() == 'nano') + ext = ROOT / 'guest-data' / 'gh' / 'extensions' / 'gh-probe' + ext.mkdir(parents=True) + (ext / 'gh-probe').write_text('#!/bin/sh\ngh config set pager cat\ngh api user --hostname enterprise.two.test --jq .login\nprintf "extension-out\\n"\nprintf "extension-err\\n" >&2\nexit 7\n') + (ext / 'gh-probe').chmod(0o755) + r = self.gh('probe') + self.verify('extension streams and failure exit pass unchanged', r.returncode == 7 and r.stdout == 'host-user\nextension-out\n' and r.stderr == 'extension-err\n') + self.verify('extension change persists after failure', self.gh('config', 'get', 'pager').stdout.strip() == 'cat') + + def check_concurrent_updates(self): + self.config = self.original / 'config.yml' + self.target = ROOT / 'dotfiles-config.yml' + self.config.rename(self.target) + self.config.symlink_to(self.target) + self.ready, self.release = ROOT / 'command-ready', ROOT / 'command-release' + self.verify('concurrent alias saves', self.gh('alias', 'set', 'pending-editor', '!gh config set editor pending; touch "$GH_TEST_READY"; while test ! -f "$GH_TEST_RELEASE"; do sleep 0.05; done').returncode == 0) + p = self.pending() + try: + self.verify('concurrent invocation reached user mutation', self.wait_ready()) + self.verify('parallel gh saves unrelated pager', self.gh('config', 'set', 'pager', 'less').returncode == 0) + self.target.write_text(self.target.read_text().replace('editor: nano', 'editor: emacs') + '\nprompt: disabled\n') + self.release.touch() + p.communicate(timeout=20) + self.verify('parallel and dotfiles changes survive', p.returncode == 0 and self.gh('config', 'get', 'editor').stdout.strip() == 'pending' and self.gh('config', 'get', 'pager').stdout.strip() == 'less' and self.gh('config', 'get', 'prompt').stdout.strip() == 'disabled' and self.config.is_symlink()) + finally: + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + self.ready.unlink() + self.release.unlink() + self.verify('prepare interrupted mutation', self.gh('config', 'set', 'editor', 'vi').returncode == 0) + p = self.pending() + try: + self.verify('interrupted invocation reached mutation', self.wait_ready()) + finally: os.killpg(p.pid, signal.SIGKILL) p.communicate() - r = gh('config', 'get', 'editor', env={'DEVSY_LOG_LEVEL': 'debug'}) - verify('pending writeback recovered after original repaired', r.returncode == 0 and r.stdout.strip() == 'pending' and 'gh relay: configuration recovered' in r.stderr) - - - ready.unlink() - release.unlink() - verify('prepare live-child ownership', gh('config', 'set', 'editor', 'vi').returncode == 0) - p = pending() - try: - verify('live child reached mutation', wait_ready()) - os.kill(p.pid, signal.SIGKILL) - r = gh('config', 'get', 'editor') - verify('live child prevents premature recovery', r.returncode == 0 and r.stdout.strip() == 'vi' and bool(list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*')))) - release.touch() - p.communicate(timeout=20) - verify('completed orphan child changes recover', gh('config', 'get', 'editor').stdout.strip() == 'pending') - finally: - if p.poll() is None: + self.verify('next invocation recovers killed wrapper mutation', self.gh('config', 'get', 'editor').stdout.strip() == 'pending') + self.verify('recovered session contains no forwarded credentials', self.no_forwarded_saved()) + + def check_writeback(self): + self.ready.unlink() + self.verify('prepare pending writeback', self.gh('config', 'set', 'editor', 'vi').returncode == 0) + p = self.pending() + latest = None + try: + self.verify('writeback failure reached user mutation', self.wait_ready()) + latest = self.target.read_text() + self.target.write_text('[broken yaml') + self.release.touch() + _, writeback_stderr = p.communicate(timeout=20) + self.stderr_outputs.append(writeback_stderr.decode()) + self.verify('writeback failure does not change real gh exit', p.returncode == 0) + self.verify('writeback failure emits actionable error', 'gh relay: configuration writeback pending' in writeback_stderr.decode() and 'schema_invalid' in writeback_stderr.decode()) + finally: + if latest is not None: + self.target.write_text(latest) + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + r = self.gh('config', 'get', 'editor', env={'DEVSY_LOG_LEVEL': 'debug'}) + self.verify('pending writeback recovered after original repaired', r.returncode == 0 and r.stdout.strip() == 'pending' and 'gh relay: configuration recovered' in r.stderr) + + def check_orphan_children(self): + self.ready.unlink() + self.release.unlink() + self.verify('prepare live-child ownership', self.gh('config', 'set', 'editor', 'vi').returncode == 0) + p = self.pending() + try: + self.verify('live child reached mutation', self.wait_ready()) + os.kill(p.pid, signal.SIGKILL) + r = self.gh('config', 'get', 'editor') + self.verify('live child prevents premature recovery', r.returncode == 0 and r.stdout.strip() == 'vi' and bool(list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*')))) + self.release.touch() + p.communicate(timeout=20) + self.verify('completed orphan child changes recover', self.gh('config', 'get', 'editor').stdout.strip() == 'pending') + finally: + if p.poll() is None: + os.killpg(p.pid, signal.SIGKILL) + p.communicate() + + def check_host(self): + self.verify('host auth unchanged by guest operations', (ROOT / 'host-gh' / 'hosts.yml').read_bytes() == self.host_before and not (ROOT / 'host-gh' / 'config.yml').exists()) + + hp = ROOT / 'host-gh' / 'hosts.yml' + + self.host_before = hp.read_text() + try: + hp.write_text('{}\n') + r = self.gh('config', 'get', 'editor') + self.verify('no host accounts fall back without warning', r.returncode == 0 and 'gh relay:' not in r.stderr) + debug = self.gh('config', 'get', 'editor', env={'DEVSY_LOG_LEVEL': 'debug'}) + self.verify('no host accounts explained at debug level', debug.returncode == 0 and 'gh relay: host authentication' in debug.stderr and 'no_active_accounts' in debug.stderr) + hp.write_text('[broken yaml') + other = self.gh('config', 'get', 'editor') + self.verify('host authentication failure warns without changing gh exit', other.returncode == 0 and 'gh relay: forwarding unavailable' in other.stderr and 'config_decode_failed' in other.stderr) + finally: + hp.write_text(self.host_before) + text = hp.read_text().replace('host-user', 'second-user') + for v in secrets.values(): + text = text.replace(v['forwarded'], v['second']) + hp.write_text(text) + self.verify('host active account followed', self.api(HOSTS[2], 'second-user')) + r = subprocess.run(['runuser', '-u', 'vscode', '--', 'env', 'HOME=/home/vscode', 'GH_CONFIG_DIR=', 'XDG_CONFIG_HOME=', 'XDG_CACHE_HOME=/home/vscode/.cache', '/bin/bash', '-lc', 'exec gh api user --hostname enterprise.two.test --jq .login'], env=dict(os.environ, DBUS_SESSION_BUS_ADDRESS='unix:path=/nonexistent'), text=True, capture_output=True, timeout=20) + self.verify('non-root target user inherits wrapper and uses own HOME', r.returncode == 0 and r.stdout.strip() == 'second-user') + + def check_config_locations(self): + self.verify('local config dir override', self.gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '/fixture/custom-gh'}).returncode == 0 and (ROOT / 'custom-gh' / 'config.yml').exists()) + self.verify('XDG config location', self.gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '/fixture/custom-xdg'}).returncode == 0 and (ROOT / 'custom-xdg' / 'gh' / 'config.yml').exists()) + self.verify('HOME config location', self.gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '', 'HOME': '/fixture/custom-home'}).returncode == 0 and (ROOT / 'custom-home' / '.config' / 'gh' / 'config.yml').exists()) + self.verify('no diagnostic files created', not (ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics').exists()) + self.verify('stderr logs contain no credential values', all(secret not in output for output in self.stderr_outputs for values in secrets.values() for secret in values.values())) + self.verify('normal sessions cleaned', not list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*'))) + + def check_relay_loss(self): + self.ready.unlink() + self.release.unlink() + self.verify('prepare relay-loss recovery', self.gh('config', 'set', 'editor', 'vi').returncode == 0) + p = self.pending() + try: + self.verify('relay-loss mutation reached', self.wait_ready()) + finally: os.killpg(p.pid, signal.SIGKILL) p.communicate() - verify('host auth unchanged by guest operations', (ROOT / 'host-gh' / 'hosts.yml').read_bytes() == host_before and not (ROOT / 'host-gh' / 'config.yml').exists()) - - hp = ROOT / 'host-gh' / 'hosts.yml' - - host_before = hp.read_text() - try: - hp.write_text('{}\n') - r = gh('config', 'get', 'editor') - verify('no host accounts fall back without warning', r.returncode == 0 and 'gh relay:' not in r.stderr) - debug = gh('config', 'get', 'editor', env={'DEVSY_LOG_LEVEL': 'debug'}) - verify('no host accounts explained at debug level', debug.returncode == 0 and 'gh relay: host authentication' in debug.stderr and 'no_active_accounts' in debug.stderr) - hp.write_text('[broken yaml') - other = gh('config', 'get', 'editor') - verify('host authentication failure warns without changing gh exit', other.returncode == 0 and 'gh relay: forwarding unavailable' in other.stderr and 'config_decode_failed' in other.stderr) - finally: - hp.write_text(host_before) - text = hp.read_text().replace('host-user', 'second-user') - for v in secrets.values(): - text = text.replace(v['forwarded'], v['second']) - hp.write_text(text) - verify('host active account followed', api(HOSTS[2], 'second-user')) - r = subprocess.run(['runuser', '-u', 'vscode', '--', 'env', 'HOME=/home/vscode', 'GH_CONFIG_DIR=', 'XDG_CONFIG_HOME=', 'XDG_CACHE_HOME=/home/vscode/.cache', '/bin/bash', '-lc', 'exec gh api user --hostname enterprise.two.test --jq .login'], env=dict(os.environ, DBUS_SESSION_BUS_ADDRESS='unix:path=/nonexistent'), text=True, capture_output=True, timeout=20) - verify('non-root target user inherits wrapper and uses own HOME', r.returncode == 0 and r.stdout.strip() == 'second-user') - - verify('local config dir override', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '/fixture/custom-gh'}).returncode == 0 and (ROOT / 'custom-gh' / 'config.yml').exists()) - verify('XDG config location', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '/fixture/custom-xdg'}).returncode == 0 and (ROOT / 'custom-xdg' / 'gh' / 'config.yml').exists()) - verify('HOME config location', gh('config', 'set', 'editor', 'emacs', env={'GH_CONFIG_DIR': '', 'XDG_CONFIG_HOME': '', 'HOME': '/fixture/custom-home'}).returncode == 0 and (ROOT / 'custom-home' / '.config' / 'gh' / 'config.yml').exists()) - verify('no diagnostic files created', not (ROOT / 'guest-cache' / 'devsy' / 'gh-relay' / 'diagnostics').exists()) - verify('stderr logs contain no credential values', all(secret not in output for output in stderr_outputs for values in secrets.values() for secret in values.values())) - verify('normal sessions cleaned', not list((ROOT / 'guest-cache' / 'devsy' / 'gh-relay').glob('session-*'))) - - ready.unlink() - release.unlink() - verify('prepare relay-loss recovery', gh('config', 'set', 'editor', 'vi').returncode == 0) - p = pending() - try: - verify('relay-loss mutation reached', wait_ready()) - finally: - os.killpg(p.pid, signal.SIGKILL) - p.communicate() - print(json.dumps(results), flush=True) + def run(self): + self.check_installation() + self.check_dispatcher() + self.check_envfile() + self.check_dispatch() + self.check_overlay_and_selection() + self.check_self_references() + self.check_preferences() + self.check_authentication() + self.check_local_authentication() + self.check_keyring() + self.check_extensions() + self.check_concurrent_updates() + self.check_writeback() + self.check_orphan_children() + self.check_host() + self.check_config_locations() + self.check_relay_loss() + print(json.dumps(self.results), flush=True) if sys.argv[1] == 'server': server() elif sys.argv[1] == 'probe': probe() else: - check() + RelayChecks().run() diff --git a/e2e/tests/up-docker-compose/up_docker_compose.go b/e2e/tests/up-docker-compose/up_docker_compose.go index 8daac00df9..d2cec6a118 100644 --- a/e2e/tests/up-docker-compose/up_docker_compose.go +++ b/e2e/tests/up-docker-compose/up_docker_compose.go @@ -996,9 +996,9 @@ var _ = ginkgo.Describe( projectAPath = "tests/up-docker-compose/testdata/docker-compose-features-shared-image-a" projectBPath = "tests/up-docker-compose/testdata/docker-compose-features-shared-image-b" ) - commandPresenceCheck := func(command string) string { + commandVersionCheck := func(command string) string { return fmt.Sprintf( - "if command -v %s >/dev/null 2>&1; then echo present; else echo missing; fi", + "if %s --version >/dev/null 2>&1; then echo present; else echo missing; fi", command, ) } @@ -1043,7 +1043,7 @@ var _ = ginkgo.Describe( sharedImage, "sh", "-lc", - commandPresenceCheck("gh"), + commandVersionCheck("gh"), }, docker.Streams{Stdout: &hostGhOut, Stderr: io.Discard}, ) @@ -1067,7 +1067,7 @@ var _ = ginkgo.Describe( ghLookupOutput, err := tc.f.DevsySSH( ctx, tempDirB, - fmt.Sprintf("sh -lc %q", commandPresenceCheck("gh")), + fmt.Sprintf("sh -lc %q", commandVersionCheck("gh")), ) framework.ExpectNoError(err) gomega.Expect(strings.TrimSpace(ghLookupOutput)). @@ -1081,7 +1081,7 @@ var _ = ginkgo.Describe( nodeLookupOutput, err := tc.f.DevsySSH( ctx, tempDirA, - fmt.Sprintf("sh -lc %q", commandPresenceCheck("node")), + fmt.Sprintf("sh -lc %q", commandVersionCheck("node")), ) framework.ExpectNoError(err) gomega.Expect(strings.TrimSpace(nodeLookupOutput)). From 8ed92561bc2b3a8936bf8f128bed8cb75bc00b02 Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 21:43:06 +0900 Subject: [PATCH 11/20] fix(credentials): install the gh wrapper from non-root services MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Non-root browser and proxy service sessions could not create the wrapper in root-owned /usr/local/bin. Elevate only installation through the same plain sudo invocation style as the existing SSH signature helper. Keep root installs direct and keep credentials-server under its original user. Re-execute the current absolute Devsy binary via a hidden Cobra command that validates the relay port, requires root and installs only at the fixed destination. Reuse existing backup, locking and replacement logic. Report installation failures with safe operation/error classes and continue the credentials service. Keep installation when gh is absent for later lazy backend installation. No new sudo flags, prompting policy or I/O wiring. Validation: standard E2E build, targeted package tests and cli:lint:ci passed. The corrected non-root real-gh2.100.0 isolated fixture failed with only the previous installer overlaid, then passed with the change in75.206 seconds. It verifies live server identity, protected directory ownership, forwarding and existing auth/config/recovery contracts. Fake-sudo tests cover executable paths with spaces, exact arguments and child failure. Independent code and doc reviews accepted the final snapshot. Live browser workspace startup and hosted CI for this patch were not exercised. ## Transcript - U1: > PRのレビューの対応は必要ないか？ - A: Read current PR reviews and traced non-root browser service startup, wrapper installation and the host-gh fixture dependency. Identified non-root installation as a valid merge-blocking issue and declined the no-backend installation guard because it conflicts with lazy installation. - U2: > E2Eのホスト側gh依存 ← CIで成功しているならいいのでは？ - A: Accepted that a working CI host-gh prerequisite does not require a change for this PR and withdrew the assertion that it was mandatory. - U3: > 非root経路でのwrapper設置失敗 どう対応するの？ - A: Proposed elevating only wrapper installation while leaving credentials-server at its original identity, initially referencing the shared-file sudo fallback pattern. - U4: > 他のlocal binに配置する処理と同じってことだな？ - A: Clarified that other /usr/local/bin installation implementations had not yet been verified and did not claim identical code reuse. - U5: > 調べてよ - A: Traced existing SSH signature helper installation, RStudio setup and Docker credential helper placement. SSH signing uses plain sudo only for placement; RStudio installs during root setup; Docker helper writes directly. Concluded that elevating placement is consistent with the signing helper, with the existing Go backup and locking code retained. - U6: > sudoの振る舞いが既存を踏襲するようにして。変えたときにその実装の保守責任が発生するが、既存と同じならそれが根拠になる。 - A: Confirmed the clean atty303/gh-relay checkout at75060043f, fetched both remotes and implemented plain sudo without -n, environment-preservation flags, custom prompting or stream wiring. Root installation retains its existing direct path. Added a hidden Cobra installer with fixed destination, validated port and root guard; classified installation failures for a warning while retaining service continuation. Added process-boundary unit checks and changed the existing isolated real-gh case to start credentials-server asvscode against root-owned protected /usr/local/bin. Initial test attempts exposed a fixture PATH omission for runuser, corrected by its absolute path before causal comparison. - U7: > gh未導入ならwrapperを設置しない提案は採用しません。 ← それでいい。ghはコンテナに後から導入する(mise lazy tools)経路をサポートする意図だ - A: Preserved unconditional wrapper installation for later mise lazy tools without adding mise handling or a gh-presence guard. Using a Go compiler overlay of only the previous install.go, the corrected fixture showed a live non-root server answering /owner asvscode but no backup, then failed readiness in27.849 seconds. The modified implementation passed the same real-gh2.100.0 dummy-auth/mock-endpoint/keyring case in75.206 seconds, including non-root identity and protected directory checks plus existing auth/config/recovery checks. Standard E2E build, targeted package tests and final cli:lint:ci passed. Independent code review accepted the snapshot; doc review simplified the required G204 directive and accepted the final delta. Verified the prior Compose and CodeFactor CI failures are now passing on GitHub. No changes to host credentials, dotfiles, repo launcher or real workspaces; live browser-workspace startup and CI for this new patch remain unverified at this checkpoint. Co-authored-by: Codex --- .../agentcontainer/credentials_server.go | 2 +- cmd/internal/gh_test.go | 10 +++++++ cmd/internal/install_gh_wrapper.go | 28 ++++++++++++++++++ cmd/internal/internal.go | 1 + e2e/tests/ssh/gh_credentials.go | 29 +++++++++++-------- e2e/tests/ssh/testdata/gh-relay/fixture.py | 4 +++ pkg/ghcredentials/install.go | 14 +++++++-- pkg/ghcredentials/install_test.go | 24 +++++++++++++++ 8 files changed, 97 insertions(+), 15 deletions(-) create mode 100644 cmd/internal/install_gh_wrapper.go diff --git a/cmd/internal/agentcontainer/credentials_server.go b/cmd/internal/agentcontainer/credentials_server.go index e21da6f62e..8534d08c6f 100644 --- a/cmd/internal/agentcontainer/credentials_server.go +++ b/cmd/internal/agentcontainer/credentials_server.go @@ -121,7 +121,7 @@ func (cmd *CredentialsServerCmd) Run(ctx context.Context, port int) error { cmd.maybeForwardPorts(runCtx, tunnelClient) if cmd.ConfigureGitHubHelper { if err := ghcredentials.Install(port); err != nil { - log.Debug("GitHub CLI wrapper installation unavailable") + log.Warnf("GitHub CLI wrapper installation unavailable: %v", err) } } diff --git a/cmd/internal/gh_test.go b/cmd/internal/gh_test.go index e5d9b5c434..bada9e0387 100644 --- a/cmd/internal/gh_test.go +++ b/cmd/internal/gh_test.go @@ -47,3 +47,13 @@ func TestGHCommandForwardsArgumentsAndConfiguresLogging(t *testing.T) { }) } } + +func TestGHWrapperInstallCommandRejectsInvalidPorts(t *testing.T) { + for _, port := range []string{"0", "65536", "not-a-port"} { + t.Run(port, func(t *testing.T) { + cmd := NewInstallGHWrapperCmd() + cmd.SetArgs([]string{port}) + require.ErrorContains(t, cmd.Execute(), "invalid credentials server port") + }) + } +} diff --git a/cmd/internal/install_gh_wrapper.go b/cmd/internal/install_gh_wrapper.go new file mode 100644 index 0000000000..1c2dcfdffe --- /dev/null +++ b/cmd/internal/install_gh_wrapper.go @@ -0,0 +1,28 @@ +package cmdinternal + +import ( + "errors" + "os" + "strconv" + + "github.com/devsy-org/devsy/pkg/ghcredentials" + "github.com/spf13/cobra" +) + +func NewInstallGHWrapperCmd() *cobra.Command { + return &cobra.Command{ + Use: "install-gh-wrapper ", + Hidden: true, + Args: cobra.ExactArgs(1), + RunE: func(_ *cobra.Command, args []string) error { + port, err := strconv.Atoi(args[0]) + if err != nil || port < 1 || port > 65535 { + return errors.New("invalid credentials server port") + } + if os.Geteuid() != 0 { + return errors.New("GitHub CLI wrapper installation requires root") + } + return ghcredentials.Install(port) + }, + } +} diff --git a/cmd/internal/internal.go b/cmd/internal/internal.go index 1ca6173231..e27f19aa16 100644 --- a/cmd/internal/internal.go +++ b/cmd/internal/internal.go @@ -49,6 +49,7 @@ func NewInternalCmd(globalFlags *flags.GlobalFlags) *cobra.Command { cmd.AddCommand(withPreRun(NewGetImagePlatformsCmd(globalFlags))) cmd.AddCommand(withPreRun(NewBrowserTunnelCmd(globalFlags))) cmd.AddCommand(withPreRun(NewWidenSharedFileCmd(globalFlags))) + cmd.AddCommand(withPreRun(NewInstallGHWrapperCmd())) return cmd } diff --git a/e2e/tests/ssh/gh_credentials.go b/e2e/tests/ssh/gh_credentials.go index f24fe60493..c3638e868c 100644 --- a/e2e/tests/ssh/gh_credentials.go +++ b/e2e/tests/ssh/gh_credentials.go @@ -212,16 +212,7 @@ func prepareHostAuthentication(t ginkgo.GinkgoTInterface, dir string) { t.Setenv("GIT_CONFIG_NOSYSTEM", "1") } -func startFixture( - t ginkgo.GinkgoTInterface, - ctx context.Context, - dir string, - fixture ghRelayFixture, -) (context.CancelFunc, <-chan error) { - t.Helper() - name := fixture.name - image := name + ":test" - command := `cp /fixture/gh /usr/bin/gh +const ghRelayStartCommand = `cp /fixture/gh /usr/bin/gh cp /fixture/gh /usr/local/bin/gh cp /fixture/devsy /usr/local/bin/devsy mkdir -p /root/.local/share/keyrings @@ -232,9 +223,23 @@ exec dbus-run-session -- sh -c ' python3 /fixture/fixture.py server >/dev/null 2>/tmp/api.log & /usr/local/bin/devsy internal agent container ssh-server --address 127.0.0.1:12050 >/tmp/ssh.log 2>&1 & python3 /fixture/fixture.py probe - /usr/local/bin/devsy internal agent container credentials-server --user root --configure-github-helper + /usr/sbin/runuser -u vscode -- sh -c " + echo \$\$ > /home/vscode/gh-relay-server-pid + exec /usr/local/bin/devsy internal agent container credentials-server --user vscode --configure-github-helper + " sleep 600 '` + +func startFixture( + t ginkgo.GinkgoTInterface, + ctx context.Context, + dir string, + fixture ghRelayFixture, +) (context.CancelFunc, <-chan error) { + t.Helper() + name := fixture.name + image := name + ":test" + //nolint:gosec // All mounts, commands, and image names belong to this disposable fixture. guest := exec.CommandContext( ctx, @@ -255,7 +260,7 @@ exec dbus-run-session -- sh -c ' image, "sh", "-c", - command, + ghRelayStartCommand, ) reader, err := guest.StdoutPipe() require.NoError(t, err) diff --git a/e2e/tests/ssh/testdata/gh-relay/fixture.py b/e2e/tests/ssh/testdata/gh-relay/fixture.py index f0fe80dc2d..11e34030d7 100644 --- a/e2e/tests/ssh/testdata/gh-relay/fixture.py +++ b/e2e/tests/ssh/testdata/gh-relay/fixture.py @@ -164,6 +164,10 @@ def __init__(self): self.stderr_outputs = [] def check_installation(self): + server_pid = Path('/home/vscode/gh-relay-server-pid').read_text().strip() + server_uid = next(line.split()[1:] for line in Path('/proc', server_pid, 'status').read_text().splitlines() if line.startswith('Uid:')) + self.verify('credentials server stays non-root', all(uid != '0' for uid in server_uid)) + self.verify('wrapper directory remains root-owned and protected', Path('/usr/local/bin').stat().st_uid == 0 and Path('/usr/local/bin').stat().st_mode & 0o022 == 0) self.verify('SSH server predates wrapper installation', (ROOT / 'pre-install-gh').read_text().strip() == '/usr/local/bin/gh') self.verify('standard wrapper installed', Path('/usr/local/bin/gh').read_text().startswith('#!/bin/sh\n# devsy gh relay\n')) self.verify('real gh preserved in fixed backup', Path('/usr/local/bin/gh.devsy-original').read_bytes() == (ROOT / 'gh').read_bytes()) diff --git a/pkg/ghcredentials/install.go b/pkg/ghcredentials/install.go index 494e882a78..1c872f4d73 100644 --- a/pkg/ghcredentials/install.go +++ b/pkg/ghcredentials/install.go @@ -6,7 +6,9 @@ import ( "fmt" "io" "os" + "os/exec" "path/filepath" + "strconv" "time" "al.essio.dev/pkg/shellescape" @@ -21,9 +23,17 @@ const ( func Install(port int) error { binary, err := os.Executable() if err != nil { - return err + return boundary("install.binary", err) + } + if os.Geteuid() != 0 { + return boundary("install.sudo", installWithSudo(binary, port)) } - return installWrapper(wrapperPath, binary, port) + return boundary("install.wrapper", installWrapper(wrapperPath, binary, port)) +} + +func installWithSudo(binary string, port int) error { + //nolint:gosec + return exec.Command("sudo", binary, "internal", "install-gh-wrapper", strconv.Itoa(port)).Run() } func installWrapper(path, binary string, port int) error { diff --git a/pkg/ghcredentials/install_test.go b/pkg/ghcredentials/install_test.go index c29866c28a..ba5b582e6c 100644 --- a/pkg/ghcredentials/install_test.go +++ b/pkg/ghcredentials/install_test.go @@ -2,6 +2,7 @@ package ghcredentials import ( "os" + "os/exec" "path/filepath" "sync" "testing" @@ -9,6 +10,29 @@ import ( "github.com/stretchr/testify/require" ) +func TestInstallWithSudoPreservesArgumentsAndFailure(t *testing.T) { + dir := t.TempDir() + record := filepath.Join(dir, "args") + writeTestExecutable(t, filepath.Join(dir, "sudo"), []byte( + "#!/bin/sh\nprintf '%s\\n' \"$@\" > \"$DEVSY_TEST_SUDO_ARGS\"\n"+ + "exit \"$DEVSY_TEST_SUDO_EXIT\"\n", + )) + t.Setenv("PATH", dir) + t.Setenv("DEVSY_TEST_SUDO_ARGS", record) + t.Setenv("DEVSY_TEST_SUDO_EXIT", "0") + require.NoError(t, installWithSudo("/absolute path/devsy", 12049)) + root, err := os.OpenRoot(dir) + require.NoError(t, err) + defer func() { _ = root.Close() }() + args, err := root.ReadFile("args") + require.NoError(t, err) + require.Equal(t, "/absolute path/devsy\ninternal\ninstall-gh-wrapper\n12049\n", string(args)) + t.Setenv("DEVSY_TEST_SUDO_EXIT", "1") + var exit *exec.ExitError + require.ErrorAs(t, installWithSudo("/absolute path/devsy", 12049), &exit) + require.Equal(t, 1, exit.ExitCode()) +} + const ( testOriginalName = "real-gh" testExitScript = "#!/bin/sh\nexit 0" From 78b8d883d9cdc1f9672e2bed5bbbbc8a90800bee Mon Sep 17 00:00:00 2001 From: Koji AGAWA Date: Sun, 18 Oct 2026 22:45:34 +0900 Subject: [PATCH 12/20] fix(credentials): use the gh backup when PATH omits its wrapper MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Explicit wrapper invocations could exit 127 when PATH omitted the wrapper's directory even though its preserved GitHub CLI was executable. Use that backup when PATH traversal finds no executable, retaining the existing PATH order and backup selection at the wrapper's position. Resolve symlinked wrapper paths and reject a backup pointing to the wrapper itself. Add unit regressions for direct and symlinked invocations, PATH precedence, missing backups and recursive backups. The isolated real-gh E2E now makes a mock API request through an explicit wrapper with an isolated PATH. Full credential package tests, CLI lint and the focused SSH credential E2E pass. ## Transcript - U1: > 追加レビューは？ - A: Fetched the PR state and examined the additional review against the current source. One unresolved P2 identified explicit wrapper or GH_PATH invocations failing to consider the fixed backup when PATH omits the wrapper. Confirmed the selection loop only considered the backup upon encountering its own wrapper. Proposed a fallback after exhausting PATH, preserving ordinary PATH priority. - U2: > 修正してpushして対応を返信してresolve - A: Worked in the existing atty303/gh-relay checkout from clean HEAD 8ed92561b, fetched both remotes and preserved unrelated source and remote history. Added a regression and observed it fail before the implementation. Added validated backup resolution and the PATH-exhaustion fallback without changing authentication or writeback. Added isolated real-gh fixture coverage; full pkg/ghcredentials tests, cli:lint:ci, standard E2E build and the final focused SSH real-gh E2E passed. Rebuilt the final Linux arm64 agent and E2E binary. Independent code and public-reply documentation reviews found no remaining issues on the final patch. Verified the exact patch and user sequence at this commit checkpoint. Push, reply and thread resolution are explicitly authorized and pending this checkpoint. No real workspace, host credentials, dotfiles, launcher or mise configuration was changed; validation used isolated dummy credentials and mock services, not public GitHub or GHE authentication. Co-authored-by: Codex --- e2e/tests/ssh/testdata/gh-relay/fixture.py | 5 ++++ pkg/ghcredentials/install_test.go | 29 ++++++++++++++++++++++ pkg/ghcredentials/wrapper.go | 25 ++++++++++++++++--- 3 files changed, 55 insertions(+), 4 deletions(-) diff --git a/e2e/tests/ssh/testdata/gh-relay/fixture.py b/e2e/tests/ssh/testdata/gh-relay/fixture.py index 11e34030d7..e625b21974 100644 --- a/e2e/tests/ssh/testdata/gh-relay/fixture.py +++ b/e2e/tests/ssh/testdata/gh-relay/fixture.py @@ -251,6 +251,11 @@ def check_overlay_and_selection(self): r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--hostname', HOSTS[0], '--jq', '.login'], env=dict(os.environ, PATH=str(selected) + ':/usr/local/bin:/usr/bin:/bin', DEVSY_LOG_LEVEL='debug'), text=True, capture_output=True, timeout=20) self.stderr_outputs.append(r.stderr) self.verify('repo-selected gh precedes fixed backup', r.returncode == 0 and r.stdout.strip() == 'host-user' and 'gh relay: executable selected' in r.stderr and str(selected / 'gh') in r.stderr) + isolated = ROOT / 'isolated-path' + isolated.mkdir() + r = subprocess.run(['/usr/local/bin/gh', 'api', 'user', '--jq', '.login'], env=dict(os.environ, PATH=str(isolated), DEVSY_LOG_LEVEL='debug'), text=True, capture_output=True, timeout=20) + self.stderr_outputs.append(r.stderr) + self.verify('explicit wrapper uses backup outside PATH', r.returncode == 0 and r.stdout.strip() == 'host-user' and '/usr/local/bin/gh.devsy-original' in r.stderr) def check_self_references(self): git_config = Path.home() / '.gitconfig' diff --git a/pkg/ghcredentials/install_test.go b/pkg/ghcredentials/install_test.go index ba5b582e6c..d8a72ad8cf 100644 --- a/pkg/ghcredentials/install_test.go +++ b/pkg/ghcredentials/install_test.go @@ -174,6 +174,10 @@ func TestRealBinaryRespectsPATHAndBackup(t *testing.T) { got, err = realBinary(wrapper) require.NoError(t, err) require.Equal(t, selected, got) + t.Setenv("PATH", late) + got, err = realBinary(wrapper) + require.NoError(t, err) + require.Equal(t, later, got) require.NoError(t, os.Remove(selected)) require.NoError(t, os.Remove(backup)) require.NoError(t, os.Link(wrapper, backup)) @@ -181,3 +185,28 @@ func TestRealBinaryRespectsPATHAndBackup(t *testing.T) { require.NoError(t, err) require.Equal(t, later, got) } + +func TestRealBinaryUsesBackupOutsidePATH(t *testing.T) { + dir := t.TempDir() + wrapper := filepath.Join(dir, "gh") + require.NoError(t, installWrapper(wrapper, testInstallBinary, 12049)) + t.Setenv("PATH", t.TempDir()) + _, err := realBinary(wrapper) + require.ErrorIs(t, err, exec.ErrNotFound) + backup := wrapper + backupSuffix + writeTestExecutable(t, backup, []byte(testExitScript)) + got, err := realBinary(wrapper) + require.NoError(t, err) + want, err := filepath.EvalSymlinks(backup) + require.NoError(t, err) + require.Equal(t, want, got) + alias := filepath.Join(t.TempDir(), "gh") + require.NoError(t, os.Symlink(wrapper, alias)) + got, err = realBinary(alias) + require.NoError(t, err) + require.Equal(t, want, got) + require.NoError(t, os.Remove(backup)) + require.NoError(t, os.Link(wrapper, backup)) + _, err = realBinary(wrapper) + require.ErrorIs(t, err, exec.ErrNotFound) +} diff --git a/pkg/ghcredentials/wrapper.go b/pkg/ghcredentials/wrapper.go index 20341017dc..2a97a56a89 100644 --- a/pkg/ghcredentials/wrapper.go +++ b/pkg/ghcredentials/wrapper.go @@ -126,8 +126,8 @@ func (inv *invocation) withSession(s *session) int { } func realBinary(wrapper string) (string, error) { - installed, _ := filepath.EvalSymlinks(wrapper) self, _ := os.Stat(wrapper) + backup := wrapperBackup(wrapper, self) for _, dir := range filepath.SplitList(os.Getenv("PATH")) { path := filepath.Join(dir, "gh") info, err := executableInfo(path) @@ -135,18 +135,35 @@ func realBinary(wrapper string) (string, error) { continue } if self != nil && os.SameFile(self, info) { - backup := installed + backupSuffix - original, err := executableInfo(backup) - if err == nil && !os.SameFile(self, original) { + if backup != "" { return filepath.Abs(backup) } continue } return filepath.Abs(path) } + if backup != "" { + return filepath.Abs(backup) + } return "", exec.ErrNotFound } +func wrapperBackup(wrapper string, self os.FileInfo) string { + if self == nil { + return "" + } + installed, err := filepath.EvalSymlinks(wrapper) + if err != nil { + return "" + } + backup := installed + backupSuffix + original, err := executableInfo(backup) + if err != nil || os.SameFile(self, original) { + return "" + } + return backup +} + func replaceEnv(env []string, name, value string) []string { out := make([]string, 0, len(env)+1) for _, e := range env { From 988659c5c3b9892c26e6b2dbc3f06e191161a352 Mon Sep 17 00:00:00 2001 From: Samuel K Date: Mon, 19 Oct 2026 20:55:48 -0600 Subject: [PATCH 13/20] refactor(e2e): split credential race fixture checks --- .../ghfixture/auth_race_extras_linux.go | 32 +++++++++-- e2e/framework/ghfixture/auth_races_linux.go | 56 +++++++++++-------- 2 files changed, 59 insertions(+), 29 deletions(-) diff --git a/e2e/framework/ghfixture/auth_race_extras_linux.go b/e2e/framework/ghfixture/auth_race_extras_linux.go index 13ade7afe7..34d06d4e75 100644 --- a/e2e/framework/ghfixture/auth_race_extras_linux.go +++ b/e2e/framework/ghfixture/auth_race_extras_linux.go @@ -27,6 +27,14 @@ func (c *checks) checkIndependentHostRace() bool { } base[Hosts[0]] = c.expectedAuthentication(base[Hosts[0]], refreshOperation, Hosts[0]) base[Hosts[2]] = c.expectedAuthentication(raceHost{}, loginOperation, Hosts[2]) + return c.checkIndependentHostResult(prefix, p, base) +} + +func (c *checks) checkIndependentHostResult( + prefix string, + p *pendingCommand, + base map[string]raceHost, +) bool { return c.verify( prefix+"no authentication conflict", !strings.Contains(p.stderr.String(), "authentication_conflict"), @@ -72,6 +80,15 @@ func (c *checks) checkIndependentKeyringRace() bool { expected.User, expected.Token = secondUser, "" expected.Users[secondUser] = raceAccount{} base[host] = expected + return c.checkIndependentKeyringResult(prefix, p, base) +} + +func (c *checks) checkIndependentKeyringResult( + prefix string, + p *pendingCommand, + base map[string]raceHost, +) bool { + host := Hosts[0] token := c.originalCommand("", "auth", "token", "--hostname", host) return c.verify( prefix+"conflict reported", @@ -113,6 +130,13 @@ func (c *checks) checkLogoutRecovery() bool { return false } defer restoreFixture(c.target, config, err) + if !c.checkFailedLogoutWriteback(prefix) || !writeFixture(c.target, string(config)) { + return false + } + return c.finishLogoutRecovery(prefix, base) +} + +func (c *checks) checkFailedLogoutWriteback(prefix string) bool { p := c.pendingAlias(authRaceAlias) defer p.stop() if !c.verify(prefix+"mutation reached barrier", c.waitReady()) || @@ -121,14 +145,10 @@ func (c *checks) checkLogoutRecovery() bool { !c.verify(prefix+"failed writeback preserves exit", p.join()) { return false } - if !c.verify( + return c.verify( prefix+"failed writeback remains sanitized", c.hasSessions() && c.authArtifactsClean(), - ) || - !writeFixture(c.target, string(config)) { - return false - } - return c.finishLogoutRecovery(prefix, base) + ) } func (c *checks) finishLogoutRecovery(prefix string, base map[string]raceHost) bool { diff --git a/e2e/framework/ghfixture/auth_races_linux.go b/e2e/framework/ghfixture/auth_races_linux.go index c19306a149..2135ae5e9d 100644 --- a/e2e/framework/ghfixture/auth_races_linux.go +++ b/e2e/framework/ghfixture/auth_races_linux.go @@ -90,6 +90,15 @@ func (c *checks) checkAuthenticationRace(mode string, race authRace) bool { if !c.verify(prefix+"pending command preserves exit", completed) { return false } + return c.checkAuthenticationRaceResult(prefix, race, base, p) +} + +func (c *checks) checkAuthenticationRaceResult( + prefix string, + race authRace, + base map[string]raceHost, + p *pendingCommand, +) bool { if !c.verify( prefix+"conflict reported", strings.Contains(p.stderr.String(), "authentication_conflict"), @@ -252,31 +261,32 @@ func localRaceToken(account Tokens) string { func (c *checks) authArtifactsClean() bool { for _, root := range []string{c.original, filepath.Join(c.root, "guest-cache", "devsy", "gh-relay")} { - err := filepath.WalkDir(root, func(path string, entry os.DirEntry, err error) error { - if os.IsNotExist(err) { - return nil - } - if err != nil || entry.IsDir() { - return err - } - target, resolveErr := filepath.EvalSymlinks(path) - if resolveErr != nil { - return resolveErr - } - b, readErr := readFixture(target) - if readErr != nil { - return readErr - } - for _, account := range c.accounts { - if strings.Contains(string(b), account.Forwarded) { - return os.ErrInvalid - } - } - return nil - }) - if err != nil { + if filepath.WalkDir(root, c.checkAuthArtifact) != nil { return false } } return true } + +func (c *checks) checkAuthArtifact(path string, entry os.DirEntry, err error) error { + if os.IsNotExist(err) { + return nil + } + if err != nil || entry.IsDir() { + return err + } + target, resolveErr := filepath.EvalSymlinks(path) + if resolveErr != nil { + return resolveErr + } + b, readErr := readFixture(target) + if readErr != nil { + return readErr + } + for _, account := range c.accounts { + if strings.Contains(string(b), account.Forwarded) { + return os.ErrInvalid + } + } + return nil +} From d3e66a6a134e2d5216ae0551e9df38a95d1a5a3d Mon Sep 17 00:00:00 2001 From: Samuel K Date: Tue, 20 Oct 2026 11:36:50 -0600 Subject: [PATCH 14/20] fix(credentials): handle SSH relay handoff Retry transient occupied relay listeners without replacing an active owner. Isolate each public SSH fixture's API state and retain bounded failure information before cleanup. Cover draining listeners, cancellation, live owners, and io.Copy output bounds with regression tests. --- .../agentcontainer/credentials_port.go | 49 ++++++++ .../agentcontainer/credentials_port_test.go | 107 ++++++++++++++++++ .../agentcontainer/credentials_server.go | 2 +- e2e/framework/ghfixture/command_linux.go | 5 +- e2e/tests/ssh/gh_public.go | 35 ++++-- e2e/tests/ssh/gh_public_diagnostics.go | 56 +++++++++ e2e/tests/ssh/gh_public_diagnostics_test.go | 18 +++ e2e/tests/ssh/gh_public_session.go | 17 +-- 8 files changed, 268 insertions(+), 21 deletions(-) create mode 100644 cmd/internal/agentcontainer/credentials_port.go create mode 100644 cmd/internal/agentcontainer/credentials_port_test.go create mode 100644 e2e/tests/ssh/gh_public_diagnostics.go create mode 100644 e2e/tests/ssh/gh_public_diagnostics_test.go diff --git a/cmd/internal/agentcontainer/credentials_port.go b/cmd/internal/agentcontainer/credentials_port.go new file mode 100644 index 0000000000..dc278273d4 --- /dev/null +++ b/cmd/internal/agentcontainer/credentials_port.go @@ -0,0 +1,49 @@ +package agentcontainer + +import ( + "context" + "errors" + "net" + "time" + + "github.com/devsy-org/devsy/pkg/credentials" +) + +func (cmd *CredentialsServerCmd) claimPort(ctx context.Context, port int) (net.Listener, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + if !cmd.ConfigureGitHubHelper { + return claimPort(port) + } + return claimPortAfterDrain(ctx, port) +} + +func claimPortAfterDrain(ctx context.Context, port int) (net.Listener, error) { + claimCtx, cancel := context.WithTimeout(ctx, 2*time.Second) + defer cancel() + ticker := time.NewTicker(50 * time.Millisecond) + defer ticker.Stop() + for { + if err := ctx.Err(); err != nil { + return nil, err + } + ln, err := claimPort(port) + if !errors.Is(err, errPortOwnedByAnotherSession) { + return ln, err + } + _, probeErr := credentials.ProbeReadiness( + claimCtx, + port, + credentials.Readiness{GitHubAllowed: true}, + ) + if probeErr == nil { + return nil, err + } + select { + case <-claimCtx.Done(): + return nil, errors.Join(err, ctx.Err()) + case <-ticker.C: + } + } +} diff --git a/cmd/internal/agentcontainer/credentials_port_test.go b/cmd/internal/agentcontainer/credentials_port_test.go new file mode 100644 index 0000000000..b408e16ec7 --- /dev/null +++ b/cmd/internal/agentcontainer/credentials_port_test.go @@ -0,0 +1,107 @@ +package agentcontainer + +import ( + "context" + "encoding/json" + "net" + "net/http" + "testing" + "time" + + "github.com/devsy-org/devsy/pkg/credentials" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestGitHubCredentialsClaimWaitsForDrainingListener(t *testing.T) { + ln, err := net.Listen("tcp", "localhost:0") + require.NoError(t, err) + port := ln.Addr().(*net.TCPAddr).Port + server := &http.Server{ReadHeaderTimeout: time.Second} + server.Handler = http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusServiceUnavailable) + _ = ln.Close() + }) + joined := make(chan struct{}) + go func() { defer close(joined); _ = server.Serve(ln) }() + t.Cleanup(func() { _ = server.Close(); <-joined }) + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + cmd := &CredentialsServerCmd{ConfigureGitHubHelper: true} + claimed, err := cmd.claimPort(ctx, port) + require.NoError(t, err, "a draining prior session must not suppress the next service") + require.NoError(t, claimed.Close()) +} + +func TestGitHubCredentialsClaimPreservesLiveOwner(t *testing.T) { + ln, err := net.Listen("tcp", "localhost:0") + require.NoError(t, err) + port := ln.Addr().(*net.TCPAddr).Port + status := credentials.Readiness{ + Version: credentials.ReadinessVersion, Owner: "other-user", HostOwner: "other-host", + Instance: "live-instance", GitHubAllowed: true, GitHubState: "installed", + } + server := &http.Server{ + ReadHeaderTimeout: time.Second, + Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + assert.Equal(t, "/ready", r.URL.Path) + _ = json.NewEncoder(w).Encode(status) + }), + } + joined := make(chan struct{}) + go func() { defer close(joined); _ = server.Serve(ln) }() + t.Cleanup(func() { _ = server.Close(); <-joined }) + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + cmd := &CredentialsServerCmd{User: "vscode", HostOwner: "new-host", ConfigureGitHubHelper: true} + claimed, err := cmd.claimPort(ctx, port) + require.Nil(t, claimed) + require.ErrorIs(t, err, errPortOwnedByAnotherSession) + _, err = credentials.ProbeReadiness(ctx, port, status) + require.NoError(t, err, "a live owner must remain untouched") +} + +func TestGitHubCredentialsClaimCancellationPreservesUnresponsiveListener(t *testing.T) { + ln, err := net.Listen("tcp", "localhost:0") + require.NoError(t, err) + t.Cleanup(func() { _ = ln.Close() }) + port := ln.Addr().(*net.TCPAddr).Port + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + cmd := &CredentialsServerCmd{ConfigureGitHubHelper: true} + done := make(chan error, 1) + go func() { + claimed, claimErr := cmd.claimPort(ctx, port) + if claimed != nil { + _ = claimed.Close() + } + done <- claimErr + }() + require.NoError(t, ln.(*net.TCPListener).SetDeadline(time.Now().Add(3*time.Second))) + conn, err := ln.Accept() + require.NoError(t, err) + t.Cleanup(func() { _ = conn.Close() }) + cancel() + err = <-done + require.ErrorIs(t, err, context.Canceled) + require.ErrorIs(t, err, errPortOwnedByAnotherSession) + _, err = claimPort(port) + require.ErrorIs(t, err, errPortOwnedByAnotherSession) +} + +func TestGitHubCredentialsClaimBoundsUnavailableListener(t *testing.T) { + ln, err := net.Listen("tcp", "localhost:0") + require.NoError(t, err) + t.Cleanup(func() { _ = ln.Close() }) + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + cmd := &CredentialsServerCmd{ConfigureGitHubHelper: true} + claimed, err := cmd.claimPort(ctx, ln.Addr().(*net.TCPAddr).Port) + require.Nil(t, claimed) + require.ErrorIs(t, err, errPortOwnedByAnotherSession) + require.NoError( + t, + ctx.Err(), + "an unavailable listener must not consume the caller's entire deadline", + ) +} diff --git a/cmd/internal/agentcontainer/credentials_server.go b/cmd/internal/agentcontainer/credentials_server.go index 84d78f09d0..a01068da0a 100644 --- a/cmd/internal/agentcontainer/credentials_server.go +++ b/cmd/internal/agentcontainer/credentials_server.go @@ -102,7 +102,7 @@ func (cmd *CredentialsServerCmd) Run(ctx context.Context, port int) error { runCtx, cancel := context.WithCancel(ctx) defer cancel() - ln, err := claimPort(port) + ln, err := cmd.claimPort(ctx, port) if err != nil { if errors.Is(err, errPortOwnedByAnotherSession) { cmd.logPortOwnedByAnotherSession(ctx, port) diff --git a/e2e/framework/ghfixture/command_linux.go b/e2e/framework/ghfixture/command_linux.go index 150696674f..333df8e4f6 100644 --- a/e2e/framework/ghfixture/command_linux.go +++ b/e2e/framework/ghfixture/command_linux.go @@ -7,6 +7,7 @@ import ( "context" "encoding/json" "errors" + "fmt" "io" "os" "os/exec" @@ -415,9 +416,11 @@ func readFixture(path string) ([]byte, error) { func serveFixture(ctx context.Context, root string, accounts Accounts, certificatePath string) int { server, err := StartAPI(root, certificatePath, "127.0.0.1:8123", accounts) if err != nil { + _, _ = fmt.Fprintf(os.Stderr, "fixture API startup failed: %v\n", err) return 1 } - if os.WriteFile(filepath.Join(root, "api-ready"), nil, 0o600) != nil { + if err := os.WriteFile(filepath.Join(root, "api-ready"), nil, 0o600); err != nil { + _, _ = fmt.Fprintf(os.Stderr, "fixture API readiness publication failed: %v\n", err) return 1 } <-ctx.Done() diff --git a/e2e/tests/ssh/gh_public.go b/e2e/tests/ssh/gh_public.go index d1799a42cb..2027d6262e 100644 --- a/e2e/tests/ssh/gh_public.go +++ b/e2e/tests/ssh/gh_public.go @@ -19,6 +19,8 @@ const publicGHAPICommand = "GH_CONFIG_DIR=/tmp/public-gh HTTPS_PROXY=http://127. const publicGHUser = "host-user" +const flagNoServices = "--start-services=false" + // This exercises the public command with no credential-listener or wrapper // polling. Only the independent, dummy GitHub API starts during workspace setup. func testPublicGitHubCLIForwarding(ctx context.Context) { @@ -68,10 +70,11 @@ type publicGHFixture struct { func (p *publicGHFixture) testUser(t ginkgo.GinkgoTInterface, ctx context.Context, user string) { t.Helper() workspace := t.TempDir() + apiDir := p.apiDirectory(t) configuration := map[string]any{ "image": p.fixture.name + ":test", "remoteUser": user, - "mounts": []string{"type=bind,source=" + p.dir + ",target=/fixture"}, + "mounts": []string{"type=bind,source=" + apiDir + ",target=/fixture"}, "postCreateCommand": `sudo -n sh -c 'nohup /opt/ghfixture server-public >/tmp/fixture-api.log 2>&1 remaining { + p = p[:remaining] + } + _, _ = b.buffer.Write(p) + return n, nil +} + +func (b *publicSessionStderr) String() string { + b.mu.Lock() + defer b.mu.Unlock() + return b.buffer.String() +} + +func (p *publicGHFixture) registerDiagnostics( + t ginkgo.GinkgoTInterface, + workspace string, + deleted *bool, +) { + t.Helper() + t.Cleanup(func() { + if !t.Failed() || *deleted { + return + } + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) + defer cancel() + out, _, err := p.f.ExecCommandCapture(ctx, []string{ + cmdWorkspace, + cmdSSH, + workspace, + flagNoServices, + flagCommand, + "head -c 4096 /tmp/fixture-api.log; pgrep -a ghfixture; stat -c '%a %u %g %n' /fixture/api-ready /fixture/cert.pem", + }) + t.Logf("public API fixture diagnostics: %s (error: %v)", out, err) + }) +} diff --git a/e2e/tests/ssh/gh_public_diagnostics_test.go b/e2e/tests/ssh/gh_public_diagnostics_test.go new file mode 100644 index 0000000000..8777384e1f --- /dev/null +++ b/e2e/tests/ssh/gh_public_diagnostics_test.go @@ -0,0 +1,18 @@ +package ssh + +import ( + "io" + "strings" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestPublicSessionStderrKeepsBoundThroughIOCopy(t *testing.T) { + input := strings.Repeat("x", 32768) + var output publicSessionStderr + n, err := io.Copy(&output, io.LimitReader(strings.NewReader(input), int64(len(input)))) + require.NoError(t, err) + require.Equal(t, int64(len(input)), n) + require.Equal(t, input[:8192], output.String()) +} diff --git a/e2e/tests/ssh/gh_public_session.go b/e2e/tests/ssh/gh_public_session.go index af2676dfc9..1225bc145f 100644 --- a/e2e/tests/ssh/gh_public_session.go +++ b/e2e/tests/ssh/gh_public_session.go @@ -29,7 +29,8 @@ func (p *publicGHFixture) testLiveServiceReuse( cmd.WaitDelay = 250 * time.Millisecond stdout, err := cmd.StdoutPipe() require.NoError(t, err) - cmd.Stderr = io.Discard + var stderr publicSessionStderr + cmd.Stderr = &stderr require.NoError(t, cmd.Start()) done := make(chan struct{}) go func() { _ = cmd.Wait(); close(done) }() @@ -45,14 +46,14 @@ func (p *publicGHFixture) testLiveServiceReuse( }() select { case line := <-first: - require.Equal( - t, - publicGHUser, - line, - "first public command must authenticate without a setup wait", - ) + if line != publicGHUser { + stop() + require.Equal(t, publicGHUser, line, + "first public command must authenticate without a setup wait: %s", stderr.String()) + } case <-time.After(20 * time.Second): - t.Fatal("first public GitHub command did not complete") + stop() + t.Fatalf("first public GitHub command did not complete: %s", stderr.String()) } out, err := p.f.DevsySSHOnce(ctx, workspace, publicGHAPICommand) require.NoError(t, err, "second connection must reuse the still-live owner") From b2f2579f96157588414c0fbae662c51cad135f5a Mon Sep 17 00:00:00 2001 From: Samuel K Date: Tue, 20 Oct 2026 12:54:43 -0600 Subject: [PATCH 15/20] fix(credentials): validate authentication recovery input --- .../agentcontainer/credentials_status_test.go | 43 ++++++ e2e/framework/ghfixture/auth_races_linux.go | 2 +- e2e/framework/ghfixture/cases.go | 14 ++ e2e/framework/ghfixture/checks_linux.go | 3 +- .../ghfixture/malformed_recovery_linux.go | 127 ++++++++++++++++++ pkg/ghcredentials/auth_merge.go | 20 +-- pkg/ghcredentials/config.go | 15 ++- pkg/ghcredentials/config_test.go | 19 +++ pkg/ghcredentials/host_real_test.go | 28 ++++ pkg/ghcredentials/hosts_config.go | 66 +++++++++ pkg/ghcredentials/hosts_config_test.go | 66 +++++++++ pkg/ghcredentials/journal_test.go | 102 ++++++++++++++ pkg/ghcredentials/session.go | 6 +- pkg/ghcredentials/session_test.go | 8 +- pkg/tunnel/browser_preparation_test.go | 2 +- pkg/tunnel/services_preparation.go | 9 +- pkg/tunnel/services_preparation_test.go | 60 ++++++++- pkg/tunnel/services_test.go | 4 +- 18 files changed, 567 insertions(+), 27 deletions(-) create mode 100644 cmd/internal/agentcontainer/credentials_status_test.go create mode 100644 e2e/framework/ghfixture/malformed_recovery_linux.go create mode 100644 pkg/ghcredentials/hosts_config.go create mode 100644 pkg/ghcredentials/hosts_config_test.go diff --git a/cmd/internal/agentcontainer/credentials_status_test.go b/cmd/internal/agentcontainer/credentials_status_test.go new file mode 100644 index 0000000000..e6cec3a5c7 --- /dev/null +++ b/cmd/internal/agentcontainer/credentials_status_test.go @@ -0,0 +1,43 @@ +package agentcontainer + +import ( + "context" + "encoding/json" + "net" + "net/http" + "net/http/httptest" + "sync/atomic" + "testing" + "time" + + "github.com/devsy-org/devsy/pkg/credentials" + "github.com/stretchr/testify/require" +) + +func TestCredentialsStatusWaitsThroughPriorSessionIdentity(t *testing.T) { + current := credentials.Readiness{ + Version: credentials.ReadinessVersion, Owner: "vscode", HostOwner: "current-host", + Instance: "current-instance", GitHubAllowed: true, GitHubState: "installed", + } + prior := current + prior.HostOwner, prior.Instance = "prior-host", "prior-instance" + prior.GitHubAllowed, prior.GitHubState = false, "disabled" + var requests atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + status := current + if requests.Add(1) == 1 { + status = prior + } + _ = json.NewEncoder(w).Encode(status) + })) + defer server.Close() + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + status := waitCredentialsStatus(ctx, server.Listener.Addr().(*net.TCPAddr).Port, current) + require.True( + t, + status.Matches(current), + "a prior listener must not terminate preparation before the new session is ready", + ) + require.GreaterOrEqual(t, requests.Load(), int32(2)) +} diff --git a/e2e/framework/ghfixture/auth_races_linux.go b/e2e/framework/ghfixture/auth_races_linux.go index 2135ae5e9d..33d148b58c 100644 --- a/e2e/framework/ghfixture/auth_races_linux.go +++ b/e2e/framework/ghfixture/auth_races_linux.go @@ -61,7 +61,7 @@ func (c *checks) checkAuthenticationRaces() bool { } } if !c.checkIndependentHostRace() || !c.checkIndependentKeyringRace() || - !c.checkLogoutRecovery() { + !c.checkLogoutRecovery() || !c.checkMalformedLatestHostsRecovery() { return false } return c.gh("", nil, "alias", "delete", authRaceAlias).code == 0 diff --git a/e2e/framework/ghfixture/cases.go b/e2e/framework/ghfixture/cases.go index f7f74980f1..879d03577d 100644 --- a/e2e/framework/ghfixture/cases.go +++ b/e2e/framework/ghfixture/cases.go @@ -1,6 +1,20 @@ package ghfixture var ExpectedChecks = []string{ + "malformed working hosts: alias prepared", + "malformed working hosts: child exit preserved", + "malformed working hosts: lost intent reported", + "malformed working hosts: original authentication preserved", + "malformed working hosts: forwarded credentials scrubbed", + "malformed working hosts: next invocation recovers", + "malformed latest hosts: alias prepared", + "malformed latest hosts: mutation reached barrier", + "malformed latest hosts: child exit preserved", + "malformed latest hosts: pending writeback sanitized", + "malformed latest hosts: repeated recovery preserves invalid original and CLI exit", + "malformed latest hosts: repaired original recovers", + "malformed latest hosts: deliberate local login preserved", + "malformed latest hosts: forwarded credentials absent from artifacts", "auth race different hosts: alias prepared", "auth race different hosts: mutation reached barrier", "auth race different hosts: parallel login succeeds", diff --git a/e2e/framework/ghfixture/checks_linux.go b/e2e/framework/ghfixture/checks_linux.go index 8b9fb22b86..f95400a281 100644 --- a/e2e/framework/ghfixture/checks_linux.go +++ b/e2e/framework/ghfixture/checks_linux.go @@ -14,7 +14,8 @@ func (c *checks) run() bool { for _, phase := range []func() bool{ c.checkInstallation, c.checkGuestRegistrations, c.checkDispatcher, c.checkEnvfile, c.checkDispatch, c.checkSelection, c.checkSelfReferences, c.checkPreferences, c.checkAuthentication, c.checkLocalAuthentication, - c.checkKeyring, c.checkExtensions, c.checkConcurrentUpdates, c.checkWriteback, c.checkOrphanChildren, + c.checkKeyring, c.checkExtensions, c.checkConcurrentUpdates, c.checkWriteback, + c.checkMalformedWorkingHosts, c.checkOrphanChildren, c.checkAuthenticationRaces, c.checkHost, c.checkConfigLocations, c.checkRelayLoss, } { if !phase() { diff --git a/e2e/framework/ghfixture/malformed_recovery_linux.go b/e2e/framework/ghfixture/malformed_recovery_linux.go new file mode 100644 index 0000000000..ee02ee32e7 --- /dev/null +++ b/e2e/framework/ghfixture/malformed_recovery_linux.go @@ -0,0 +1,127 @@ +//go:build linux + +package ghfixture + +import ( + "path/filepath" + "strings" +) + +func (c *checks) checkMalformedWorkingHosts() bool { + path := filepath.Join(c.original, "hosts.yml") + before, err := readFixture(path) + defer restoreFixture(path, before, err) + alias := `!umask 077 +file="$GH_CONFIG_DIR/hosts.yml" +scratch="$GH_CONFIG_DIR/malformed-hosts" +printf '%s\n' 'invalid.test:' ' users: [alice, bob]' > "$scratch" && +cat "$file" >> "$scratch" && mv "$scratch" "$file"` + if !c.verify( + "malformed working hosts: alias prepared", + c.gh("", nil, "alias", "set", "malformed-hosts", alias).code == 0, + ) { + return false + } + r := c.gh("", nil, "malformed-hosts") + if !c.verify("malformed working hosts: child exit preserved", r.code == 0) || + !c.verify( + "malformed working hosts: lost intent reported", + strings.Contains(r.stderr, "configuration writeback pending") && + strings.Contains(r.stderr, "session.lost_intent"), + ) || + !c.verify( + "malformed working hosts: original authentication preserved", + c.saved() == string(before), + ) || + !c.verify( + "malformed working hosts: forwarded credentials scrubbed", + c.authArtifactsClean(), + ) { + return false + } + return c.verify( + "malformed working hosts: next invocation recovers", + c.gh("", nil, "alias", "delete", "malformed-hosts").code == 0 && !c.hasSessions(), + ) +} + +func (c *checks) checkMalformedLatestHostsRecovery() bool { + path := filepath.Join(c.original, "hosts.yml") + before, err := readFixture(path) + defer restoreFixture(path, before, err) + base, ok := c.prepareAuthenticationRace("mixed", authRace{first: loginOperation}) + if !c.verify("malformed latest hosts: alias prepared", ok) { + return false + } + baseline, err := readFixture(path) + if err != nil { + return false + } + if !c.failMalformedLatestWriteback(path) { + return false + } + return c.finishMalformedLatestRecovery(path, baseline, base) +} + +func (c *checks) failMalformedLatestWriteback(path string) bool { + p := c.pendingAlias(authRaceAlias) + defer p.stop() + if !c.verify("malformed latest hosts: mutation reached barrier", c.waitReady()) { + return false + } + if !writeFixture(path, malformedLatestHosts) || !writeFixture(c.release, "") || + !c.verify("malformed latest hosts: child exit preserved", p.join()) { + return false + } + c.stderr = append(c.stderr, p.stderr.String()) + return c.verify( + "malformed latest hosts: pending writeback sanitized", + c.hasSessions() && c.authArtifactsClean(), + ) +} + +func (c *checks) finishMalformedLatestRecovery( + path string, + baseline []byte, + base map[string]raceHost, +) bool { + patch := map[string]string{envGHConfig: filepath.Join(c.root, "recovery-probe-gh")} + if !c.checkRepeatedMalformedRecovery(patch) || !writeFixture(path, string(baseline)) { + return false + } + r := c.gh("", patch, "--version") + if !c.verify( + "malformed latest hosts: repaired original recovers", + r.code == 0 && !c.hasSessions(), + ) { + return false + } + base[Hosts[0]] = c.expectedAuthentication(base[Hosts[0]], loginOperation, Hosts[0]) + return c.verify( + "malformed latest hosts: deliberate local login preserved", + c.originalIdentity(Hosts[0], secondUser) && c.authenticationMatches(base), + ) && + c.verify( + "malformed latest hosts: forwarded credentials absent from artifacts", + c.authArtifactsClean(), + ) +} + +func (c *checks) checkRepeatedMalformedRecovery(patch map[string]string) bool { + for range 2 { + if c.gh("", patch, "--version").code != 0 || c.saved() != malformedLatestHosts || + !c.hasSessions() || + !c.authArtifactsClean() { + return c.verify( + "malformed latest hosts: repeated recovery preserves invalid original and CLI exit", + false, + ) + } + } + return c.verify( + "malformed latest hosts: repeated recovery preserves invalid original and CLI exit", + true, + ) +} + +const malformedLatestHosts = "github.com:\n users: [alice, bob]\n" diff --git a/pkg/ghcredentials/auth_merge.go b/pkg/ghcredentials/auth_merge.go index 0bb98c2685..a3e85fa280 100644 --- a/pkg/ghcredentials/auth_merge.go +++ b/pkg/ghcredentials/auth_merge.go @@ -32,14 +32,14 @@ func authentication(h *yaml.Node) *yaml.Node { return nil } a := mapping() - for _, key := range []string{"user", "oauth_token"} { + for _, key := range []string{activeUserKey, oauthTokenKey} { put(a, key, cloneNode(get(h, key))) } if users := get(h, "users"); users != nil { registrations := mapping() for i := 0; i+1 < len(users.Content); i += 2 { u := mapping() - put(u, "oauth_token", cloneNode(get(users.Content[i+1], "oauth_token"))) + put(u, oauthTokenKey, cloneNode(get(users.Content[i+1], oauthTokenKey))) put(registrations, users.Content[i].Value, u) } if len(registrations.Content) > 0 { @@ -57,12 +57,12 @@ func preferences(h *yaml.Node) *yaml.Node { return nil } p := cloneNode(h) - put(p, "user", nil) - put(p, "oauth_token", nil) + put(p, activeUserKey, nil) + put(p, oauthTokenKey, nil) if users := get(p, "users"); users != nil { for _, key := range sortedKeys(users) { u := get(users, key) - put(u, "oauth_token", nil) + put(u, oauthTokenKey, nil) if u == nil || len(u.Content) == 0 { put(users, key, nil) } @@ -88,15 +88,15 @@ func sortedKeys(nodes ...*yaml.Node) []string { } func mergeHosts(before, desired, latest []byte) ([]byte, error) { - b, err := decode(before) + b, err := decodeHosts(before) if err != nil { return nil, err } - d, err := decode(desired) + d, err := decodeHosts(desired) if err != nil { return nil, err } - l, err := decode(latest) + l, err := decodeHosts(latest) if err != nil { return nil, err } @@ -150,13 +150,13 @@ func selectAuthentication(base, want, current *yaml.Node) (*yaml.Node, error) { func applyAuthentication(p, chosen *yaml.Node) { removeUnregisteredPreferences(p, get(chosen, "users")) - for _, key := range []string{"user", "oauth_token"} { + for _, key := range []string{activeUserKey, oauthTokenKey} { put(p, key, cloneNode(get(chosen, key))) } users := get(chosen, "users") for _, user := range sortedKeys(users) { u := ensure(ensure(p, "users"), user) - put(u, "oauth_token", cloneNode(get(get(users, user), "oauth_token"))) + put(u, oauthTokenKey, cloneNode(get(get(users, user), oauthTokenKey))) } } diff --git a/pkg/ghcredentials/config.go b/pkg/ghcredentials/config.go index 4e384d8140..ee79252094 100644 --- a/pkg/ghcredentials/config.go +++ b/pkg/ghcredentials/config.go @@ -1,6 +1,7 @@ package ghcredentials import ( + "bytes" "errors" "io" "os" @@ -12,8 +13,11 @@ import ( ) const ( - hostsFile = "hosts.yml" - configFile = "config.yml" + hostsFile = "hosts.yml" + // #nosec G101 -- YAML field name, not a credential. + oauthTokenKey = "oauth_token" + activeUserKey = "user" + configFile = "config.yml" ) func ConfigDir() (string, error) { @@ -68,11 +72,16 @@ func readFile(path string) ([]byte, error) { } func decode(data []byte) (*yaml.Node, error) { + decoder := yaml.NewDecoder(bytes.NewReader(data)) var doc yaml.Node - if err := yaml.Unmarshal(data, &doc); err != nil { + if err := decoder.Decode(&doc); err != nil && !errors.Is(err, io.EOF) { // YAML errors may include credential values. return nil, errInvalidConfig } + var trailing yaml.Node + if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) { + return nil, errInvalidConfig + } if len(doc.Content) == 0 { return mapping(), nil } diff --git a/pkg/ghcredentials/config_test.go b/pkg/ghcredentials/config_test.go index 97671c032d..b4b1657ae3 100644 --- a/pkg/ghcredentials/config_test.go +++ b/pkg/ghcredentials/config_test.go @@ -42,3 +42,22 @@ func TestNativeConfigDirectoryPrecedenceAndEmptyValues(t *testing.T) { require.NoError(t, err) require.Equal(t, os.Getenv("GH_CONFIG_DIR"), got) } + +func TestNativeConfigRejectsTrailingDocumentsAndMalformedContent(t *testing.T) { + for _, data := range []string{ + "{}\ninvalid.test:\n users: [alice, bob]\n", + "{}\n---\neditor: newer\n", + "{}\n---\n", + "{}\n...\n[malformed", + } { + _, err := decode([]byte(data)) + require.ErrorIs(t, err, errInvalidConfig) + } +} + +func TestNativeConfigAcceptsCompleteSingleDocuments(t *testing.T) { + for _, data := range []string{"", "# comment\n", "{}\n", "---\neditor: vi\n...\n# tail\n", "custom: [one, two]\n"} { + _, err := decode([]byte(data)) + require.NoError(t, err) + } +} diff --git a/pkg/ghcredentials/host_real_test.go b/pkg/ghcredentials/host_real_test.go index 4eeea016f8..2a3f6afd63 100644 --- a/pkg/ghcredentials/host_real_test.go +++ b/pkg/ghcredentials/host_real_test.go @@ -5,7 +5,9 @@ import ( "os" "os/exec" "path/filepath" + "strings" "testing" + "time" "github.com/stretchr/testify/require" ) @@ -54,3 +56,29 @@ enterprise.example.invalid: ) require.Equal(t, "dummy-unrelated-environment-token", os.Getenv("GH_TOKEN")) } + +func TestNativeRealCLIScalarUsernamesAcceptedByRelay(t *testing.T) { + requireRealGH(t) + dir := t.TempDir() + t.Setenv("GH_CONFIG_DIR", dir) + binary, err := exec.LookPath("gh") + require.NoError(t, err) + const host = "scalar-user-fixture.invalid" + for _, user := range []string{"12345", testScalarBooleanUser} { + data := host + ":\n user: " + user + "\n users:\n " + user + + ":\n oauth_token: dummy-scalar-user-token\n" + _, err := decodeHosts([]byte(data)) + require.NoError(t, err) + require.NoError(t, os.WriteFile(filepath.Join(dir, hostsFile), []byte(data), 0o600)) + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + //nolint:gosec // Installed CLI, private dummy configuration, and a reserved invalid hostname. + cmd := exec.CommandContext(ctx, binary, "auth", "token", "--hostname", host, "--user", user) + cmd.Env = hostLookupEnv(os.Environ()) + cmd.WaitDelay = 250 * time.Millisecond + out := &tokenOutput{} + err = runTokenProbe(cmd, out) + cancel() + require.NoError(t, err) + require.True(t, strings.TrimSpace(string(out.data)) == "dummy-scalar-user-token") + } +} diff --git a/pkg/ghcredentials/hosts_config.go b/pkg/ghcredentials/hosts_config.go new file mode 100644 index 0000000000..01909aa923 --- /dev/null +++ b/pkg/ghcredentials/hosts_config.go @@ -0,0 +1,66 @@ +package ghcredentials + +import "gopkg.in/yaml.v3" + +func decodeHosts(data []byte) (*yaml.Node, error) { + hosts, err := decode(data) + if err != nil { + return nil, err + } + if !validAuthMapping(hosts, func(_ string, host *yaml.Node) bool { return validHost(host) }) { + return nil, errInvalidConfig + } + return hosts, nil +} + +func validAuthMapping(n *yaml.Node, validValue func(string, *yaml.Node) bool) bool { + if nullable(n) { + return true + } + if n.Kind != yaml.MappingNode { + return false + } + seen := make(map[string]bool) + for i := 0; i+1 < len(n.Content); i += 2 { + key := n.Content[i] + if key.Kind != yaml.ScalarNode || seen[key.Value] || + !validValue(key.Value, n.Content[i+1]) { + return false + } + seen[key.Value] = true + } + return true +} + +func validHost(n *yaml.Node) bool { + return validAuthMapping(n, func(key string, value *yaml.Node) bool { + switch key { + case activeUserKey, oauthTokenKey: + return validAuthScalar(value) + case "users": + return validAuthMapping( + value, + func(_ string, user *yaml.Node) bool { return validUser(user) }, + ) + default: + return true + } + }) +} + +func validUser(n *yaml.Node) bool { + if n != nil && n.Kind == yaml.ScalarNode && n.Value == "" { + return true + } + return validAuthMapping(n, func(key string, value *yaml.Node) bool { + return key != oauthTokenKey || validAuthScalar(value) + }) +} + +func validAuthScalar(n *yaml.Node) bool { + return n == nil || n.Kind == yaml.ScalarNode +} + +func nullable(n *yaml.Node) bool { + return n == nil || n.Kind == yaml.ScalarNode && n.Tag == "!!null" +} diff --git a/pkg/ghcredentials/hosts_config_test.go b/pkg/ghcredentials/hosts_config_test.go new file mode 100644 index 0000000000..01867e977a --- /dev/null +++ b/pkg/ghcredentials/hosts_config_test.go @@ -0,0 +1,66 @@ +package ghcredentials + +import ( + "testing" + + "github.com/stretchr/testify/require" +) + +func TestHostsRejectMalformedAuthenticationAtEveryMergeBoundary(t *testing.T) { + for _, malformed := range []string{ + "github.com: [a, b]\n", + "github.com:\n users: [a, b]\n", + "github.com:\n users:\n alice: [a, b]\n", + "github.com:\n user: [alice, bob]\n", + "github.com:\n oauth_token: {secret: value}\n", + "github.com:\n users:\n alice:\n oauth_token: [a, b]\n", + "github.com:\n users: {}\n users: [a, b]\n", + "github.com:\n users:\n alice: {}\n alice: [a, b]\n", + } { + for index := range 3 { + inputs := [3][]byte{ + []byte(authFixture("alice", "dummy-a")), + []byte(authFixture("alice", "dummy-b")), + []byte(authFixture("alice", "dummy-a")), + } + inputs[index] = []byte(malformed) + _, err := mergeHosts(inputs[0], inputs[1], inputs[2]) + require.ErrorIs(t, err, errInvalidConfig) + } + } +} + +func TestHostsAcceptNullRegistrationsAndArbitraryPreferences(t *testing.T) { + data := []byte(`github.com: + user: alice + oauth_token: + users: + alice: + bob: {} + dave: "" + carol: + oauth_token: "" + custom: [one, two] + custom: [one, two] +`) + _, err := decodeHosts(data) + require.NoError(t, err) + _, err = mergeHosts(nil, data, nil) + require.NoError(t, err) + _, err = mergeYAML(nil, []byte("custom: [one, two]\n"), nil) + require.NoError(t, err) +} + +func TestHostsPreserveScalarUsernamesAcceptedByRealCLI(t *testing.T) { + for _, user := range []string{"12345", testScalarBooleanUser, "1.5"} { + data := []byte("github.com:\n user: " + user + "\n users:\n " + user + ": {}\n") + n, err := decodeHosts(data) + require.NoError(t, err) + require.True(t, registeredAccount(get(n, "github.com"))) + merged, err := mergeHosts(data, data, data) + require.NoError(t, err) + require.Equal(t, data, merged) + } +} + +const testScalarBooleanUser = "true" diff --git a/pkg/ghcredentials/journal_test.go b/pkg/ghcredentials/journal_test.go index 3a14cbc5e9..2c15b65b5f 100644 --- a/pkg/ghcredentials/journal_test.go +++ b/pkg/ghcredentials/journal_test.go @@ -215,3 +215,105 @@ func TestMalformedWorkingAuthenticationDoesNotDeleteLocalLogin(t *testing.T) { require.NoError(t, err) require.True(t, string(b) == local, "discarded work deleted original authentication") } + +func TestJournalMalformedAuthenticationIsScrubbedBeforeRecovery(t *testing.T) { + for _, body := range []string{"users: [a, b]", "users:\n alice: [a, b]", "oauth_token: [a, b]"} { + t.Run(body, func(t *testing.T) { + s := journalSession(t) + local := authFixture("local-user", "dummy-local-login") + writeTestFile(t, filepath.Join(s.Original, hostsFile), local) + writeTestFile(t, filepath.Join(s.dir, "before", hostsFile), local) + writeTestFile( + t, + filepath.Join(s.work(), hostsFile), + "github.com:\n "+body+"\n extra: "+testForwardedToken+"\n", + ) + require.ErrorIs(t, s.finish(context.Background()), errLostIntent) + assertNoForwardedArtifacts(t, s.dir) + require.NoError(t, recoverSession(context.Background(), s.dir)) + saved, err := readFile(filepath.Join(s.Original, hostsFile)) + require.NoError(t, err) + require.True( + t, + string(saved) == local, + "malformed work changed original authentication", + ) + }) + } +} + +func TestJournalMalformedLatestAuthenticationRemainsPendingUntilRepaired(t *testing.T) { + for _, tc := range []struct{ name, data string }{ + {"structure", "github.com:\n users: [a, b]\n"}, + {"trailing content", "{}\nignored.test:\n user: newer-user\n"}, + {"second document", "{}\n---\nignored.test:\n user: newer-user\n"}, + } { + t.Run(tc.name, func(t *testing.T) { + malformed := tc.data + s := journalSession(t) + local := authFixture("local-user", "dummy-local-login") + writeTestFile(t, filepath.Join(s.work(), hostsFile), local) + path := filepath.Join(s.Original, hostsFile) + writeTestFile(t, path, malformed) + require.ErrorIs(t, s.finish(context.Background()), errInvalidConfig) + for range 2 { + assertNoForwardedArtifacts(t, s.dir) + require.ErrorIs(t, recoverSession(context.Background(), s.dir), errInvalidConfig) + saved, err := readFile(path) + require.NoError(t, err) + require.Equal(t, malformed, string(saved)) + } + require.NoError(t, os.Remove(path)) + require.NoError(t, recoverSession(context.Background(), s.dir)) + require.NoError(t, recoverSession(context.Background(), s.dir)) + saved, err := readFile(path) + require.NoError(t, err) + want, err := decodeHosts([]byte(local)) + require.NoError(t, err) + got, err := decodeHosts(saved) + require.NoError(t, err) + require.True(t, equal(want, got), "recovery changed deliberate local authentication") + assertNoForwardedArtifacts(t, filepath.Dir(s.dir)) + }) + } +} + +func TestJournalPreservesSecureLoginEmptyStringRegistration(t *testing.T) { + s := journalSession(t) + working := testHost + ":\n user: local-user\n users:\n " + testUser + + ":\n oauth_token: " + testForwardedToken + "\n local-user: \"\"\n" + writeTestFile(t, filepath.Join(s.work(), hostsFile), working) + require.NoError(t, s.finish(context.Background())) + assertNoForwardedArtifacts(t, filepath.Dir(s.dir)) + saved, err := readFile(filepath.Join(s.Original, hostsFile)) + require.NoError(t, err) + n, err := decodeHosts(saved) + require.NoError(t, err) + host := get(n, testHost) + require.Equal(t, "local-user", get(host, "user").Value) + require.NotNil(t, get(get(host, "users"), "local-user")) + a := Account{Host: testHost, User: testUser, Token: testForwardedToken} + next, err := startSession(context.Background(), s.Original, []Account{a}) + require.NoError(t, err) + t.Cleanup(func() { _ = next.lock.Close() }) + require.NoError(t, next.inject(context.Background(), "/usr/bin/false", nil)) + require.Empty(t, next.Accounts, "unavailable keyring must preserve the registered guest") + require.NoError(t, next.finish(context.Background())) +} + +func TestJournalTrailingWorkingAuthenticationIsScrubbedWithLostIntent(t *testing.T) { + for _, prefix := range []string{"{}\n", "{}\n---\n"} { + t.Run(prefix, func(t *testing.T) { + s := journalSession(t) + writeTestFile( + t, + filepath.Join(s.work(), hostsFile), + prefix+authFixture(testUser, testForwardedToken), + ) + require.ErrorIs(t, s.finish(context.Background()), errLostIntent) + assertNoForwardedArtifacts(t, s.dir) + require.NoError(t, recoverSession(context.Background(), s.dir)) + require.NoFileExists(t, filepath.Join(s.Original, hostsFile)) + }) + } +} diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index ee467d0bcd..c3115b7e69 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -339,7 +339,7 @@ func (s *session) inject(ctx context.Context, binary string, env []string) error if err != nil { return boundary("config.read", err) } - hosts, err := decode(b) + hosts, err := decodeHosts(b) if err != nil { return boundary("config.decode", err) } @@ -417,11 +417,11 @@ func copyConfigFiles(src, dst string) error { } func (s *session) strip(before, data []byte) ([]byte, error) { - original, err := decode(before) + original, err := decodeHosts(before) if err != nil { return nil, err } - hosts, err := decode(data) + hosts, err := decodeHosts(data) if err != nil { return nil, err } diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index 5246d4a6f3..3d116c502b 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -149,13 +149,17 @@ func TestDanglingDotfilesSymlinkKeepsPendingChanges(t *testing.T) { } func TestSessionPreservesTokenlessRegisteredIdentity(t *testing.T) { + const dotfilesUser = "dotfiles-user" for _, tc := range []struct { user, tokenField, userBody string }{ {testUser, "", "{}"}, - {"dotfiles-user", "", ""}, + {dotfilesUser, "", ""}, + {dotfilesUser, "", "\"\""}, + {"12345", "", "{}"}, + {testScalarBooleanUser, "", "{}"}, {testUser, " oauth_token: \"\"\n", "\n oauth_token: \"\""}, - {"dotfiles-user", " oauth_token:\n", "\n oauth_token:"}, + {dotfilesUser, " oauth_token:\n", "\n oauth_token:"}, } { t.Run(tc.user+tc.tokenField, func(t *testing.T) { root := t.TempDir() diff --git a/pkg/tunnel/browser_preparation_test.go b/pkg/tunnel/browser_preparation_test.go index e60f7ead37..92f76ba1e4 100644 --- a/pkg/tunnel/browser_preparation_test.go +++ b/pkg/tunnel/browser_preparation_test.go @@ -28,7 +28,7 @@ func TestBrowserAutoOpenWaitsForInstallationAndServing(t *testing.T) { if !waitPreparationSignal(ctx, serving) { return credentials.Readiness{GitHubState: credentialsUnavailable} } - return credentials.Readiness{GitHubState: "installed"} + return credentials.Readiness{GitHubState: installedPreparation} }, } go func() { diff --git a/pkg/tunnel/services_preparation.go b/pkg/tunnel/services_preparation.go index ed5f3760be..11de393433 100644 --- a/pkg/tunnel/services_preparation.go +++ b/pkg/tunnel/services_preparation.go @@ -39,7 +39,13 @@ type servicePreparation struct { func (p servicePreparation) start(ctx context.Context, enabled bool) credentials.Readiness { done := make(chan error, 1) - go func() { done <- p.run() }() + go func() { + err := p.run() + if err != nil && ctx.Err() == nil { + log.Debug("credential services stopped unexpectedly") + } + done <- err + }() if !enabled { return credentials.Readiness{GitHubState: "disabled"} } @@ -61,7 +67,6 @@ func (p servicePreparation) await( return status case err := <-done: if err != nil { - log.Debug("credential services preparation unavailable") return credentials.Readiness{GitHubState: credentialsUnavailable} } done = nil diff --git a/pkg/tunnel/services_preparation_test.go b/pkg/tunnel/services_preparation_test.go index 2203b481c1..64d222063c 100644 --- a/pkg/tunnel/services_preparation_test.go +++ b/pkg/tunnel/services_preparation_test.go @@ -3,11 +3,14 @@ package tunnel import ( "context" "errors" + "fmt" "testing" "time" "github.com/devsy-org/devsy/pkg/credentials" + "github.com/devsy-org/devsy/pkg/log" "github.com/stretchr/testify/require" + "go.uber.org/zap/zapcore" ) func TestServicePreparationWaitsForInstallAndServingWithoutEndingLifetime(t *testing.T) { @@ -28,7 +31,7 @@ func TestServicePreparationWaitsForInstallAndServingWithoutEndingLifetime(t *tes if !waitPreparationSignal(ctx, serve) { return credentials.Readiness{GitHubState: credentialsUnavailable} } - return credentials.Readiness{GitHubState: "installed"} + return credentials.Readiness{GitHubState: installedPreparation} }, } ready := make(chan credentials.Readiness, 1) @@ -39,7 +42,7 @@ func TestServicePreparationWaitsForInstallAndServingWithoutEndingLifetime(t *tes require.True(t, waitPreparationSignal(ctx, installDone)) assertPreparationPending(t, ready) close(serve) - require.Equal(t, "installed", receivePreparation(t, ctx, ready).GitHubState) + require.Equal(t, installedPreparation, receivePreparation(t, ctx, ready).GitHubState) assertPreparationPending(t, runDone) cancel() require.True(t, waitPreparationSignal(context.Background(), runDone)) @@ -136,3 +139,56 @@ const disabledPreparation = "disabled" const missingCLIPreparation = "missing-cli" const startupFailedPreparation = "startup-failed" + +func TestServicePreparationReportsFailureThroughoutServiceLifetime(t *testing.T) { + for _, tc := range []struct { + name string + enabled, ready, cancelled bool + }{ + {name: disabledPreparation}, + {name: "startup", enabled: true}, + {name: "after-ready", enabled: true, ready: true}, + {name: cancelledPreparation, enabled: true, cancelled: true}, + } { + t.Run(tc.name, func(t *testing.T) { + logs := log.InitTestObserved(t, zapcore.DebugLevel) + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + fail := make(chan struct{}) + runner := servicePreparation{ + run: func() error { <-fail; return errors.New("dummy-secret-must-not-be-logged") }, + probe: func(ctx context.Context) credentials.Readiness { + if tc.ready { + return credentials.Readiness{GitHubState: installedPreparation} + } + <-ctx.Done() + return credentials.Readiness{GitHubState: credentialsUnavailable} + }, + } + if tc.cancelled { + cancel() + } + if !tc.ready { + close(fail) + } + status := runner.start(ctx, tc.enabled) + if tc.ready { + require.Equal(t, installedPreparation, status.GitHubState) + close(fail) + } + if tc.cancelled { + require.Empty(t, logs.All()) + return + } + require.Eventually( + t, + func() bool { return logs.FilterMessage("credential services stopped unexpectedly").Len() == 1 }, + time.Second, + time.Millisecond, + ) + require.NotContains(t, fmt.Sprint(logs.All()), "dummy-secret") + }) + } +} + +const installedPreparation = "installed" diff --git a/pkg/tunnel/services_test.go b/pkg/tunnel/services_test.go index 806bd22cae..878686fbbe 100644 --- a/pkg/tunnel/services_test.go +++ b/pkg/tunnel/services_test.go @@ -163,7 +163,7 @@ func TestOptionalGitHubServiceFailurePreservesUnrelatedServices(t *testing.T) { } func TestOptionalGitHubServiceDoesNotRetrySuccessDisabledOrCanceled(t *testing.T) { - for _, mode := range []string{"success", "disabled", "canceled"} { + for _, mode := range []string{"success", disabledPreparation, "canceled"} { t.Run(mode, func(t *testing.T) { ctx, cancel := context.WithCancel(context.Background()) defer cancel() @@ -177,7 +177,7 @@ func TestOptionalGitHubServiceDoesNotRetrySuccessDisabledOrCanceled(t *testing.T } err := runOptionalGitHubServices( ctx, - RunServicesOptions{ConfigureGitHubCredentials: mode != "disabled"}, + RunServicesOptions{ConfigureGitHubCredentials: mode != disabledPreparation}, func(RunServicesOptions) error { calls++ return want From 6c407809be0ee6bf74e4902afc6cf8c7c1461d9f Mon Sep 17 00:00:00 2001 From: Samuel K Date: Tue, 20 Oct 2026 14:42:03 -0600 Subject: [PATCH 16/20] fix(credentials): serialize aliases and observe service exits --- e2e/framework/ghfixture/cases.go | 2 + e2e/framework/ghfixture/recovery_linux.go | 14 +- pkg/client/clientimplementation/services.go | 15 ++ pkg/ghcredentials/journal.go | 15 +- pkg/ghcredentials/journal_test.go | 25 ++- pkg/ghcredentials/session.go | 93 +++++++++-- pkg/ghcredentials/session_test.go | 106 +++++++++++++ pkg/ide/opener/opener.go | 12 +- pkg/tunnel/browser.go | 16 +- pkg/tunnel/browser_preparation_test.go | 167 +++++++++++++++++++- pkg/tunnel/services_preparation.go | 82 ++++++++-- 11 files changed, 493 insertions(+), 54 deletions(-) diff --git a/e2e/framework/ghfixture/cases.go b/e2e/framework/ghfixture/cases.go index 879d03577d..b91088ec02 100644 --- a/e2e/framework/ghfixture/cases.go +++ b/e2e/framework/ghfixture/cases.go @@ -230,6 +230,8 @@ var ExpectedChecks = []string{ "concurrent alias saves", "concurrent invocation reached user mutation", "parallel gh saves unrelated pager", + "configuration directory alias prepared", + "directory aliases observe both concurrent preferences", "parallel and dotfiles changes survive", "prepare interrupted mutation", "interrupted invocation reached mutation", diff --git a/e2e/framework/ghfixture/recovery_linux.go b/e2e/framework/ghfixture/recovery_linux.go index 09862e0f18..4db7bf575f 100644 --- a/e2e/framework/ghfixture/recovery_linux.go +++ b/e2e/framework/ghfixture/recovery_linux.go @@ -38,9 +38,14 @@ func (c *checks) checkConcurrentUpdates() bool { } func (c *checks) checkParallelConfigMutation(p *pendingCommand) bool { + alias := filepath.Join(c.root, "config-directory-alias") + if !c.verify("configuration directory alias prepared", os.Symlink(c.original, alias) == nil) { + return false + } + defer func() { _ = os.Remove(alias) }() if !c.verify( "parallel gh saves unrelated pager", - c.gh("", nil, "config", "set", "pager", "less").code == 0, + c.gh("", map[string]string{envGHConfig: alias}, "config", "set", "pager", "less").code == 0, ) { return false } @@ -55,6 +60,13 @@ func (c *checks) checkParallelConfigMutation(p *pendingCommand) bool { return false } completed := p.join() + if !c.verify("directory aliases observe both concurrent preferences", completed && + c.setting("editor") == pendingEditor && + strings.TrimSpace( + c.gh("", map[string]string{envGHConfig: alias}, "config", "get", "pager").stdout, + ) == "less") { + return false + } return c.checkParallelConfigResult(completed) } diff --git a/pkg/client/clientimplementation/services.go b/pkg/client/clientimplementation/services.go index ca658e254a..7a1ddc863a 100644 --- a/pkg/client/clientimplementation/services.go +++ b/pkg/client/clientimplementation/services.go @@ -52,6 +52,21 @@ func StartPreparedServicesDaemon( return tunnel.StartPreparedServices(ctx, services) } +func RunPreparedServicesDaemon( + ctx context.Context, + opts StartServicesDaemonOptions, + prepared func(credentials.Readiness), +) error { + services, err := daemonServiceOptions(ctx, opts) + if err != nil { + return err + } + if opts.User == "" { + return nil + } + return tunnel.RunPreparedServices(ctx, services, prepared) +} + func daemonServiceOptions( ctx context.Context, opts StartServicesDaemonOptions, diff --git a/pkg/ghcredentials/journal.go b/pkg/ghcredentials/journal.go index 66873b75aa..6fe52a2b17 100644 --- a/pkg/ghcredentials/journal.go +++ b/pkg/ghcredentials/journal.go @@ -116,9 +116,9 @@ func (s *session) checkpointAt(stage string) error { } func (s *session) planPublication(name string, latest, result []byte, conflict bool) error { - target, err := resolveOriginal(filepath.Join(s.Original, name)) + target, err := s.originalPath(name) if err != nil { - return boundary("config.target", err) + return err } mode := fs.FileMode(0o600) if info, statErr := os.Stat( @@ -166,9 +166,9 @@ func (s *session) publish(p *publication) error { } func (s *session) applyPublication(p *publication) error { - target, err := resolveOriginal(filepath.Join(s.Original, p.Name)) + target, err := s.originalPath(p.Name) if err != nil { - return boundary("config.target", err) + return err } current, err := readFile(target) if err != nil { @@ -197,6 +197,13 @@ func (s *session) writePublication(p *publication) error { if err := s.checkpointAt("publishing:" + p.Name); err != nil { return err } + target, err := s.originalPath(p.Name) + if err != nil { + return err + } + if target != p.Target { + return errAuthConflict + } if err := writePublication(p); err != nil { p.Publishing = false if saveErr := s.save(); saveErr != nil { diff --git a/pkg/ghcredentials/journal_test.go b/pkg/ghcredentials/journal_test.go index 2c15b65b5f..49470bbc82 100644 --- a/pkg/ghcredentials/journal_test.go +++ b/pkg/ghcredentials/journal_test.go @@ -143,7 +143,17 @@ func TestJournalMalformedWorkIsDiscardedWithoutRetainingForwardedToken(t *testin require.NoFileExists(t, filepath.Join(s.Original, hostsFile)) } +const duringPublishingStage = "during-publishing" + func TestJournalBindsPublicationToSymlinkTarget(t *testing.T) { + for _, stage := range []string{"planned:", "publishing:", duringPublishingStage} { + t.Run(stage, func(t *testing.T) { + testJournalBindsPublicationToSymlinkTarget(t, stage) + }) + } +} + +func testJournalBindsPublicationToSymlinkTarget(t *testing.T, stagePrefix string) { s := journalSession(t) one, two := filepath.Join(s.Original, "one"), filepath.Join(s.Original, "two") writeTestFile(t, one, "editor: vi\n") @@ -152,15 +162,24 @@ func TestJournalBindsPublicationToSymlinkTarget(t *testing.T) { require.NoError(t, os.Remove(path)) require.NoError(t, os.Symlink(one, path)) writeTestFile(t, filepath.Join(s.work(), configFile), "editor: nano\n") + retarget := func() { + require.NoError(t, os.Remove(path)) + require.NoError(t, os.Symlink(two, path)) + } s.checkpoint = func(stage string) error { - if stage == "planned:config.yml" { + if stagePrefix == duringPublishingStage && stage == "publishing:"+configFile { + retarget() + return nil + } + if stage == stagePrefix+configFile { return errors.New("injected") } return nil } require.Error(t, s.finish(context.Background())) - require.NoError(t, os.Remove(path)) - require.NoError(t, os.Symlink(two, path)) + if stagePrefix != duringPublishingStage { + retarget() + } require.ErrorIs(t, recoverSession(context.Background(), s.dir), errAuthConflict) b, err := readFile(two) require.NoError(t, err) diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index c3115b7e69..0f81c5e361 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -21,16 +21,17 @@ import ( ) type session struct { - Version int `json:"version"` - Original string `json:"original"` - Overlays []overlayAccount `json:"overlays,omitempty"` - Sanitized bool `json:"sanitized"` - LostHosts bool `json:"lost_hosts,omitempty"` - Publications []publication `json:"publications,omitempty"` - Accounts []Account `json:"-"` - dir string - lock *ownerLock - checkpoint func(string) error + Version int `json:"version"` + Original string `json:"original"` + OriginalTarget string `json:"original_target,omitempty"` + Overlays []overlayAccount `json:"overlays,omitempty"` + Sanitized bool `json:"sanitized"` + LostHosts bool `json:"lost_hosts,omitempty"` + Publications []publication `json:"publications,omitempty"` + Accounts []Account `json:"-"` + dir string + lock *ownerLock + checkpoint func(string) error } func sessionRoot() (string, error) { @@ -72,6 +73,10 @@ func startSession(ctx context.Context, original string, accounts []Account) (*se } func newSession(ctx context.Context, root, original string, accounts []Account) (*session, error) { + target, err := resolveOriginal(original) + if err != nil { + return nil, boundary("snapshot.target", err) + } dir, err := os.MkdirTemp(root, "session-") if err != nil { return nil, err @@ -81,7 +86,14 @@ func newSession(ctx context.Context, root, original string, accounts []Account) _ = os.RemoveAll(dir) return nil, boundary("session.owner_lock", err) } - s := &session{Version: 2, Original: original, Accounts: accounts, dir: dir, lock: owner} + s := &session{ + Version: 2, + Original: original, + OriginalTarget: target, + Accounts: accounts, + dir: dir, + lock: owner, + } ready := false defer func() { if !ready { @@ -89,7 +101,7 @@ func newSession(ctx context.Context, root, original string, accounts []Account) _ = os.RemoveAll(dir) } }() - if err := copyTree(ctx, original, filepath.Join(dir, "before"), 0); err != nil { + if err := copyTree(ctx, target, filepath.Join(dir, "before"), 0); err != nil { return nil, boundary("snapshot.read", err) } if err := copyTree(ctx, filepath.Join(dir, "before"), s.work(), 0); err != nil { @@ -281,7 +293,7 @@ func (s *session) upgrade(b []byte) error { } func (s *session) validate() error { - if !filepath.IsAbs(s.Original) || s.Version != 2 { + if !s.validLocation() || s.Version != 2 { return errInvalidConfig } seen := map[string]bool{} @@ -299,6 +311,11 @@ func (s *session) validate() error { return nil } +func (s *session) validLocation() bool { + return filepath.IsAbs(s.Original) && + (s.OriginalTarget == "" || filepath.IsAbs(s.OriginalTarget)) +} + func (s *session) save() error { b, err := json.Marshal(s) if err != nil { @@ -460,12 +477,19 @@ func (s *session) sync(ctx context.Context) error { return err } root := filepath.Dir(s.dir) - key := sha256.Sum256([]byte(s.Original)) + target, err := s.bindOriginal() + if err != nil { + return err + } + key := sha256.Sum256([]byte(target)) l, err := lock(ctx, filepath.Join(root, fmt.Sprintf("config-%x.lock", key[:16]))) if err != nil { return boundary("config.lock", err) } defer func() { _ = l.Close() }() + if _, err := s.originalPath(""); err != nil { + return err + } ordered, err := s.changedNames() if err != nil { return err @@ -480,6 +504,42 @@ func (s *session) sync(ctx context.Context) error { return syncErr } +func (s *session) bindOriginal() (string, error) { + target, err := resolveOriginal(s.Original) + if err != nil { + return "", boundary("config.target", err) + } + if s.OriginalTarget == "" { + s.OriginalTarget = target + if err := s.save(); err != nil { + s.OriginalTarget = "" + return "", boundary("config.target_checkpoint", err) + } + } + if target != s.OriginalTarget { + return "", errors.New("GitHub CLI configuration directory changed") + } + return target, nil +} + +func (s *session) originalPath(name string) (string, error) { + target, err := s.bindOriginal() + if err != nil { + return "", err + } + path, err := resolveOriginal(filepath.Join(target, name)) + return path, boundary("config.target", err) +} + +func (s *session) readOriginal(name string) ([]byte, error) { + path, err := s.originalPath(name) + if err != nil { + return nil, err + } + data, err := readFile(path) + return data, boundary("config.latest_read", err) +} + func (s *session) changedNames() ([]string, error) { beforeDir := filepath.Join(s.dir, "before") names, err := files(beforeDir) @@ -694,10 +754,9 @@ func (s *session) syncFile(name string) error { if bytes.Equal(before, after) { return nil } - path := filepath.Join(s.Original, name) - latest, err := readFile(path) + latest, err := s.readOriginal(name) if err != nil { - return boundary("config.latest_read", err) + return err } after, err = s.mergeFile(name, before, after, latest) if err != nil && !errors.Is(err, errAuthConflict) { diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index 3d116c502b..87c670f6a7 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -63,6 +63,112 @@ func TestSessionConcurrentInvocationsAndRecovery(t *testing.T) { require.NoError(t, c.finish(context.Background())) } +func TestSessionDirectoryAliasesShareWritebackLock(t *testing.T) { + root := t.TempDir() + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + t.Setenv("HOME", root) + original, alias := filepath.Join(root, "gh"), filepath.Join(root, "alias") + writeTestFile(t, filepath.Join(original, configFile), "editor: vi\npager: less\n") + require.NoError(t, os.Symlink(original, alias)) + a, err := startSession(context.Background(), original, nil) + require.NoError(t, err) + t.Cleanup(func() { _ = a.lock.Close() }) + b, err := startSession(context.Background(), alias, nil) + require.NoError(t, err) + t.Cleanup(func() { _ = b.lock.Close() }) + writeTestFile(t, filepath.Join(a.work(), configFile), "editor: nano\npager: less\n") + writeTestFile(t, filepath.Join(b.work(), configFile), "editor: vi\npager: cat\n") + planned, release := make(chan struct{}), make(chan struct{}) + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + a.checkpoint = func(stage string) error { + if stage == "planned:"+configFile { + close(planned) + select { + case <-release: + return nil + case <-ctx.Done(): + return ctx.Err() + } + } + return nil + } + // The short-lived context makes B prove it contends before A publishes. + aDone := make(chan error, 1) + go func() { aDone <- a.sync(ctx) }() + select { + case <-planned: + case <-ctx.Done(): + t.Fatal("first writeback did not reach its publication barrier") + } + bCtx, bCancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer bCancel() + err = b.sync(bCtx) + close(release) + require.ErrorIs(t, err, context.DeadlineExceeded, "alias bypassed the configuration lock") + require.NoError(t, <-aDone) + require.NoError(t, b.sync(context.Background())) + assertMergedPreferences(t, original) +} + +func assertMergedPreferences(t *testing.T, original string) { + t.Helper() + data, err := readFile(filepath.Join(original, configFile)) + require.NoError(t, err) + n, err := decode(data) + require.NoError(t, err) + require.Equal(t, "nano", get(n, "editor").Value) + require.Equal(t, "cat", get(n, "pager").Value) +} + +func TestSessionRejectsRetargetedConfigurationDirectory(t *testing.T) { + for _, stage := range []string{"before writeback", "planned:" + configFile} { + t.Run(stage, func(t *testing.T) { + root := t.TempDir() + t.Setenv("HOME", root) + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + one := filepath.Join(root, "one") + two := filepath.Join(root, "two") + alias := filepath.Join(root, "alias") + writeTestFile(t, filepath.Join(one, configFile), "editor: vi\n") + writeTestFile(t, filepath.Join(two, configFile), "editor: vi\n") + require.NoError(t, os.Symlink(one, alias)) + s, err := startSession(context.Background(), alias, nil) + require.NoError(t, err) + t.Cleanup(func() { _ = s.lock.Close() }) + writeTestFile(t, filepath.Join(s.work(), configFile), "editor: nano\n") + retarget := func() { + require.NoError(t, os.Remove(alias)) + require.NoError(t, os.Symlink(two, alias)) + } + if stage == "before writeback" { + retarget() + } else { + s.checkpoint = func(at string) error { + if at == stage { + retarget() + } + return nil + } + } + require.Error(t, s.finish(context.Background())) + assertNoForwardedArtifacts(t, s.dir) + for _, dir := range []string{one, two} { + data, err := readFile(filepath.Join(dir, configFile)) + require.NoError(t, err) + require.Equal(t, "editor: vi\n", string(data)) + } + require.NoError(t, os.Remove(alias)) + require.NoError(t, os.Symlink(one, alias)) + err = recoverSession(context.Background(), s.dir) + if err != nil { + require.ErrorIs(t, err, errAuthConflict) + } + assertNoForwardedArtifacts(t, filepath.Dir(s.dir)) + }) + } +} + func TestStripForwardedCredentialsAndPreserveLocalLogin(t *testing.T) { a := Account{Host: testHost, User: testUser, Token: testForwardedToken} s := &session{Accounts: []Account{a}} diff --git a/pkg/ide/opener/opener.go b/pkg/ide/opener/opener.go index 23a573f9b6..38a5966e1b 100644 --- a/pkg/ide/opener/opener.go +++ b/pkg/ide/opener/opener.go @@ -13,6 +13,7 @@ import ( "github.com/devsy-org/devsy/pkg/client/clientimplementation" "github.com/devsy-org/devsy/pkg/command" "github.com/devsy-org/devsy/pkg/config" + "github.com/devsy-org/devsy/pkg/credentials" config2 "github.com/devsy-org/devsy/pkg/devcontainer/config" "github.com/devsy-org/devsy/pkg/flags/names" "github.com/devsy-org/devsy/pkg/ide/codeserver" @@ -285,7 +286,7 @@ func makeDaemonStartFunc( } defer func() { _ = toolClient.Close() }() - status := clientimplementation.StartPreparedServicesDaemon( + return clientimplementation.RunPreparedServicesDaemon( ctx, clientimplementation.StartServicesDaemonOptions{ DevsyConfig: params.DevsyConfig, @@ -296,12 +297,11 @@ func makeDaemonStartFunc( ExtraPorts: extraPorts, GitSSHSigningKey: params.GitSSHSigningKey, }, + func(status credentials.Readiness) { + pkglog.Debugw("browser credentials prepared", "status", status.GitHubState) + prepared() + }, ) - pkglog.Debugw("browser credentials prepared", "status", status.GitHubState) - prepared() - <-ctx.Done() - - return nil } } diff --git a/pkg/tunnel/browser.go b/pkg/tunnel/browser.go index f1b0b1101d..2337aa6848 100644 --- a/pkg/tunnel/browser.go +++ b/pkg/tunnel/browser.go @@ -97,8 +97,8 @@ func runBrowserTunnelServices( p BrowserTunnelParams, containerClient *ssh.Client, ) error { - return runPreparedBrowserServices(ctx, p, func() credentials.Readiness { - return StartPreparedServices( + return runPreparedBrowserServices(ctx, p, func(prepared func(credentials.Readiness)) error { + return RunPreparedServices( ctx, RunServicesOptions{ DevsyConfig: p.DevsyConfig, @@ -123,6 +123,7 @@ func runBrowserTunnelServices( ExtraListeners: p.ExtraListeners, DisableIdleTimeout: p.DisableIdleTimeout, }, + prepared, ) }) } @@ -130,13 +131,12 @@ func runBrowserTunnelServices( func runPreparedBrowserServices( ctx context.Context, p BrowserTunnelParams, - prepare func() credentials.Readiness, + run func(func(credentials.Readiness)) error, ) error { - status := prepare() - log.Debugw("browser credentials prepared", "status", status.GitHubState) - notifyBrowserPrepared(ctx, p) - <-ctx.Done() - return nil + return run(func(status credentials.Readiness) { + log.Debugw("browser credentials prepared", "status", status.GitHubState) + notifyBrowserPrepared(ctx, p) + }) } func notifyBrowserPrepared(ctx context.Context, p BrowserTunnelParams) { diff --git a/pkg/tunnel/browser_preparation_test.go b/pkg/tunnel/browser_preparation_test.go index 92f76ba1e4..00fff2af3e 100644 --- a/pkg/tunnel/browser_preparation_test.go +++ b/pkg/tunnel/browser_preparation_test.go @@ -3,10 +3,12 @@ package tunnel import ( "context" "errors" + "io" "testing" "time" "github.com/devsy-org/devsy/pkg/credentials" + "github.com/devsy-org/devsy/pkg/transport" "github.com/stretchr/testify/require" ) @@ -34,7 +36,7 @@ func TestBrowserAutoOpenWaitsForInstallationAndServing(t *testing.T) { go func() { exited <- runPreparedBrowserServices(ctx, BrowserTunnelParams{ OnPrepared: func(context.Context) { close(opened) }, - }, func() credentials.Readiness { return runner.start(ctx, true) }) + }, func(prepared func(credentials.Readiness)) error { return runner.runPrepared(ctx, true, prepared) }) }() require.True(t, waitPreparationSignal(ctx, probeStarted)) assertPreparationPending(t, opened) @@ -92,12 +94,169 @@ func TestBrowserAutoOpenDoesNotRunAfterFailureOrCancellation(t *testing.T) { ctx, cancel := context.WithCancel(context.Background()) cancel() p := BrowserTunnelParams{OnPrepared: func(context.Context) { opened = true }} - require.NoError(t, runPreparedBrowserServices(ctx, p, func() credentials.Readiness { - return credentials.Readiness{GitHubState: credentialsUnavailable} - })) + require.NoError( + t, + runPreparedBrowserServices(ctx, p, func(prepared func(credentials.Readiness)) error { + prepared(credentials.Readiness{GitHubState: credentialsUnavailable}) + return nil + }), + ) require.False(t, opened) want := errors.New("daemon transport unavailable") p.DaemonStartFunc = func(context.Context, func()) error { return want } require.ErrorIs(t, StartBrowserTunnel(context.Background(), p), want) require.False(t, opened) } + +func TestBrowserServiceCompletionEndsHealthyTunnel(t *testing.T) { + for _, tc := range []struct { + name string + enabled, startup bool + }{ + {name: "startup failure", enabled: true, startup: true}, + {name: "failure after readiness", enabled: true}, + {name: "failure with forwarding disabled"}, + } { + t.Run(tc.name, func(t *testing.T) { + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + finish, opened := make(chan struct{}), make(chan struct{}, 1) + runner := browserCompletionRunner(ctx, finish, tc.startup) + if tc.startup { + close(finish) + } + exited := make(chan error, 1) + go func() { + exited <- runPreparedBrowserServices(ctx, BrowserTunnelParams{ + OnPrepared: func(context.Context) { opened <- struct{}{} }, + }, func(prepared func(credentials.Readiness)) error { return runner.runPrepared(ctx, tc.enabled, prepared) }) + }() + if !tc.startup { + require.True(t, waitPreparationSignal(ctx, opened)) + assertPreparationPending(t, exited) + close(finish) + } + select { + case err := <-exited: + require.Error(t, err) + require.NotContains(t, err.Error(), "dummy-secret") + case <-time.After(time.Second): + t.Fatal("completed services left the browser tunnel alive") + } + require.NoError(t, ctx.Err()) + if tc.startup { + assertPreparationPending(t, opened) + } + }) + } +} + +func TestBrowserPreservesCleanServiceReuseLifetime(t *testing.T) { + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + completed, opened := make(chan struct{}), make(chan struct{}) + runner := servicePreparation{ + run: func() error { close(completed); return nil }, + probe: func(context.Context) credentials.Readiness { + return credentials.Readiness{GitHubState: installedPreparation} + }, + } + exited := make(chan error, 1) + go func() { + exited <- runPreparedBrowserServices(ctx, BrowserTunnelParams{ + OnPrepared: func(context.Context) { close(opened) }, + }, func(prepared func(credentials.Readiness)) error { return runner.runPrepared(ctx, true, prepared) }) + }() + require.True(t, waitPreparationSignal(ctx, completed)) + require.True(t, waitPreparationSignal(ctx, opened)) + select { + case <-exited: + t.Fatal("benign service reuse ended browser port forwarding") + case <-time.After(100 * time.Millisecond): + } + cancel() + select { + case err := <-exited: + require.NoError(t, err) + case <-time.After(time.Second): + t.Fatal("reused browser service lifetime did not stop") + } +} + +func browserCompletionRunner( + ctx context.Context, + finish <-chan struct{}, + startup bool, +) servicePreparation { + return servicePreparation{ + run: func() error { + select { + case <-finish: + return errors.New("dummy-secret-service-output") + case <-ctx.Done(): + return ctx.Err() + } + }, + probe: func(ctx context.Context) credentials.Readiness { + if startup { + <-ctx.Done() + return credentials.Readiness{GitHubState: credentialsUnavailable} + } + return credentials.Readiness{GitHubState: installedPreparation} + }, + } +} + +func TestBrowserServiceFailureClosesManagedTransport(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + transportStopped := make(chan struct{}) + conn, err := transport.OpenCallbackConn( + ctx, + func(ctx context.Context, _ io.Reader, _ io.Writer) error { + defer close(transportStopped) + <-ctx.Done() + return ctx.Err() + }, + transport.CallbackConnOptions{}, + ) + require.NoError(t, err) + defer func() { _ = conn.Close() }() + opened, fail := make(chan struct{}), make(chan struct{}) + exited := make(chan error, 1) + go func() { + exited <- transport.RunManaged(transport.RunManagedOptions{ + Parent: ctx, + Conn: conn, + TransportSide: transport.SideProvider, + Handler: func(ctx context.Context) error { + runner := servicePreparation{ + run: func() error { + select { + case <-fail: + return errors.New("dummy-secret-service-output") + case <-ctx.Done(): + return ctx.Err() + } + }, + probe: func(context.Context) credentials.Readiness { + return credentials.Readiness{GitHubState: installedPreparation} + }, + } + return runPreparedBrowserServices(ctx, BrowserTunnelParams{ + OnPrepared: func(context.Context) { close(opened) }, + }, func(prepared func(credentials.Readiness)) error { return runner.runPrepared(ctx, true, prepared) }) + }, + }) + }() + require.True(t, waitPreparationSignal(ctx, opened)) + close(fail) + select { + case err := <-exited: + require.EqualError(t, err, "credential services stopped unexpectedly") + case <-ctx.Done(): + t.Fatal("failed services did not tear down the managed connection") + } + require.NoError(t, ctx.Err()) + require.True(t, waitPreparationSignal(ctx, transportStopped)) +} diff --git a/pkg/tunnel/services_preparation.go b/pkg/tunnel/services_preparation.go index 11de393433..b9a8869fa2 100644 --- a/pkg/tunnel/services_preparation.go +++ b/pkg/tunnel/services_preparation.go @@ -5,6 +5,7 @@ import ( "crypto/sha256" "encoding/hex" "encoding/json" + "errors" "io" "os" "time" @@ -24,12 +25,23 @@ func hostCredentialOwner() string { } func StartPreparedServices(ctx context.Context, opts RunServicesOptions) credentials.Readiness { + return prepareServices(ctx, opts).start(ctx, opts.ConfigureGitHubCredentials) +} + +func RunPreparedServices( + ctx context.Context, + opts RunServicesOptions, + prepared func(credentials.Readiness), +) error { + return prepareServices(ctx, opts).runPrepared(ctx, opts.ConfigureGitHubCredentials, prepared) +} + +func prepareServices(ctx context.Context, opts RunServicesOptions) servicePreparation { opts.HostOwner = hostCredentialOwner() - runner := servicePreparation{ + return servicePreparation{ run: func() error { return RunServices(ctx, opts) }, probe: func(ctx context.Context) credentials.Readiness { return probePreparedServices(ctx, opts) }, } - return runner.start(ctx, opts.ConfigureGitHubCredentials) } type servicePreparation struct { @@ -38,35 +50,83 @@ type servicePreparation struct { } func (p servicePreparation) start(ctx context.Context, enabled bool) credentials.Readiness { - done := make(chan error, 1) + status, _ := p.startWithLifetime(ctx, enabled) + return status +} + +type serviceCompletion struct { + done chan struct{} + err error +} + +func (p servicePreparation) startWithLifetime( + ctx context.Context, + enabled bool, +) (credentials.Readiness, *serviceCompletion) { + completion := &serviceCompletion{done: make(chan struct{})} go func() { - err := p.run() - if err != nil && ctx.Err() == nil { + completion.err = p.run() + if completion.err != nil && ctx.Err() == nil { log.Debug("credential services stopped unexpectedly") } - done <- err + close(completion.done) }() if !enabled { - return credentials.Readiness{GitHubState: "disabled"} + return credentials.Readiness{GitHubState: "disabled"}, completion } probeCtx, cancel := context.WithTimeout(ctx, 12*time.Second) defer cancel() ready := make(chan credentials.Readiness, 1) go func() { ready <- p.probe(probeCtx) }() - return p.await(probeCtx, done, ready) + return p.await(probeCtx, completion, ready), completion +} + +func (p servicePreparation) runPrepared( + ctx context.Context, + enabled bool, + prepared func(credentials.Readiness), +) error { + status, completion := p.startWithLifetime(ctx, enabled) + select { + case <-completion.done: + if completion.err != nil { + return completion.result(ctx) + } + default: + } + if ctx.Err() == nil { + prepared(status) + } + select { + case <-completion.done: + if err := completion.result(ctx); err != nil { + return err + } + <-ctx.Done() + case <-ctx.Done(): + } + return nil +} + +func (c *serviceCompletion) result(ctx context.Context) error { + if c.err != nil && ctx.Err() == nil { + return errors.New("credential services stopped unexpectedly") + } + return nil } func (p servicePreparation) await( ctx context.Context, - done <-chan error, + completion *serviceCompletion, ready <-chan credentials.Readiness, ) credentials.Readiness { + done := completion.done for { select { case status := <-ready: return status - case err := <-done: - if err != nil { + case <-done: + if completion.err != nil { return credentials.Readiness{GitHubState: credentialsUnavailable} } done = nil From ef8b6cf202c0c03433d33dad6165b539bdc0a208 Mon Sep 17 00:00:00 2001 From: Samuel K Date: Tue, 20 Oct 2026 15:35:27 -0600 Subject: [PATCH 17/20] fix(ci): use runner scratch for Go cross-builds Ubuntu 26.04 mounts /tmp with a per-user tmpfs quota. Both Linux build attempts exhausted it while compiling ARM64 dependencies. Use the runner temp directory for Go scratch. Refs: actions/runner-images#14777 --- .github/workflows/pr-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/pr-ci.yml b/.github/workflows/pr-ci.yml index 9ed4398a69..367c172638 100644 --- a/.github/workflows/pr-ci.yml +++ b/.github/workflows/pr-ci.yml @@ -307,6 +307,8 @@ jobs: args: build --id devsy-${{ steps.os.outputs.runner_os }} --snapshot env: DEVSY_CLI_VERSION: v0.0.0 + # Ubuntu 26.04 quotas /tmp: https://github.com/actions/runner-images/issues/14777 + GOTMPDIR: ${{ runner.temp }} - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 with: From 70dc59b4d906722160160be93dab2005e3eb4b96 Mon Sep 17 00:00:00 2001 From: Samuel K Date: Tue, 20 Oct 2026 20:50:12 -0600 Subject: [PATCH 18/20] fix(credentials): tolerate guest profiles and stale links Keep target-user backend discovery working when login profiles write stdout. Skip missing snapshot targets without relaxing publication checks. --- pkg/ghcredentials/install.go | 12 ++++- pkg/ghcredentials/install_test.go | 84 +++++++++++++++++++++++++++++++ pkg/ghcredentials/session.go | 7 +++ pkg/ghcredentials/session_test.go | 49 ++++++++++++++++++ 4 files changed, 150 insertions(+), 2 deletions(-) diff --git a/pkg/ghcredentials/install.go b/pkg/ghcredentials/install.go index 8ba0247b7a..421a39e55c 100644 --- a/pkg/ghcredentials/install.go +++ b/pkg/ghcredentials/install.go @@ -1,6 +1,7 @@ package ghcredentials import ( + "bytes" "context" "encoding/json" "errors" @@ -100,6 +101,11 @@ func discoverForUser(ctx context.Context, binary, targetUser string) (InstallRes if os.Geteuid() != 0 { return InstallResult{}, errors.New("GitHub CLI target user differs from service user") } + return discoverWithLoginShell(ctx, binary, targetUser) +} + +func discoverWithLoginShell(ctx context.Context, binary, targetUser string) (InstallResult, error) { + // Separate the response even when the login profile prints without a newline. //nolint:gosec // su drops to the requested guest user before reading their login environment. cmd := exec.CommandContext( ctx, @@ -107,7 +113,7 @@ func discoverForUser(ctx context.Context, binary, targetUser string) (InstallRes "-", targetUser, "-c", - shellescape.Quote(binary)+" internal gh-backend", + "printf '\\n'; exec "+shellescape.Quote(binary)+" internal gh-backend", ) cmd.WaitDelay = 250 * time.Millisecond out := &tokenOutput{} @@ -117,8 +123,10 @@ func discoverForUser(ctx context.Context, binary, targetUser string) (InstallRes } return InstallResult{}, err } + response := bytes.TrimSpace(out.data) + response = response[bytes.LastIndexByte(response, '\n')+1:] var result InstallResult - if out.overflow || json.Unmarshal(out.data, &result) != nil { + if out.overflow || json.Unmarshal(response, &result) != nil { return InstallResult{}, errInvalidConfig } return result, nil diff --git a/pkg/ghcredentials/install_test.go b/pkg/ghcredentials/install_test.go index f55a10f74b..86a468ae24 100644 --- a/pkg/ghcredentials/install_test.go +++ b/pkg/ghcredentials/install_test.go @@ -7,6 +7,7 @@ import ( "os" "os/exec" "path/filepath" + "strings" "sync" "testing" "time" @@ -14,6 +15,89 @@ import ( "github.com/stretchr/testify/require" ) +func TestBackendDiscoveryToleratesLoginShellOutput(t *testing.T) { + const availableResult = `{"status":"available"}` + const missingResult = `{"status":"backend_missing"}` + for _, tc := range []struct { + name, profile, output, exit, want string + invalid bool + }{ + {name: "plain response", output: availableResult, want: stateAvailable}, + {name: "login banner", profile: "Welcome to the workspace\n", output: availableResult, want: stateAvailable}, + { + name: "banner without newline", profile: "Activating user environment", + output: availableResult, want: stateAvailable, + }, + { + name: "backend missing", profile: "User profile loaded\n", + output: missingResult + "\n\n", want: "backend_missing", + }, + {name: "JSON in profile", profile: availableResult + "\n", output: missingResult, want: "backend_missing"}, + {name: "malformed response", profile: "Welcome\n", output: "not JSON", invalid: true}, + {name: "empty response", profile: "Welcome\n", invalid: true}, + { + name: "malformed response after JSON profile", profile: availableResult + "\n", + output: "invalid", invalid: true, + }, + {name: "failed command", output: availableResult, exit: "17", invalid: true}, + { + name: "oversized banner", profile: strings.Repeat("x", maxTokenOutput+1), + output: availableResult, invalid: true, + }, + } { + t.Run(tc.name, func(t *testing.T) { + binary, record := loginShellFixture(t) + t.Setenv("DEVSY_TEST_PROFILE_OUTPUT", tc.profile) + t.Setenv("DEVSY_TEST_BACKEND_OUTPUT", tc.output) + t.Setenv("DEVSY_TEST_BACKEND_EXIT", tc.exit) + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + result, err := discoverWithLoginShell(ctx, binary, "guest-user") + if tc.invalid { + require.Error(t, err) + } else { + require.NoError(t, err) + require.Equal(t, tc.want, result.Status) + } + args, err := readHelperFile(record) + require.NoError(t, err) + require.True(t, strings.HasPrefix(string(args), "-\nguest-user\n-c\n")) + }) + } +} + +func loginShellFixture(t *testing.T) (binary, record string) { + t.Helper() + dir := t.TempDir() + record = filepath.Join(dir, "su-args") + writeTestExecutable(t, filepath.Join(dir, "su"), []byte(`#!/bin/sh +printf '%s\n' "$@" > "$DEVSY_TEST_SU_ARGS" +printf '%s' "$DEVSY_TEST_PROFILE_OUTPUT" +exec /bin/sh -c "$4" +`)) + binary = filepath.Join(dir, "agent's binary") + writeTestExecutable(t, binary, []byte(`#!/bin/sh +test "$#" = 2 && test "$1" = internal && test "$2" = gh-backend || exit 42 +printf '%s\n' "$DEVSY_TEST_BACKEND_OUTPUT" +exit "${DEVSY_TEST_BACKEND_EXIT:-0}" +`)) + t.Setenv("PATH", dir) + t.Setenv("DEVSY_TEST_SU_ARGS", record) + return binary, record +} + +func TestBackendDiscoveryLoginShellCancellationIsBounded(t *testing.T) { + dir := t.TempDir() + writeTestExecutable(t, filepath.Join(dir, "su"), []byte("#!/bin/sh\nexec /bin/sleep 10\n")) + t.Setenv("PATH", dir) + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + started := time.Now() + _, err := discoverWithLoginShell(ctx, "/unused-agent", "guest-user") + require.ErrorIs(t, err, context.DeadlineExceeded) + require.Less(t, time.Since(started), time.Second) +} + func TestInstallWithSudoPreservesArgumentsAndFailure(t *testing.T) { dir := t.TempDir() record := filepath.Join(dir, "args") diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index 0f81c5e361..3224bd747a 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -163,6 +163,9 @@ func (s *treeSnapshot) copyEntry(src, dst string, depth int) error { return errors.New("GitHub CLI configuration has too many entries") } info, err := os.Stat(src) // Follow symlinks into a private copy, never a link to the original. + if errors.Is(err, os.ErrNotExist) { + return nil + } if err != nil { return err } @@ -172,6 +175,10 @@ func (s *treeSnapshot) copyEntry(src, dst string, depth int) error { if !info.Mode().IsRegular() { return nil } + return s.copyFile(src, dst, info) +} + +func (s *treeSnapshot) copyFile(src, dst string, info fs.FileInfo) error { if err := s.reserveBytes(info.Size()); err != nil { return err } diff --git a/pkg/ghcredentials/session_test.go b/pkg/ghcredentials/session_test.go index 87c670f6a7..dfe2d8d466 100644 --- a/pkg/ghcredentials/session_test.go +++ b/pkg/ghcredentials/session_test.go @@ -3,6 +3,7 @@ package ghcredentials import ( + "bytes" "context" "os" "path/filepath" @@ -38,6 +39,54 @@ func TestSessionMergePreservesDotfilesAndConcurrentChanges(t *testing.T) { require.Equal(t, "disabled", get(n, "prompt").Value) } +func TestSessionSnapshotIgnoresDanglingSymlinks(t *testing.T) { + root := t.TempDir() + t.Setenv("HOME", root) + t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) + original := filepath.Join(root, "gh") + writeTestFile(t, filepath.Join(original, configFile), "editor: vi\n") + for _, name := range []string{"unused.yml", filepath.Join("extensions", "unused")} { + path := filepath.Join(original, name) + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o700)) + require.NoError(t, os.Symlink("missing-target", path)) + } + s, err := startSession(context.Background(), original, []Account{ + {Host: testHost, User: testUser, Token: testForwardedToken}, + }) + require.NoError(t, err) + t.Cleanup(func() { _ = s.lock.Close() }) + env := []string{"HOME=" + root, "GH_CONFIG_DIR=" + original, testPathEnv} + require.NoError(t, s.inject(context.Background(), "/usr/bin/false", env)) + data, err := readFile(filepath.Join(s.work(), hostsFile)) + require.NoError(t, err) + require.True(t, bytes.Contains(data, []byte(testForwardedToken)), "forwarding was not prepared") + writeTestFile(t, filepath.Join(s.work(), configFile), "editor: nano\n") + require.NoError(t, s.finish(context.Background())) + data, err = readFile(filepath.Join(original, configFile)) + require.NoError(t, err) + require.Equal(t, "editor: nano\n", string(data)) + for _, name := range []string{"unused.yml", filepath.Join("extensions", "unused")} { + target, err := os.Readlink(filepath.Join(original, name)) + require.NoError(t, err) + require.Equal(t, "missing-target", target) + } + data, err = readFile(filepath.Join(original, hostsFile)) + require.NoError(t, err) + require.False( + t, + bytes.Contains(data, []byte(testForwardedToken)), + "forwarded credentials were published", + ) + assertNoForwardedArtifacts(t, filepath.Join(root, "cache")) +} + +func TestSessionSnapshotStillRejectsSymlinkCycles(t *testing.T) { + root := t.TempDir() + link := filepath.Join(root, "cycle") + require.NoError(t, os.Symlink("cycle", link)) + require.Error(t, copyTree(context.Background(), root, t.TempDir(), 0)) +} + func TestSessionConcurrentInvocationsAndRecovery(t *testing.T) { root := t.TempDir() t.Setenv("XDG_CACHE_HOME", filepath.Join(root, "cache")) From 6887c98c355a4928384ac6a1add78cf68ec1378b Mon Sep 17 00:00:00 2001 From: Samuel K Date: Tue, 20 Oct 2026 22:41:31 -0600 Subject: [PATCH 19/20] fix(credentials): preserve forwarding service boundaries --- .github/workflows/pr-ci.yml | 4 + e2e/framework/ghfixture/cases.go | 4 + e2e/framework/ghfixture/checks_linux.go | 2 +- e2e/framework/ghfixture/host_guard_linux.go | 44 +++++++ pkg/client/clientimplementation/services.go | 7 +- pkg/credentials/github_test.go | 18 ++- pkg/credentials/host_test.go | 73 +++++++++++ pkg/credentials/server.go | 16 +++ pkg/credentials/server_test.go | 2 +- pkg/ide/opener/browser_preparation_test.go | 52 ++++++++ pkg/ide/opener/browser_tunnel.go | 27 ++-- .../opener/browser_tunnel_state_unix_test.go | 13 +- pkg/tunnel/services.go | 38 +++++- pkg/tunnel/services_preparation.go | 2 +- pkg/tunnel/services_preparation_test.go | 2 +- pkg/tunnel/services_ssh_test.go | 117 ++++++++++++++++++ pkg/tunnel/services_startup_test.go | 41 ++++++ pkg/tunnel/services_test.go | 37 +++++- 18 files changed, 462 insertions(+), 37 deletions(-) create mode 100644 e2e/framework/ghfixture/host_guard_linux.go create mode 100644 pkg/credentials/host_test.go create mode 100644 pkg/tunnel/services_ssh_test.go create mode 100644 pkg/tunnel/services_startup_test.go diff --git a/.github/workflows/pr-ci.yml b/.github/workflows/pr-ci.yml index 367c172638..3419603930 100644 --- a/.github/workflows/pr-ci.yml +++ b/.github/workflows/pr-ci.yml @@ -174,6 +174,10 @@ jobs: -run '^(TestNative.*|TestHostLookupEnvironmentAndGuestEnvironmentAreIndependent|TestRegisteredGuestDeclinesAmbiguousKeyringResult)$' - name: test strict fixture report validation run: go test -race -count=1 -v ./e2e/framework/ghfixture + - name: test native credential service boundaries + run: >- + go test -race -count=1 -v -timeout 2m ./pkg/tunnel ./pkg/credentials + -run '^(TestCredentials.*Host.*|TestOptionalGitHubService.*|TestServicesStartup.*)$' windows-transport-tests: name: Windows Transport Tests diff --git a/e2e/framework/ghfixture/cases.go b/e2e/framework/ghfixture/cases.go index b91088ec02..6fde115739 100644 --- a/e2e/framework/ghfixture/cases.go +++ b/e2e/framework/ghfixture/cases.go @@ -1,6 +1,10 @@ package ghfixture var ExpectedChecks = []string{ + "relay rejects non-loopback Host: /ready", + "relay rejects non-loopback Host: /owner", + "relay rejects non-loopback Host: /github-credentials", + "relay rejects non-loopback Host: /git-credentials", "malformed working hosts: alias prepared", "malformed working hosts: child exit preserved", "malformed working hosts: lost intent reported", diff --git a/e2e/framework/ghfixture/checks_linux.go b/e2e/framework/ghfixture/checks_linux.go index f95400a281..b9e5b95695 100644 --- a/e2e/framework/ghfixture/checks_linux.go +++ b/e2e/framework/ghfixture/checks_linux.go @@ -12,7 +12,7 @@ import ( func (c *checks) run() bool { for _, phase := range []func() bool{ - c.checkInstallation, c.checkGuestRegistrations, c.checkDispatcher, c.checkEnvfile, c.checkDispatch, + c.checkInstallation, c.checkRelayHosts, c.checkGuestRegistrations, c.checkDispatcher, c.checkEnvfile, c.checkDispatch, c.checkSelection, c.checkSelfReferences, c.checkPreferences, c.checkAuthentication, c.checkLocalAuthentication, c.checkKeyring, c.checkExtensions, c.checkConcurrentUpdates, c.checkWriteback, c.checkMalformedWorkingHosts, c.checkOrphanChildren, diff --git a/e2e/framework/ghfixture/host_guard_linux.go b/e2e/framework/ghfixture/host_guard_linux.go new file mode 100644 index 0000000000..08f7151bb3 --- /dev/null +++ b/e2e/framework/ghfixture/host_guard_linux.go @@ -0,0 +1,44 @@ +//go:build linux + +package ghfixture + +import ( + "context" + "net/http" + "time" +) + +func (c *checks) checkRelayHosts() bool { + client := &http.Client{ + Transport: &http.Transport{Proxy: nil}, + Timeout: 2 * time.Second, + CheckRedirect: func(*http.Request, []*http.Request) error { + return http.ErrUseLastResponse + }, + } + defer client.CloseIdleConnections() + for _, path := range []string{"/ready", "/owner", "/github-credentials", "/git-credentials"} { + ctx, cancel := context.WithTimeout(c.ctx, 2*time.Second) + req, err := http.NewRequestWithContext( + ctx, + http.MethodGet, + "http://127.0.0.1:12049"+path, + nil, + ) + if err != nil { + cancel() + return false + } + req.Host = "attacker.invalid:12049" + response, err := client.Do(req) + ok := err == nil && response.StatusCode == http.StatusForbidden + if response != nil { + _ = response.Body.Close() + } + cancel() + if !c.verify("relay rejects non-loopback Host: "+path, ok) { + return false + } + } + return true +} diff --git a/pkg/client/clientimplementation/services.go b/pkg/client/clientimplementation/services.go index 7a1ddc863a..41082d2b93 100644 --- a/pkg/client/clientimplementation/services.go +++ b/pkg/client/clientimplementation/services.go @@ -8,6 +8,7 @@ import ( "github.com/devsy-org/devsy/pkg/config" "github.com/devsy-org/devsy/pkg/credentials" daemon "github.com/devsy-org/devsy/pkg/daemon/platform" + "github.com/devsy-org/devsy/pkg/log" "github.com/devsy-org/devsy/pkg/tunnel" "golang.org/x/crypto/ssh" ) @@ -46,7 +47,11 @@ func StartPreparedServicesDaemon( opts StartServicesDaemonOptions, ) credentials.Readiness { services, err := daemonServiceOptions(ctx, opts) - if err != nil || opts.User == "" { + if err != nil { + log.Warn("credential services preparation unavailable") + return credentials.Readiness{GitHubState: "unavailable"} + } + if opts.User == "" { return credentials.Readiness{GitHubState: "unavailable"} } return tunnel.StartPreparedServices(ctx, services) diff --git a/pkg/credentials/github_test.go b/pkg/credentials/github_test.go index 6f14290f0a..06178012f7 100644 --- a/pkg/credentials/github_test.go +++ b/pkg/credentials/github_test.go @@ -36,7 +36,7 @@ func TestReadinessRejectsDeadTransportWithoutCredentialLookup(t *testing.T) { client := &githubCredentialsClient{code: codes.Unavailable} h := newPreparedHandler(context.Background(), client, "vscode", readyFixture()) w := httptest.NewRecorder() - h.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/ready", nil)) + h.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "http://localhost/ready", nil)) require.Equal(t, http.StatusServiceUnavailable, w.Code) require.Zero(t, client.calls.Load()) } @@ -46,7 +46,7 @@ func TestCredentialPostPinsVerifiedHostAndInstanceBeforeRPC(t *testing.T) { identity := readyFixture() h := newPreparedHandler(context.Background(), client, identity.Owner, identity) for _, pair := range [][2]string{{"", ""}, {"old-instance", identity.HostOwner}, {identity.Instance, "other-host"}} { - r := httptest.NewRequest(http.MethodPost, "/github-credentials", nil) + r := httptest.NewRequest(http.MethodPost, "http://localhost/github-credentials", nil) r.Header.Set(HeaderCredentialsInstance, pair[0]) r.Header.Set(HeaderCredentialsHostOwner, pair[1]) w := httptest.NewRecorder() @@ -56,7 +56,7 @@ func TestCredentialPostPinsVerifiedHostAndInstanceBeforeRPC(t *testing.T) { require.Equal(t, "no-store", w.Header().Get("Cache-Control")) } require.Zero(t, client.calls.Load()) - r := httptest.NewRequest(http.MethodPost, "/github-credentials", nil) + r := httptest.NewRequest(http.MethodPost, "http://localhost/github-credentials", nil) r.Header.Set(HeaderCredentialsInstance, identity.Instance) r.Header.Set(HeaderCredentialsHostOwner, identity.HostOwner) w := httptest.NewRecorder() @@ -93,11 +93,17 @@ func TestGitHubHTTPBoundaryMethodsStatusesAndHeaders(t *testing.T) { client := &githubCredentialsClient{code: tc.code} h := newCredentialsHandler(context.Background(), client, "guest") get := httptest.NewRecorder() - h.ServeHTTP(get, httptest.NewRequest(http.MethodGet, "/github-credentials", nil)) + h.ServeHTTP( + get, + httptest.NewRequest(http.MethodGet, "http://localhost/github-credentials", nil), + ) require.Equal(t, http.StatusMethodNotAllowed, get.Code) require.Zero(t, client.calls.Load()) post := httptest.NewRecorder() - h.ServeHTTP(post, httptest.NewRequest(http.MethodPost, "/github-credentials", nil)) + h.ServeHTTP( + post, + httptest.NewRequest(http.MethodPost, "http://localhost/github-credentials", nil), + ) require.Equal(t, http.StatusOK, post.Code) require.Equal(t, "application/json", post.Header().Get("Content-Type")) require.Equal(t, "no-store", post.Header().Get("Cache-Control")) @@ -112,7 +118,7 @@ func TestGitHubHTTPBoundaryMethodsStatusesAndHeaders(t *testing.T) { func TestGitHubHTTPMissingOptionalMethod(t *testing.T) { h := newCredentialsHandler(context.Background(), &mockCredentialsClient{}, "guest") w := httptest.NewRecorder() - h.ServeHTTP(w, httptest.NewRequest(http.MethodPost, "/github-credentials", nil)) + h.ServeHTTP(w, httptest.NewRequest(http.MethodPost, "http://localhost/github-credentials", nil)) require.JSONEq(t, `{"status":"rpc_unavailable"}`, w.Body.String()) require.Equal(t, "no-store", w.Header().Get("Cache-Control")) } diff --git a/pkg/credentials/host_test.go b/pkg/credentials/host_test.go new file mode 100644 index 0000000000..dd3ab4609d --- /dev/null +++ b/pkg/credentials/host_test.go @@ -0,0 +1,73 @@ +package credentials + +import ( + "context" + "net/http" + "net/http/httptest" + "sync/atomic" + "testing" + + "github.com/devsy-org/devsy/pkg/agent/tunnel" + "github.com/stretchr/testify/require" + "google.golang.org/grpc" +) + +type hostGuardClient struct { + githubCredentialsClient + pings atomic.Int32 +} + +func (c *hostGuardClient) Ping( + context.Context, + *tunnel.Empty, + ...grpc.CallOption, +) (*tunnel.Empty, error) { + c.pings.Add(1) + return &tunnel.Empty{}, nil +} + +func TestCredentialsRejectNonLoopbackHostBeforeDispatch(t *testing.T) { + for _, host := range []string{ + "attacker.invalid:12049", "localhost.attacker", "127.0.0.1.attacker", + "192.0.2.1", "[2001:db8::1]:12049", "0.0.0.0", "", + } { + t.Run(host, func(t *testing.T) { + client := &hostGuardClient{} + identity := readyFixture() + handler := newPreparedHandler(t.Context(), client, identity.Owner, identity) + for _, path := range []string{"/ready", ownerPath, "/github-credentials", "/git-credentials"} { + request := httptest.NewRequest(http.MethodPost, "http://localhost"+path, nil) + if path == "/ready" || path == ownerPath { + request.Method = http.MethodGet + } + request.Host = host + request.Header.Set(HeaderCredentialsInstance, identity.Instance) + request.Header.Set(HeaderCredentialsHostOwner, identity.HostOwner) + response := httptest.NewRecorder() + handler.ServeHTTP(response, request) + require.Equal(t, http.StatusForbidden, response.Code, path) + } + require.Zero(t, client.pings.Load()) + require.Zero(t, client.calls.Load()) + }) + } +} + +func TestCredentialsAcceptLoopbackHosts(t *testing.T) { + for _, host := range []string{ + "localhost", "LOCALHOST:12049", "127.0.0.1", "127.0.0.1:12049", + "127.0.0.2:12049", "[::1]", "[::1]:12049", + } { + t.Run(host, func(t *testing.T) { + client := &hostGuardClient{} + handler := newPreparedHandler(t.Context(), client, "guest", readyFixture()) + request := httptest.NewRequest(http.MethodGet, "http://localhost/ready", nil) + request.Host = host + response := httptest.NewRecorder() + handler.ServeHTTP(response, request) + require.Equal(t, http.StatusOK, response.Code) + require.EqualValues(t, 1, client.pings.Load()) + require.Zero(t, client.calls.Load()) + }) + } +} diff --git a/pkg/credentials/server.go b/pkg/credentials/server.go index 9185e2153a..da70f61eff 100644 --- a/pkg/credentials/server.go +++ b/pkg/credentials/server.go @@ -176,6 +176,10 @@ func newPreparedHandler( } return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) { + if !loopbackHost(request.Host) { + http.Error(writer, "loopback Host required", http.StatusForbidden) + return + } log.Debugf("incoming client connection: path=%s", request.URL.Path) handler, ok := routes[request.URL.Path] if !ok { @@ -188,6 +192,18 @@ func newPreparedHandler( }) } +func loopbackHost(authority string) bool { + host, _, err := net.SplitHostPort(authority) + if err != nil { + host = authority + } + if strings.EqualFold(host, "localhost") { + return true + } + ip := net.ParseIP(strings.TrimPrefix(strings.TrimSuffix(host, "]"), "[")) + return ip != nil && ip.IsLoopback() +} + func pinnedGitHubHandler(status Readiness) credentialsHandlerFunc { return func(ctx context.Context, w http.ResponseWriter, r *http.Request, client CredentialsClient) error { if r.Method == http.MethodPost && status.validIdentity() && diff --git a/pkg/credentials/server_test.go b/pkg/credentials/server_test.go index 10d56dac4a..e45ac6ecab 100644 --- a/pkg/credentials/server_test.go +++ b/pkg/credentials/server_test.go @@ -107,7 +107,7 @@ func TestOwnerEndpoint_ReturnsConfiguredOwner(t *testing.T) { mock := &mockCredentialsClient{} handler := newCredentialsHandler(context.Background(), mock, "alice") - req := httptest.NewRequest(http.MethodGet, ownerPath, nil) + req := httptest.NewRequest(http.MethodGet, "http://localhost"+ownerPath, nil) w := httptest.NewRecorder() handler.ServeHTTP(w, req) diff --git a/pkg/ide/opener/browser_preparation_test.go b/pkg/ide/opener/browser_preparation_test.go index e30ff16471..f656052c5a 100644 --- a/pkg/ide/opener/browser_preparation_test.go +++ b/pkg/ide/opener/browser_preparation_test.go @@ -6,7 +6,9 @@ import ( "testing" "time" + "github.com/devsy-org/devsy/pkg/log" "github.com/stretchr/testify/require" + "go.uber.org/zap/zapcore" ) func TestBrowserReuseWaitsForPreparedHelperIdentity(t *testing.T) { @@ -46,3 +48,53 @@ func TestBrowserReuseRejectsLegacyAndCanceledPreparation(t *testing.T) { require.NoError(t, marker.Mark()) require.False(t, waitBrowserPreparation(ctx, "ctx", "ws", &expected)) } + +func TestBrowserReuseReleasesLockWhilePreparationIsPending(t *testing.T) { + setupTempHome(t) + logs := log.InitTestObserved(t, zapcore.InfoLevel) + created, err := helperCreateTime(os.Getpid()) + require.NoError(t, err) + existing := TunnelState{PID: os.Getpid(), CreateTime: created, PreparationVersion: 1} + require.NoError(t, WriteTunnelState("ctx", "ws", existing)) + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + done := make(chan struct{}) + go func() { + defer close(done) + _, _ = runDaemonBrowserTunnel(ctx, daemonTunnelArgs{ + ContextName: "ctx", WorkspaceID: "ws", + Inv: browserIDEInvocation{OpenBrowser: true, RequirePreparation: true}, + }) + }() + require.Eventually(t, func() bool { + return logs.FilterMessageSnippet("browser tunnel already running").Len() == 1 + }, time.Second, time.Millisecond) + unlocked := make(chan error, 1) + go func() { + unlock, err := acquireTunnelLock("ctx", "ws") + if err == nil { + unlock() + } + unlocked <- err + }() + select { + case err := <-unlocked: + require.NoError(t, err) + case <-time.After(2 * time.Second): + cancel() + <-done + <-unlocked + t.Fatal("browser preparation held the shared tunnel lock") + } + select { + case <-done: + t.Fatal("browser reuse returned before pending preparation completed") + default: + } + cancel() + select { + case <-done: + case <-time.After(2 * time.Second): + t.Fatal("browser reuse did not observe cancellation") + } +} diff --git a/pkg/ide/opener/browser_tunnel.go b/pkg/ide/opener/browser_tunnel.go index e1d8cb3d73..17ffc69f24 100644 --- a/pkg/ide/opener/browser_tunnel.go +++ b/pkg/ide/opener/browser_tunnel.go @@ -9,6 +9,7 @@ import ( "os" "os/exec" "path/filepath" + "sync" "syscall" "time" @@ -210,11 +211,14 @@ func startDetachedBrowserTunnel( err, ) } + unlock = sync.OnceFunc(unlock) defer unlock() // Reuse an existing live tunnel if present; clear stale state otherwise. - if reusedURL, ok := tryReuseExistingTunnel(ctx, contextName, workspaceID, inv); ok { - return reusedURL, nil + if existing, ok := tryReuseExistingTunnel(contextName, workspaceID, inv); ok { + unlock() + inv.openReusedBrowser(ctx, contextName, workspaceID, existing) + return existing.TargetURL, nil } pid, logLocation, err := spawnTunnelHelper(spawnHelperOpts{ @@ -462,29 +466,27 @@ func openBrowserAsync(ctx context.Context, url string) { // workspace, if any. On reuse it returns the recorded TargetURL so the // caller can surface the live URL (not its own freshly-computed one). // If state exists but the recorded PID is dead, the stale state file is -// removed and ("", false) is returned so the caller can proceed with a +// removed and (nil, false) is returned so the caller can proceed with a // fresh spawn. func tryReuseExistingTunnel( - ctx context.Context, contextName, workspaceID string, inv browserIDEInvocation, -) (string, bool) { +) (*TunnelState, bool) { existing, _ := ReadTunnelState(contextName, workspaceID) if existing == nil { - return "", false + return nil, false } if helperMatchesState(existing) { pkglog.Infof( "%s browser tunnel already running (PID %d). Reusing existing tunnel at %s", inv.Label, existing.PID, existing.TargetURL, ) - inv.openReusedBrowser(ctx, contextName, workspaceID, existing) - return existing.TargetURL, true + return existing, true } if statePath, err := TunnelStateFilePath(contextName, workspaceID); err == nil { _ = os.Remove(statePath) } - return "", false + return nil, false } func (inv browserIDEInvocation) openReusedBrowser( @@ -526,10 +528,13 @@ func runDaemonBrowserTunnel(ctx context.Context, a daemonTunnelArgs) (string, er err, ) } + unlock = sync.OnceFunc(unlock) defer unlock() - if reusedURL, ok := tryReuseExistingTunnel(ctx, a.ContextName, a.WorkspaceID, a.Inv); ok { - return reusedURL, nil + if existing, ok := tryReuseExistingTunnel(a.ContextName, a.WorkspaceID, a.Inv); ok { + unlock() + a.Inv.openReusedBrowser(ctx, a.ContextName, a.WorkspaceID, existing) + return existing.TargetURL, nil } // Derive a cancellable context for the probe goroutine. When diff --git a/pkg/ide/opener/browser_tunnel_state_unix_test.go b/pkg/ide/opener/browser_tunnel_state_unix_test.go index 6cd374a64a..5b4902f7b9 100644 --- a/pkg/ide/opener/browser_tunnel_state_unix_test.go +++ b/pkg/ide/opener/browser_tunnel_state_unix_test.go @@ -228,17 +228,15 @@ func TestTryReuseExistingTunnel_ReturnsRecordedURL(t *testing.T) { t.Fatalf("WriteTunnelState: %v", err) } - // OpenBrowser=false so the test doesn't spawn an actual browser process. - gotURL, ok := tryReuseExistingTunnel( - t.Context(), + gotState, ok := tryReuseExistingTunnel( "ctx", "ws", browserIDEInvocation{Label: LabelVSCodeBrowser, OpenBrowser: false}, ) if !ok { t.Fatal("expected tryReuseExistingTunnel to reuse live helper") } - if gotURL != recordedURL { - t.Errorf("reused URL = %q, want %q", gotURL, recordedURL) + if gotState.TargetURL != recordedURL { + t.Errorf("reused URL = %q, want %q", gotState.TargetURL, recordedURL) } } @@ -261,13 +259,12 @@ func TestTryReuseExistingTunnel_DeadHelperReturnsFalse(t *testing.T) { t.Fatalf("TunnelStateFilePath: %v", err) } - gotURL, ok := tryReuseExistingTunnel( - t.Context(), + gotState, ok := tryReuseExistingTunnel( "ctx", "ws", browserIDEInvocation{Label: LabelVSCodeBrowser, OpenBrowser: false}, ) if ok { - t.Errorf("expected reuse=false for non-matching helper, got url=%q", gotURL) + t.Errorf("expected reuse=false for non-matching helper, got state=%v", gotState) } if _, err := os.Stat(statePath); !os.IsNotExist(err) { t.Errorf("expected stale state file removed; stat err=%v", err) diff --git a/pkg/tunnel/services.go b/pkg/tunnel/services.go index 4d525f09a5..5922f0d2bc 100644 --- a/pkg/tunnel/services.go +++ b/pkg/tunnel/services.go @@ -5,7 +5,9 @@ import ( "context" "encoding/base64" "encoding/json" + "errors" "fmt" + "io" "net" "os" "strconv" @@ -223,13 +225,14 @@ func runServicesIteration( }() command := buildCredentialsCommand(ctx, opts) + diagnostic := &boundedPreparationOutput{} err = devssh.Run(cancelCtx, devssh.RunOptions{ Client: opts.ContainerClient, Command: command, Stdin: stdinReader, Stdout: stdoutWriter, - Stderr: writer, + Stderr: io.MultiWriter(writer, diagnostic), }) if err != nil { // Drain errChan to allow goroutine to exit cleanly @@ -237,7 +240,7 @@ func runServicesIteration( case <-errChan: default: } - return err + return classifyServicesStartupError(err, diagnostic) } return <-errChan } @@ -286,7 +289,8 @@ func runOptionalGitHubServices( run func(RunServicesOptions) error, ) error { err := run(opts) - if err == nil || !opts.ConfigureGitHubCredentials || ctx.Err() != nil { + if !errors.Is(err, errUnsupportedGitHubServiceOptions) || + !opts.ConfigureGitHubCredentials || ctx.Err() != nil { return err } log.Debug("GitHub credential service unavailable; retrying other credential services") @@ -295,6 +299,34 @@ func runOptionalGitHubServices( return run(opts) } +var errUnsupportedGitHubServiceOptions = errors.New( + "agent does not support GitHub credential service options", +) + +func classifyServicesStartupError(err error, diagnostic *boundedPreparationOutput) error { + var exit *devssh.ExitError + if !errors.As(err, &exit) || exit.ExitCode == 0 || diagnostic.overflow { + return err + } + decoder := json.NewDecoder(bytes.NewReader(diagnostic.data)) + for { + var record struct { + Kind string `json:"kind"` + Message string `json:"message"` + } + if decoder.Decode(&record) != nil { + return err + } + if record.Kind != "error" { + continue + } + switch record.Message { + case "unknown flag: --host-owner", "unknown flag: --configure-github-helper": + return fmt.Errorf("%w: %w", errUnsupportedGitHubServiceOptions, err) + } + } +} + // buildPortAttributeResolver loads port attributes from the container result. func buildPortAttributeResolver(result *config2.Result) PortAttributeResolver { if result == nil || result.MergedConfig == nil { diff --git a/pkg/tunnel/services_preparation.go b/pkg/tunnel/services_preparation.go index b9a8869fa2..b969f50f21 100644 --- a/pkg/tunnel/services_preparation.go +++ b/pkg/tunnel/services_preparation.go @@ -67,7 +67,7 @@ func (p servicePreparation) startWithLifetime( go func() { completion.err = p.run() if completion.err != nil && ctx.Err() == nil { - log.Debug("credential services stopped unexpectedly") + log.Warn("credential services stopped unexpectedly") } close(completion.done) }() diff --git a/pkg/tunnel/services_preparation_test.go b/pkg/tunnel/services_preparation_test.go index 64d222063c..78b00f1c80 100644 --- a/pkg/tunnel/services_preparation_test.go +++ b/pkg/tunnel/services_preparation_test.go @@ -151,7 +151,7 @@ func TestServicePreparationReportsFailureThroughoutServiceLifetime(t *testing.T) {name: cancelledPreparation, enabled: true, cancelled: true}, } { t.Run(tc.name, func(t *testing.T) { - logs := log.InitTestObserved(t, zapcore.DebugLevel) + logs := log.InitTestObserved(t, zapcore.WarnLevel) ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) defer cancel() fail := make(chan struct{}) diff --git a/pkg/tunnel/services_ssh_test.go b/pkg/tunnel/services_ssh_test.go new file mode 100644 index 0000000000..ec5cf4b295 --- /dev/null +++ b/pkg/tunnel/services_ssh_test.go @@ -0,0 +1,117 @@ +package tunnel + +import ( + "context" + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "io" + "net" + "testing" + "time" + + devssh "github.com/devsy-org/devsy/pkg/ssh" + "github.com/stretchr/testify/require" + "golang.org/x/crypto/ssh" +) + +func TestOptionalGitHubServiceFallbackAcrossSSH(t *testing.T) { + for _, tc := range []struct { + name, message string + fallback bool + }{ + {"legacy-agent", "unknown flag: --host-owner", true}, + {"runtime-failure", "credential transport stopped", false}, + } { + t.Run(tc.name, func(t *testing.T) { + client := failingServicesSSH(t, `{"kind":"error","message":"`+tc.message+`"}`) + ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + opts := RunServicesOptions{ + ContainerClient: client, + User: "vscode", + HostOwner: "fixture-owner", + ConfigureGitHubCredentials: true, + ConfigureGitCredentials: true, + } + calls := 0 + err := runOptionalGitHubServices(ctx, opts, func(current RunServicesOptions) error { + calls++ + if calls == 1 { + return runServicesIteration(ctx, current, nil, nil) + } + require.True(t, tc.fallback) + require.False(t, current.ConfigureGitHubCredentials) + require.True(t, current.ConfigureGitCredentials) + return nil + }) + if tc.fallback { + require.NoError(t, err) + require.Equal(t, 2, calls) + } else { + var exit *devssh.ExitError + require.ErrorAs(t, err, &exit) + require.Equal(t, 1, calls) + } + }) + } +} + +func failingServicesSSH(t *testing.T, diagnostic string) *ssh.Client { + t.Helper() + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + require.NoError(t, err) + signer, err := ssh.NewSignerFromKey(key) + require.NoError(t, err) + serverConfig := &ssh.ServerConfig{NoClientAuth: true} + serverConfig.AddHostKey(signer) + listener, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + done := make(chan struct{}) + go func() { + defer close(done) + conn, err := listener.Accept() + if err != nil { + return + } + serveServicesFailure(conn, serverConfig, diagnostic) + }() + t.Cleanup(func() { _ = listener.Close(); <-done }) + conn, err := net.DialTimeout("tcp", listener.Addr().String(), 5*time.Second) + require.NoError(t, err) + t.Cleanup(func() { _ = conn.Close() }) + client, err := devssh.ClientFromConn(conn, "vscode", nil) + require.NoError(t, err) + t.Cleanup(func() { _ = client.Close() }) + return client +} + +func serveServicesFailure(conn net.Conn, serverConfig *ssh.ServerConfig, diagnostic string) { + defer func() { _ = conn.Close() }() + _, channels, requests, err := ssh.NewServerConn(conn, serverConfig) + if err != nil { + return + } + go ssh.DiscardRequests(requests) + for incoming := range channels { + channel, requests, err := incoming.Accept() + if err != nil { + return + } + for request := range requests { + if request.Type != "exec" { + _ = request.Reply(false, nil) + continue + } + _ = request.Reply(true, nil) + _, _ = io.WriteString(channel.Stderr(), diagnostic+"\n") + _, _ = channel.SendRequest( + "exit-status", + false, + ssh.Marshal(struct{ Status uint32 }{1}), + ) + _ = channel.Close() + break + } + } +} diff --git a/pkg/tunnel/services_startup_test.go b/pkg/tunnel/services_startup_test.go new file mode 100644 index 0000000000..1dc8d6c0f7 --- /dev/null +++ b/pkg/tunnel/services_startup_test.go @@ -0,0 +1,41 @@ +package tunnel + +import ( + "errors" + "strings" + "testing" + + devssh "github.com/devsy-org/devsy/pkg/ssh" + "github.com/stretchr/testify/require" +) + +func TestServicesStartupClassifiesOnlyUnsupportedGitHubFlags(t *testing.T) { + const unknownOwner = `{"kind":"error","message":"unknown flag: --host-owner"}` + exit := &devssh.ExitError{ExitCode: 1} + for _, tc := range []struct { + name, output string + err error + unsupported bool + }{ + {"legacy-owner", unknownOwner, exit, true}, + {"legacy-helper", `{"kind":"error","message":"unknown flag: --configure-github-helper"}`, exit, true}, + {"preceding-log", "{\"kind\":\"info\",\"message\":\"starting\"}\n" + unknownOwner, exit, true}, + {"unrelated-flag", `{"kind":"error","message":"unknown flag: --configure-git-helper"}`, exit, false}, + {"runtime-error", `{"kind":"error","message":"connection reset"}`, exit, false}, + {"not-error-record", `{"kind":"info","message":"unknown flag: --host-owner"}`, exit, false}, + {"partial-output", `{"kind":"error","message":"unknown flag: --host-owner"`, exit, false}, + {"unstructured", "unknown flag: --host-owner", exit, false}, + {"transport-error", unknownOwner, errors.New("SSH disconnected"), false}, + {"successful-command", unknownOwner, nil, false}, + {"overflow", unknownOwner + strings.Repeat(" ", 4096), exit, false}, + } { + t.Run(tc.name, func(t *testing.T) { + output := &boundedPreparationOutput{} + _, err := output.Write([]byte(tc.output)) + require.NoError(t, err) + got := classifyServicesStartupError(tc.err, output) + require.Equal(t, tc.unsupported, errors.Is(got, errUnsupportedGitHubServiceOptions)) + require.ErrorIs(t, got, tc.err) + }) + } +} diff --git a/pkg/tunnel/services_test.go b/pkg/tunnel/services_test.go index 878686fbbe..7891a2f19d 100644 --- a/pkg/tunnel/services_test.go +++ b/pkg/tunnel/services_test.go @@ -13,6 +13,8 @@ import ( const testCtxName = "default" +const testServiceOwner = "fixture-owner" + // newConfigWithExitAfterTimeout returns a *config.Config whose default // context sets EXIT_AFTER_TIMEOUT to the given value. func newConfigWithExitAfterTimeout(value string) *config.Config { @@ -116,7 +118,7 @@ func TestBuildCredentialsCommand_NoSigningKey(t *testing.T) { func TestDisabledGitHubServicePreservesLegacyAgentCommand(t *testing.T) { opts := RunServicesOptions{ User: "root", - HostOwner: "fixture-owner", + HostOwner: testServiceOwner, ConfigureGitCredentials: true, ConfigureDockerCredentials: true, } @@ -127,13 +129,13 @@ func TestDisabledGitHubServicePreservesLegacyAgentCommand(t *testing.T) { assert.Contains(t, command, "--configure-docker-helper") opts.ConfigureGitHubCredentials = true command = buildCredentialsCommand(context.Background(), opts) - assert.Contains(t, command, "--host-owner fixture-owner") + assert.Contains(t, command, "--host-owner "+testServiceOwner) assert.Contains(t, command, "--configure-github-helper") } func TestOptionalGitHubServiceFailurePreservesUnrelatedServices(t *testing.T) { opts := RunServicesOptions{ - User: "vscode", HostOwner: "fixture-owner", ConfigureGitHubCredentials: true, + User: "vscode", HostOwner: testServiceOwner, ConfigureGitHubCredentials: true, ConfigureGitCredentials: true, ConfigureDockerCredentials: true, ConfigureGitSSHSignatureHelper: true, ForwardPorts: true, } @@ -144,7 +146,7 @@ func TestOptionalGitHubServiceFailurePreservesUnrelatedServices(t *testing.T) { func(current RunServicesOptions) error { calls++ if current.ConfigureGitHubCredentials { - return errors.New("optional service unavailable") + return errUnsupportedGitHubServiceOptions } expected := opts expected.ConfigureGitHubCredentials = false @@ -188,3 +190,30 @@ func TestOptionalGitHubServiceDoesNotRetrySuccessDisabledOrCanceled(t *testing.T }) } } + +func TestOptionalGitHubServicePreservesPolicyAfterUnrelatedFailure(t *testing.T) { + for _, message := range []string{"SSH transport closed", "relay stopped after readiness", "unrelated helper failed"} { + t.Run(message, func(t *testing.T) { + want := errors.New(message) + calls := 0 + opts := RunServicesOptions{ + ConfigureGitHubCredentials: true, + HostOwner: testServiceOwner, + } + err := runOptionalGitHubServices( + t.Context(), + opts, + func(current RunServicesOptions) error { + calls++ + assert.Equal(t, opts, current) + if !current.ConfigureGitHubCredentials { + return nil + } + return want + }, + ) + assert.ErrorIs(t, err, want) + assert.Equal(t, 1, calls) + }) + } +} From c5107c77aa9a7e04389c55e08ca15ecc7c8a5456 Mon Sep 17 00:00:00 2001 From: Samuel K Date: Wed, 21 Oct 2026 00:31:50 -0600 Subject: [PATCH 20/20] fix(credentials): preserve edits during publication checkpoints --- pkg/ghcredentials/journal.go | 32 ++++++++++++--- pkg/ghcredentials/journal_test.go | 41 +++++++++++++++++++ pkg/ghcredentials/session.go | 4 ++ .../developing-in-workspaces/credentials.mdx | 4 +- 4 files changed, 74 insertions(+), 7 deletions(-) diff --git a/pkg/ghcredentials/journal.go b/pkg/ghcredentials/journal.go index 6fe52a2b17..25e97a2d11 100644 --- a/pkg/ghcredentials/journal.go +++ b/pkg/ghcredentials/journal.go @@ -197,13 +197,10 @@ func (s *session) writePublication(p *publication) error { if err := s.checkpointAt("publishing:" + p.Name); err != nil { return err } - target, err := s.originalPath(p.Name) - if err != nil { + needsWrite, err := s.publicationNeedsWrite(p) + if err != nil || !needsWrite { return err } - if target != p.Target { - return errAuthConflict - } if err := writePublication(p); err != nil { p.Publishing = false if saveErr := s.save(); saveErr != nil { @@ -220,6 +217,29 @@ func (s *session) writePublication(p *publication) error { return nil } +func (s *session) publicationNeedsWrite(p *publication) (bool, error) { + target, err := s.originalPath(p.Name) + if err != nil { + return false, err + } + if target != p.Target { + return false, errAuthConflict + } + current, err := readFile(target) + if err != nil { + return false, boundary("config.latest_read", err) + } + switch imageDigest(current) { + case p.Postimage: + return false, nil + case p.Preimage: + return true, nil + default: + p.Conflict = true + return false, nil + } +} + func (s *session) markApplied(p *publication) error { p.Applied = true if err := s.save(); err != nil { @@ -239,7 +259,7 @@ func writePublication(p *publication) error { if p.Result == nil { return removeFile(p.Target) } - return writeOriginal(p.Target, p.Result, p.Mode) + return writeResolvedOriginal(p.Target, p.Result, p.Mode) } func validPublication(p *publication) bool { diff --git a/pkg/ghcredentials/journal_test.go b/pkg/ghcredentials/journal_test.go index 49470bbc82..59ac9dbe5a 100644 --- a/pkg/ghcredentials/journal_test.go +++ b/pkg/ghcredentials/journal_test.go @@ -143,6 +143,47 @@ func TestJournalMalformedWorkIsDiscardedWithoutRetainingForwardedToken(t *testin require.NoFileExists(t, filepath.Join(s.Original, hostsFile)) } +func TestJournalPreservesEditsDuringPublishingCheckpoint(t *testing.T) { + for _, tc := range []struct { + name string + data []byte + }{ + {name: "login", data: []byte(authFixture("new-user", "dummy-new-login"))}, + {name: "logout"}, + } { + t.Run(tc.name, func(t *testing.T) { + s := journalSession(t) + baseline := authFixture("before-user", "dummy-before-login") + writeTestFile(t, filepath.Join(s.dir, "before", hostsFile), baseline) + writeTestFile(t, filepath.Join(s.Original, hostsFile), baseline) + writeTestFile( + t, + filepath.Join(s.work(), hostsFile), + authFixture("local-user", "dummy-local-login"), + ) + path := filepath.Join(s.Original, hostsFile) + s.checkpoint = func(stage string) error { + if stage == "publishing:"+hostsFile { + if tc.data == nil { + return removeFile(path) + } + return os.WriteFile(path, tc.data, 0o600) + } + return nil + } + require.ErrorIs(t, s.finish(context.Background()), errAuthConflict) + got, err := readFile(path) + require.NoError(t, err) + require.True(t, bytes.Equal(got, tc.data), "concurrent authentication edit overwritten") + require.NoError(t, recoverSession(context.Background(), s.dir)) + got, err = readFile(path) + require.NoError(t, err) + require.True(t, bytes.Equal(got, tc.data), "recovery replayed earlier authentication") + assertNoForwardedArtifacts(t, s.dir) + }) + } +} + const duringPublishingStage = "during-publishing" func TestJournalBindsPublicationToSymlinkTarget(t *testing.T) { diff --git a/pkg/ghcredentials/session.go b/pkg/ghcredentials/session.go index 3224bd747a..67e51d4680 100644 --- a/pkg/ghcredentials/session.go +++ b/pkg/ghcredentials/session.go @@ -574,6 +574,10 @@ func writeOriginal(path string, data []byte, mode fs.FileMode) error { if err != nil { return err } + return writeResolvedOriginal(path, data, mode) +} + +func writeResolvedOriginal(path string, data []byte, mode fs.FileMode) error { if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { return err } diff --git a/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx b/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx index 7287ec548a..70e6ad30cd 100644 --- a/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx +++ b/sites/docs-devsy-sh/content/docs/developing-in-workspaces/credentials.mdx @@ -55,7 +55,9 @@ The host must have an active `user` registered in its GitHub CLI configuration, Nonempty explicit guest authentication environment variables and existing local file or keyring tokens take precedence, even if an API rejects them. Devsy does not retry an authentication failure with the host account. A registered guest account also prevents forwarding if its token cannot be retrieved: GitHub CLI cannot reliably distinguish an absent keyring entry from a locked or unavailable keyring. Repair or deliberately remove that guest registration to use forwarding. -Forwarded invocations use private temporary configuration. Forwarded values are stripped before user changes are written back. Deliberate local logins and keyring changes stay under GitHub CLI's control. Saving the exact forwarded token back to a file may remain treated as forwarding; disable forwarding before a login that must be independent of the host. Concurrent authentication conflicts preserve the newer complete local identity and report a conflict instead of combining accounts or resurrecting a logout. +Forwarded invocations use private temporary configuration. Forwarded values are stripped before user changes are written back. Deliberate local logins and keyring changes stay under GitHub CLI's control. Saving the exact forwarded token back to a file may remain treated as forwarding; disable forwarding before a login that must be independent of the host. Devsy serializes writeback from its wrappers and preserves a newer complete local identity when a conflict is detected, instead of combining accounts or resurrecting a logout. + +An unwrapped `gh` process or another editor does not share Devsy's writeback lock. Changes detected before the final file replacement are preserved, including changes made while the journal checkpoint is saved. There is no atomic comparison-and-replacement guarantee against unrelated writers: an edit between the final comparison and replacement can still be overwritten. Avoid overlapping authentication changes through the wrapper and an unwrapped CLI or editor. Independent keyring changes are not rolled back through configuration writeback. Normal completion scrubs forwarded credentials before attempting configuration writeback. An interrupted process can leave private temporary credentials until a later invocation recovers the session; cleanup failures are reported on stderr. This is not secure erasure. Host configuration and keyring accounts are not changed by guest commands.