Skip to content

fix: inspect local SSH listeners for Testbox activity - #15

Open
steipete wants to merge 2 commits into
useblacksmith:mainfrom
steipete:fix/local-ssh-activity
Open

steipete wants to merge 2 commits into
useblacksmith:mainfrom
steipete:fix/local-ssh-activity

Conversation

@steipete

@steipete steipete commented Aug 30, 2026 •

Copy link
Copy Markdown

What Problem This Solves

Testbox idle detection compares local socket output with the externally forwarded client port. A long-running command can therefore go uncounted after its one-time activity marker becomes old, even while SSH remains connected.

Two isolated live reproductions used Blacksmith CLI 0.4.57 through Crabbox with a one-minute idle limit. Both observed advertised port 64000, local SSH port 22, an established connection throughout, no match from the action's original socket pipeline, and an unchanged activity marker. The action reported idle completion anyway:

Target command Observed command outcome Hosting job
125 seconds Survived into runner teardown and returned 0 after the action had declared idle 33320794528, success
480 seconds No completion receipt; transport eventually returned 255 33321080163, success

Both leases were independently verified completed. The hosting job's success is not command-success evidence.

Why This Change Was Made

Keep activity detection with its lifecycle owner, run-testbox: resolve all local sshd listener ports with sudo -n sshd -T, then query established connections using ss's local-source-port filter. Preserve the forwarded port in connection information and phone-home data. Inspection failure fails the action rather than silently treating unknown activity as idle.

This removes the broad advertised-port grep, including accidental matches on unrelated peer ports. It does not add a client heartbeat, change idle/test timeouts, change providers, or modify any application tests. Action implementation delta: +13/-2 lines; executable test coverage: +121 lines; test workflow wiring: +3 lines.

User Impact

The intended behavior is that connected noninteractive commands, interactive sessions, and rsync transfers keep the Testbox alive; idle leases still expire normally. The command's exit status and reports remain authoritative for command results. The README makes the hosting-job/command-result distinction explicit.

Evidence

  • python3 -m unittest discover -s tests -v: five test methods pass, covering default/non-default/multiple listeners, marker fallback, idle expiry, unrelated peer ports, and failed inspection.
  • The unchanged regressions produced seven intended failing subcase assertions on the previous action.
  • yamllint -d relaxed action.yml: passed.
  • actionlint .github/workflows/test.yml and git diff --check: passed.
  • Independent Codex autoreview: scoped-clean at P0.
  • Live long-command proof passed: isolated run 33324478030 completed a 1930.029-second command with the lease idle limit still at one minute. All 194 samples saw an established local SSH connection, no match from the original advertised-port query, and an unchanged marker timestamp. The command and Crabbox wrapper returned 0.
  • A subsequent intentional exit37 propagated as a failed command with exit37. Explicit stop returned0; an independent provider query verified completed, and the local claim/key directory were removed. The hosting Actions job ended cancelled after that explicit stop; its conclusion is not the command verdict.
  • Live idle-side proof passed: run33326851073 completed normally after native transport closure. Blacksmith0.4.57 actually passes ControlMaster=auto and ControlPersist=300. An allowlisted argv observer exec'd the unchanged system SSH, while passive lsof observations of the owned endpoint showed the persistent master still established294s after command return, absent at309s, and the lease completed at340s. No SSH options were changed and no additional SSH probe refreshed activity. The action logged its one-minute idle expiry and the hosting job completed successfully.
  • Actions-side and SSH-command-side probes recorded the same network/PID namespaces. The initial three-minute post-command check used the wrong time origin: command return was not transport disconnection. The README now makes that distinction explicit.
  • OpenClaw's repository workflow validation also passed for its isolated proof branch.

The tested action revision is 7aeb7bab6f521ca30924d7dbf1fd47d6fcada294; current PR head 2b6b1be536ec7f3c73757fedf5460a27ab4856b4 differs only in the README clarification. Action and test bytes are unchanged. Upstream CI run33327156834 is action_required, awaiting maintainer approval of the fork workflow—not a passing CI claim. No merge or release has occurred.

The motivating application proof also had untimestamped Vitest worker IPC EPIPE errors. Their worker-exit cause and ordering against lease expiry remain unproved; this PR does not claim to fix or explain those errors.

@steipete
steipete marked this pull request as ready for review August 30, 2026 18:11
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 1, 2026
Adopt the reviewed VM-local SSH activity fix from useblacksmith/run-testbox#15 through an immutable temporary fork pin in the three Linux Testbox workflows. Keep existing idle and job limits unchanged.

Track premature lease expiry: #134718
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 1, 2026
Adopt the reviewed VM-local SSH activity fix from useblacksmith/run-testbox#15 through an immutable temporary fork pin in the three Linux Testbox workflows. Keep existing idle and job limits unchanged.

Track premature lease expiry: #134718
steipete added a commit to openclaw/openclaw that referenced this pull request Sep 1, 2026
* fix(ci): keep active Testbox commands alive past idle timeout

Adopt the reviewed VM-local SSH activity fix from useblacksmith/run-testbox#15 through an immutable temporary fork pin in the three Linux Testbox workflows. Keep existing idle and job limits unchanged.

Track premature lease expiry: #134718

* test(ci): align Testbox handoff contracts with reviewed action

Keep TruffleHog provisioning ordered before the named handoff, pin the reviewed Linux action, and separately retain the disabled Windows vendor marker.

Follow up the active lease-expiry fix: #134725

* test(ci): cover doctor runtime preparation selection
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Sep 2, 2026
…aw#134725)

* fix(ci): keep active Testbox commands alive past idle timeout

Adopt the reviewed VM-local SSH activity fix from useblacksmith/run-testbox#15 through an immutable temporary fork pin in the three Linux Testbox workflows. Keep existing idle and job limits unchanged.

Track premature lease expiry: openclaw#134718

* test(ci): align Testbox handoff contracts with reviewed action

Keep TruffleHog provisioning ordered before the named handoff, pin the reviewed Linux action, and separately retain the disabled Windows vendor marker.

Follow up the active lease-expiry fix: openclaw#134725

* test(ci): cover doctor runtime preparation selection
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant