fix(worker): one-shot run exits 1 on success — invert to exit 0#169
Open
RealDiligent wants to merge 2 commits into
Open
fix(worker): one-shot run exits 1 on success — invert to exit 0#169RealDiligent wants to merge 2 commits into
RealDiligent wants to merge 2 commits into
Conversation
Contributor
Author
added 2 commits
July 13, 2026 13:26
The non --loop branch did `raise SystemExit(process_once(...))`. process_once returns 1 when it handled a job and 0 when the queue was empty — a "did work" signal for the --loop sleep decision, not an exit status. So a one-shot run that successfully processed a job exited 1, which supervisors/cron/systemd read as failure (and an empty queue looked like success). Run the pass and exit 0 on normal completion; genuine failures still raise and exit nonzero. Added tests covering both process_once returns and the failure path. Closes mini-router#168
After the submission size check became warning-only, a wrong theta length no longer produces a hard error, so test_wrong_theta_length_fails could never pass: it asserted result.ok is False and relied on make_spec().n_total differing from a hardcoded 128 (that n_total is 128 in the CI environment). main's CI is currently red on this test for every PR branched off it (mini-router#160). Rename to test_wrong_theta_length_warns and assert the current behavior: a theta length that disagrees with the summary's declared n_total is surfaced as a non-fatal warning (ok stays True), independent of the canonical n_total.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
validator/src/eval_backend/worker.pymain()one-shot branch:process_oncereturns1when it processed a job and0when the queue was empty — a "did work" signal for the--loopsleep decision, not a process exit status (the--loopbranch uses it asif processed == 0: sleep(...)). Passing it toSystemExitinverts exit semantics: a run that successfully processes a job exits 1 (which supervisors/cron/systemdoneshot/CI read as failure), and an empty queue exits 0.Impact
Any one-shot invocation (
python -m eval_backend.workerper timer tick, instead of--loop) reports every successful evaluation as a failed process — spurious alerts/retries or a unit stuckfailed.Fix
A completed one-shot pass exits 0 whether or not the queue had work; genuine failures already raise (the
except Exception: ... raiseinsideprocess_once) and still exit nonzero.Tests
New
validator/tests/test_worker_exit.pystubs the DB/engine bootstrap and asserts: one-shot exits 0 for bothprocess_oncereturns (1 = did work, 0 = idle), and a raisingprocess_oncestill propagates (nonzero). Verified theprocess_once==1case exits 1 on the old code and 0 with the fix.Risk / tradeoffs
Minimal.
--loopbehavior is unchanged. Only the one-shot exit code changes, from an inverted value to correct success/failure semantics.Closes #168