Give validate and run one shared notion of an action (#241) - #438
Merged
Conversation
`ToolValidator` treated a step's `action:` as a tool name whenever the step had no `tool:` key. That is right for the legacy single-field form (`action: filesystem`) and wrong for a model step, which by design names no tool -- so `validate` reported "Tool 'generate' not found in registry" while the executor ran the very same document correctly. They disagreed because they held two separate, unshared notions of what an action is. The executor's lived in ~12 `if action_str == ...` branches in HybridControlSystem plus a `supported_actions` list in ModelBasedControlSystem, and those two already disagreed with each other: `generate_structured` was implemented but undeclared, which is how it went unnoticed that only the hyphenated spelling dispatched. core/actions.py is now the single source of truth. HybridControlSystem *dispatches off* BUILTIN_ACTION_HANDLERS rather than keeping a parallel if-chain, so the mapping cannot claim an action the executor does not run -- the drift is structurally impossible rather than merely discouraged. The validator accepts exactly the names in it. The tool lookup still runs first, so the legacy single-field form keeps its full parameter validation: `filesystem` is both a registered tool and a built-in action name, and it must resolve as the tool. The two natural-language families -- "echo ..." and "write the following content to report.md" -- stay pattern-matched, since prose has no name to register. They keep their existing order relative to each other. The strict xfail added in #437 did its job: it XPASSed the moment the two agreed, and is now the plain assertion it was standing in for. Its disappearance is the evidence this issue is closed. tests/test_builtin_actions.py pins both directions: every registered action resolves to a handler that exists on the control system, every built-in action is accepted by a strict validator, an action that is neither tool nor built-in is still rejected, and the legacy and two-field forms still resolve to the tool. Blocking suite 398 -> 435 passed, 13 skipped, 0 failed, and now 0 xfailed -- the "no xfails in supported contracts" release gate is met for this module. Collection 3194 -> 3230, 0 warnings. Example catalog 2 -> 3 of 111 validating: examples/basic/hello_world.yaml now compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Member
Author
|
CI 9/9 green, and the legacy backlog moved in the right direction. Same job, same environment,
Two legacy tests that were red now pass, nothing regressed. The Worth noting what this comparison cost: the legacy job publishes a scalar and runs with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #241.
The disagreement
ToolValidator._extract_tool_nametreated a step'saction:as a tool name whenever the step had notool:key. Correct for the legacy single-field form (action: filesystem), wrong for a model step, which by design names no tool:validaterunaction: generateTool 'generate' not found in registryThey disagreed because they held two separate, unshared notions of what an action is. The executor's lived in ~12
if action_str == ...branches inHybridControlSystem, plus asupported_actionslist inModelBasedControlSystem— and those two already disagreed with each other. That is how it went unnoticed that onlygenerate-structureddispatched whilegenerate_structuredfell through to the prompt fallback (fixed in #437).The fix
core/actions.pyis the single source of truth. The important part is thatHybridControlSystemnow dispatches offBUILTIN_ACTION_HANDLERSinstead of keeping a parallel if-chain:So the mapping cannot claim an action the executor does not actually run — drift is structurally impossible rather than merely discouraged. Adding a runtime action is one entry; both consumers follow.
Three things deliberately preserved:
filesystemis both a registered tool and a built-in action name, so the legacy single-field form keeps its full parameter validation rather than being skipped as an action."echo ..."and"write the following content to report.md"— stay pattern-matched, since prose has no name to register, and keep their existing order relative to each other.The sentinel worked
The strict xfail added in #437 XPASSed the moment the two agreed:
It is now the plain assertion it was standing in for. Its disappearance is the evidence this issue is closed, which is the whole reason it was
strict=True.Coverage
tests/test_builtin_actions.pypins both directions:allow_unknown_tools=False)tool:, neveraction:generate_structuredreturn proseNumbers
uv lock --check/ ruff onsrc/git diff --checkThe catalog gain is small but it is
examples/basic/hello_world.yaml— the flagship example, which did not compile before this.Not in this PR
success: true. Root cause located:TemplateManagercatchesUndefinedError, logs a warning, and returns the original template text (template_manager.py:516, and again at :81). That fallback exists because resolution runs in multiple passes, so simply re-raising would break legitimate deferred resolution; the fix belongs at the point where resolved parameters are handed to a tool, before any side effect. Separate change.supported_actionscapability lists. Both control systems still declare their own, and both are incomplete (HybridControlSystemomitsgenerate_text,evaluate_condition,loop_complete, …). Those lists feed control-system selection, not dispatch, so aligning them can change which system claims a task — a behavioural change that deserves its own PR rather than a drive-by.