One graph decides what runs before what

[jeremymanning]

Resolves #465 with option A: infer the dependency and schedule on it.

The defect

A template reference is a data dependency — {{ make.result.path }} cannot exist until make has run. Three parts of the compiler agreed about that and none of them told the scheduler:

mechanism	scanned parameters?	outcome
DependencyValidator	no — only for_each/condition/while/until	used for cycles, missing the commonest case
DataFlowValidator.data_flow_graph	yes	logged, then discarded
_analyze_template (regex)	yes	stored on Task.template_metadata, scheduled from by nothing

Task.dependencies came from the explicit key alone. So a pipeline validated and then failed at run time:

                          run exit    out_b.txt
before (main @ 9615f61)      1        MISSING
after                        0        made=./out_a.txt

Worth calling out from the mapping: DependencyValidator._extract_template_dependencies matched only six hard-coded suffixes (result|output|data|content|status|metadata), so make.path was not recognised as a task reference at all. The commonest reference in the language was invisible to the graph that checks for cycles.

The fix

core/dependency_graph.py is the single answer, following the review's constraints exactly:

1. One graph containing explicit dependencies, template-derived dependencies (every nested field), and control-flow dependencies — each edge tagged with its origin and the parameter path that produced it.
2. That graph is validated for cycles and self-reference.
3. Every Task.dependencies is built from it.
4. The same graph produces execution levels and the diagnostics.

Consolidation, not a fourth mechanism. Three extractors became one AST extractor in core/template_scope, which is why re is now an unused import in dependency_validator.py — the clearest confirmation the regex is gone. Being the scope-aware extractor from #461, it excludes loop-local names for free.

ControlFlowCompiler._build_task has many exits — conditional, goto, for-each, action-loop — each building through a different handler that reads dependencies off the definition. It is now a thin wrapper applying the graph once, so control-flow steps are not the one kind of step silently keeping the old behaviour.

The implicit_dependency lint

INFO severity, never failing. Inference is a supported way to write a pipeline, not a mistake, and failing here would break every pipeline relying on it. Carries stable structured fields: code, severity, component, path, suggestions, and metadata with step / referenced_step / parameter_path / origin.

Two corrections to my own work

The reproduction I filed in #465 was partly wrong. It used {{ make.path }}, which is not a field the filesystem tool returns — it returns {'result': {'path': ...}, 'success': ...}. That conflated the missing edge with a wrong field name and would have failed even with the bug fixed. I've corrected the issue. {{ make.path }} is now its own test asserting the opposite: it must still fail, because ordering a step correctly does not invent a field the producer never returned.

A mutation passed for the wrong reason. Reverting YAMLCompiler._build_task failed only 2 unit tests while the e2e reproduction still passed — because the CLI runs through ControlFlowCompiler, whose wrapper still applied the graph. The mutation was not a faithful inverse. With both compilers reverted, the e2e test fails as it should.

Verification

• 39 tests covering all 13 required cases, plus origin tagging, declaration-order stability, longer cycles, and {% if %}/filter/subscript/index/nested-list reference forms.
• Mutations, each caught: Task.dependencies ignoring the graph; the graph skipping parameters (22 failures); the cycle check removed.
• Blocking suite 831 passed, 0 failed. Catalogue 50/117, all 50 baseline files still validate. Lint clean.

Not in this PR

Warning visibility (the second half of #465) and the catalogue baseline rename — both from the review's sequence, both separate concerns.

🤖 Generated with Claude Code