Three defects the catalogue triage found (#104)

[jeremymanning]

Classifying all 87 non-validating examples by their actual validate output turned up three clusters that were not example problems. Each reported the pipeline as broken when the fault was ours — so repairing examples by hand would have meant "fixing" files that were already correct. That's the argument for triaging before repairing.

1. A JSON Schema read as a model specification (6 files)

Any nested dict carrying a name or model key counted as a model spec, which makes this look like one:

parameters.schema.properties.records.items.properties
  -> {'id': {...}, 'name': {'type': 'string'}, 'active': {...}}

name there is a field called "name", and its value is a mapping — so the "model name" came back as a dict and validated_models.add(...) raised:

TypeError: cannot use 'dict' as a set element (unhashable type: 'dict')

The TypeError escaped and surfaced as Model validation failed, blaming the pipeline for a bug in the validator.

A dict is now a model specification when reached under a model key (model, llm, language_model), not merely for having a name. The non-string guard stays too — two independent defences, one test each.

My first hypothesis was wrong and worth recording: I guessed a task-level model: written as a requirements dict. None of the three files has one, and a synthetic pipeline with one produces a different, correct error. The traceback located it; the guess would have sent me to fix something that wasn't broken.

2. execution['timestamp'] read as a task id (2 files)

References were split on ., which leaves a subscript attached — so the bracket spelling of something #454 makes valid was reported as an undefined task, on pipelines that run correctly. Same class as now() before #451.

Also fixes step outputs (get_full_context['keys']), and a[0] now resolves to a rather than a name called a[0].

	. form	['...'] form
execution.timestamp	valid	valid (was: undefined task)
execution.bogus	rejected	rejected — typos still caught

3. json_encode did not exist (2 files)

Registered as an alias of to_json — the same function object, not a second implementation, since two implementations of one filter is exactly the drift #449 removed. A test asserts identity rather than equal behaviour, because identity can't drift.

Measured

	before	after
catalogue validating	30 / 117	34 / 117
blocking suite	700 passed	711 passed, 0 failed

Mutations — four, all killed

mutation	tests that failed
treat any dict with a name key as a model	1
drop the non-string model-name guard	1
drop subscript normalisation	5
give json_encode its own implementation	1

Note on the remaining failures

The files fixed here still fail for other reasons — fact_checker.yaml and web_research_pipeline.yaml now report Undefined variable: 'item', a loop-scope issue that is next. Full classification in notes/catalog-triage.md.

🤖 Generated with Claude Code