v1.6.5 Release

v1.6.5

A correctness release for the local validator.
The schema-driven step checks now run on the nested steps where real UI actions live, the step schema has been reconciled against thousands of real Provar test cases, and a set of false positives that flagged genuine IDE-authored XML as broken are fixed.
The net effect on 2,701 real customer, POC and demo .testcase files is a slightly higher mean quality score and fewer files below threshold, with more valid test cases recognised as valid.

Highlights

• Schema-driven validation now reaches nested steps. Per-call structural rules previously ran only on top-level apiCalls, so they never fired on the nested steps where real UI actions live. They now recurse cleanly, with no duplicate node visits.
• click is no longer flagged as a hallucination. click is a real Provar interaction (the IDE recorder emits it, third in frequency behind action and set). The old denylist flipped 164 valid test cases to invalid. Fixed.
• A pre-existing critical false positive is fixed. The Provar AI namespace was listed with a value that exists in no real file, so freshly IDE-authored XML was declared "does not exist in Provar". Both the current and the legacy namespaces are now allowed, and 12 files recover is_valid.
• Argument requirements are now tiered. required / recommended / ide_emitted score at major / minor / info, so IDE-parity drift on its own can never push a valid test case below threshold.
• Over-gating on large tests is fixed. Per-step violation emission bypassed the score damping (a 71-step file could deduct enough to score 0). The tiered rules now emit a single aggregated violation carrying a count.

Fixed

• Step schema reconciled against the corpus. 59 of 73 entries were corrected. Phantom required arguments that appear zero times in real files were removed (for example ForEach valuePath/iterationPathName, StepGroup name, RestRequest webConnectionName/endpoint/method, and the Read/Split/Match/Replace argument names). A further 28 entries carried phantom argument names in optional_arguments that the provar_step_schema tool was serving verbatim to the generating model; these were dropped where the corpus has strong evidence of absence.
• Wrapper-aware argument readers. Three per-argument rules read call arguments through a shared reader that previously looked only at bare call.argument children. Real IDE-exported test cases nest those arguments under an <arguments> wrapper, so on genuine files the rules saw nothing and stayed silent. The reader now prefers the wrapper and falls back to the bare form, so the rules fire on real files as intended.
• Connection reference false positives. Reference consistency now resolves against the project's .testproject connections as well as connect-step resultNames, and scopes suppression to the affected connection family. This removed 66 false positives on project-level connections.
• Bundled example corrected. The bundled authoring example was not IDE-faithful (it omitted 19 IDE-emitted arguments) and has been fixed.

Changed

• Scoring is aggregated, not per-step. All three tier rules now emit one violation with a count, so the logarithmic damping in calculateBPScore applies as designed instead of being bypassed by linear per-step summing.
• Duplicate checks are suppressed. (apiId, argument) pairs already enforced by a dedicated mustContainArgument rule are read from the catalogue at runtime and suppressed, so one defect is never scored twice (21 pairs resolved, 16 duplicate checks removed).
• Relevance-gated examples. examples_retrieve now returns nothing for an off-topic query rather than an unrelated Salesforce UI example, and sets source (corpus / bundled / none) on every path.
• Recovery guidance resolves. Fallbacks that named an unreadable docs resource now name the provar_step_schema tool, and that tool is registered in the validation, qualityhub and authoring groups so the recovery path works in a filtered session.

Behaviour change to note

• Some scores will move. This release corrects pre-existing bugs in both directions.
  Test cases wrongly marked invalid by the click denylist or the AI-namespace check now pass again (over the corpus: 12 in, 2 out, both outward flips genuine load-blocking defects).
  A test case that carries one of the wrapper-aware defects and was previously scored as clean will now surface that violation, because the rule can finally see the arguments.
• Tiering softens IDE-parity noise. The new info-severity IDE-parity tier means IDE-always-emitted arguments no longer gate validity, so a test case that dipped below threshold purely on those will recover.

Known follow-ups (not in scope)

• Layer-1 issues[] has no count concept yet, so a high-duplication file still emits per-occurrence issues there (no gating or score impact).
• The opaque-connect guard trades a few genuine detections on an intentional-violations fixture for removing the 66 false positives.
• A handful of low-evidence schema arguments remain unproven either way and were retained pending more IDE-minted samples.

Testing

• 1,588 unit tests passing, 0 failing
• tsc --noEmit clean; eslint src test clean
• Version bumped to 1.6.5 with server.json in sync
• Docs updated: docs/mcp.md, docs/mcp-pilot-guide.md, docs/PROVAR_TEST_STEP_REFERENCE.md, regenerated docs/VALIDATION_RULE_REGISTRY.md, README tool count

---

Full Changelog: v1.6.4...v1.6.5