Expose per-check alert counts to metric formulas, for composite/density scoring across multiple rules

[theredspoon]

Problem

metric rules already support arbitrary custom formulas (Tengo expressions evaluated against f.ComputeMetrics()), not just the built-in readability formulas. What's missing is an input: there's no way for a metric formula to know how many times other rules have already fired, so a rule combining signals from several other rules is not expressible.

Motivating case

A style package like tbhb/vale-ai-tells includes about 110 independent pattern rules, each an isolated per-instance match, none currently setting limit:. One or two firing in a document reads as normal writing. Many different ones firing across the same document is a stronger signal. This would make that signal expressible as an ordinary metric rule, without requiring every contributing rule to be reconfigured first.

What exists today, and why it doesn't cover this

File.AddAlert has a limits map[string]int keyed by check name, but it's tied to a different, opt-in feature: it only increments when a rule sets limit: N in its own YAML, to cap how many alerts that rule reports. Checked all 110 rules in tbhb/vale-ai-tells, including the three in the example formula below: none set limit:. For a package like that, f.limits stays at zero for every rule, it doesn't give a metric formula what it needs.

Proposed fix

A new, unconditional per-check counter, incremented on every reported alert regardless of Limit, separate from the existing opt-in reporting-cap mechanism.

Exposed to metric formulas as an indexable object rather than as individual identifiers, since check names come from user-chosen style and rule filenames and can contain arbitrary characters that don't map cleanly onto Tengo identifiers:

check["AITells.FigurativeOwns"] + check["AITells.HedgingPhrases"] + check["AITells.EmptyPadding"] > 3

A check that's loaded and could apply to the current file, but hasn't fired yet, reads as 0. A name that doesn't correspond to a real, applicable check produces a clear error naming it, rather than silently reading as zero.

NewMetric forces scope: summary, so the composite score this enables is a whole-document signal.

Why dispatch order is safe for this

Vale's lint pipeline runs in two phases. Every format (lintMarkdown, lintADoc, lintRST, lintDITA, lintOrg, lintXML, lintHTML) funnels into lintHTMLTokens, which walks the whole document running every block-scoped rule (existence, sequence, occurrence, etc.), each synchronously reporting through f.AddAlert. Only after that walk fully completes does lintSizedScopes build the scope: summary block and run metric rules. Any per-check counter populated in AddAlert is complete by the time a metric formula reads it.

Other formulas this enables

A few beyond the vale-ai-tells density case above:

• Readability consensus. Pick one formula per distinct measurement instead of summing all seven built-in ones (several are near-duplicates of each other):

  check["Readability.FleschKincaid"] + check["Readability.ColemanLiau"] + check["Readability.GunningFog"] + check["Readability.SMOG"] > 2
  

  > 2 requires a real majority, at least three of four, instead of nagging the moment one formula crosses its own line.

  Why these four, and other notes on this example

  Several of the seven built-in formulas are near-duplicates measuring the same underlying stats (FleschReadingEase is FleschKincaid's two inputs rescaled; AutomatedReadability and LIX both reuse ColemanLiau's character-length approach), so a naive sum of all seven mostly double-counts three or four real signals wearing seven labels. Each formula picked here is built from a different input: FleschKincaid from average syllables per word, GunningFog from percent of complex words, ColemanLiau from average word length in characters, and SMOG from polysyllabic-word density, the only one of the four that skips sentence length entirely.

  Combining readability formulas wasn't previously impossible: the built-in readability check type (internal/check/readability.go, separate from the metric-based Readability style used above) already lets a rule pick several metrics and average their raw grade values against one threshold. That's a different approach from voting. Averaging blends every input into one number, so one formula reading unusually high can still pull the average past the threshold even if the other four read comfortably below it. Voting resists that: it only fires once an actual majority agrees. Both are useful. This example is specifically about the vote.

  check[...] is always a real per-document count. It reads as 0 or 1 here specifically because every metric rule (Readability.* included) is forced to scope: summary and reports at most one alert per document, so summing four of them really is counting votes. That's not true of every check: check["Bugs.MatchCase"] in the consistency-scoring example below is a multi-value count, since an existence/substitution rule can match several times in one document.

• Consistency scoring. Several narrow terminology/case-style rules (Bugs.MatchCase, Bugs.TermCase, Bugs.SameCase, and similar) each catch one kind of inconsistency. One hit is a normal typo-level nit. Several different ones firing across the same document signals a process breakdown, maybe no shared glossary, maybe several authors who never synced on terminology, maybe a style guide nobody enforced. Summing their counts turns that into a document-level signal one rule can't express alone.

• Weak-writing composite. Passive voice, weasel words, and wordiness are each a mild, individually-ignorable nudge. Combined past a threshold, they're a real "tighten this section" signal one rule alone can't express.

• Accessibility risk rollup. Several narrow a11y pattern rules (missing alt-text conventions, heading-level jumps, and the like) combined into one composite flag when enough of them fire together in the same document.

None of these need new checks, just a metric rule combining counts from checks that already exist.

Status

Implemented in #1166.