Skip to content

sequence has no way to count occurrences of the whole pattern within a scope, only occurrence does, and occurrence has no tag support #1161

Description

@theredspoon

Problem

sequence matches a tagged multi-token pattern but alerts once per match. It has no threshold or count concept. occurrence has a count threshold (Max/Min) but only matches a raw regex, no tag/upos field at all. Neither can count occurrences of a tagged pattern within a scope.

Motivating case

tbhb/vale-ai-tells includes VerbTricolonDensity.yml, an occurrence rule (max: 1) meant to flag more than one AI-cliché verb tricolon per paragraph. Its regex can't require the matched words be verbs, so it fires on plain noun-phrase lists too. The single-instance version of the same rule has an unmerged sequence-based fix using real POS tags. The density version can't get the same fix today: no way to express "this tagged pattern needs to occur 2+ times in one paragraph."

Root cause

NewSequence unconditionally rewrites every declared scope to sentence-level via sentenceScope(). Run() is invoked once per sentence regardless of what a rule declares. scope: paragraph on a sequence rule currently has no effect, it collapses to the same "every sentence, everywhere" behavior as an undeclared scope.

Practical effect: a paragraph with two real matches split across two sentences doesn't produce any density alerts, since each sentence individually has only one match, at or under any reasonable threshold.

Proposed fix

Contained to sequence.go, plus small additions to the NLP infrastructure it depends on:

  • When a rule opts into count-threshold behavior, skip the sentenceScope() rewrite. Run() is then called once per the rule's actually-declared scope (e.g. once per paragraph).
  • Tag each sentence of that scope separately, instead of tagging the whole block once and inferring sentence boundaries afterward. The existing tagging function already segments and tags sentence by sentence internally, so this reuses information the codebase already had.
  • Sentence membership becomes a direct fact this way, since it's determined by which tagging call produced a token, so a match can never span two sentences by construction and the match-walk doesn't need a separate boundary guard.
  • Threshold/message logic mirrors occurrence's existing Max/Min convention. It runs over a correctly-scoped match set built one sentence at a time.

Alternative considered: extending occurrence.go

occurrence already runs once per a rule's real declared scope correctly, including working scope: raw, and could reuse sequence's tag-matching helpers directly (same package).

Set aside: occurrence runs concurrently across rules matching one block; sequence is deliberately excluded from concurrent execution because its tagging cache isn't synchronized (confirmed with Go's race detector, not just by reading the code). Fixable, but it touches a second file plus shared dispatch infrastructure other check types rely on, and gives Occurrence two structurally different, mutually exclusive matching modes. The sequence.go-only fix touches less shared code.

Related work

This request is the remaining piece: counting the whole pattern within its real declared scope, not one token within it.

Status

Implemented in #1162.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions