Repetition detector is state-blind: paging a date picker aborts the task (the stagnant-page signal #430 flagged is now load-bearing)

[lmorchard]

Summary

checkAndHandleRepeatedAction() decides "stuck in a loop" from the action signature alone. It doesn't consider whether the action succeeded or whether the page changed. Any UI requiring repeated operation of one control — a date picker's "next month" button, pagination, a stepper, "load more" — is therefore indistinguishable from a loop, and the task aborts on the 4th press.

This is a direct consequence of #430 working as designed, and #430 named the missing piece. See below.

Relationship to #430

#430 correctly diagnosed that the old action:ref:value signature was useless because refs churn every snapshot, so genuine loops went undetected. It proposed dropping the ref, with richer role+name matching as a follow-up. That follow-up shipped — webAgent.ts:1821 now returns action:role:name:value.

That fix did what it set out to do. But making the signature ref-independent also made legitimate repeated interaction with a single control collapse to one signature, and there's no counterweight. #430 anticipated the shape of this:

There's no detection of stagnant pages (same URL/title/major content for N steps regardless of action) — a useful signal separate from repeated actions.

That signal is now load-bearing, not just nice-to-have: it's the only thing that can distinguish "clicked the same button 4 times and nothing happened" from "clicked the same button 4 times and advanced 4 months." This issue is the measured cost of not having it.

Also still outstanding from #430: the proposed rename maxRepeatedActions → repetitionWarningCount / repetitionAbortCount didn't happen. Current source still has max_repeated_actions (default 2, warn at 3, abort at 4) with the same off-by-two confusion #430 called out.

Impact

Published hundred.jsonl runs, build 7d0f381688f5394892eead1fe28fe4fd7ebf6a9f:

• 5 of pilo's 6 Booking failures, all with the identical abort.
• Booking pass rate 38% (9/24) against browser-use's 100% (17/17) on the same tasks in the same window (4 pilo runs on a pinned digest, 95% Wilson intervals non-overlapping). browser-use has no comparable guard.
• 6 of 52 failures overall; the sixth is a webSearch repetition, since webSearch isn't exempt either.

The eval dataset asks for check-in dates 2–5 months out (Sep, Nov, Dec 2026). The picker's only forward control is "next month", so reaching the target month requires 3–5 identical clicks. The 4th aborts.

Worth flagging: this had been recorded on the eval side as a site problem — "Booking's date picker is aggressively bot-hostile, headless browsers consistently fail to interact with it." That was wrong, and the traces say so: browser:action_completed carries no error and the calendar advances each time. The clicks work. Pilo stops itself. (Corrected in the eval repo.)

Mechanism

packages/core/src/webAgent.ts:1821 — signature is semantic, carries no page state:

private createActionSignature(
  action: string, value?: string | number, identity?: { role: string; name: string },
): string {
  ...
  return `${action}:${identity.role}:${normalizedName}:${normalizedValue}`;
}

Every "next month" click yields the same string, e.g. click:button:next month:.

Thresholds at webAgent.ts:1456-57, default from config/defaults.ts:519 (max_repeated_actions: 2) → warn at 3, abort at 4. webAgent.ts:345 exempts only Scroll and Wait. The counter increments at line 1468 purely on signature equality — before the action runs, and regardless of whether the previous attempt succeeded.

Observed trace (webvoyagerx--Booking--32)

 35 ACTION click   ref=E94
 43 ACTION click   ref=E94
 51 ACTION click   ref=E94
 55 STATUS Warning: Repeated action detected - click:button:mes siquiente:
 60 ACTION click   ref=E94
 64 ABORT  Excessive repetition of action 'click' (4 times). The agent appears to be stuck in a loop.

Four successful month advances, read as a loop.

Stale refs compound it: a ref invalidated mid-interaction forces a correct retry of the same semantic target, producing an identical signature and consuming budget needed for legitimate paging. Booking--22 shows both interleaved.

Suggested fix

A repeat should mean same action, same resulting state — not same signature. In preference order:

1. Reset the counter when the action changed the page. handleBrowserDisconnect already does something analogous at webAgent.ts:2176-77 (resetting repetition state after a reconnect, precisely to avoid false loop detection), so "this repeat isn't a loop" already exists as a concept in the codebase.
2. Implement Fix repetition detection signature to survive ref churn between snapshots #430's stagnant-page signal and fold a cheap page-state fingerprint into the signature — a snapshot hash, or the subtree around the target. Repeats that change the page then get distinct signatures naturally. This is the fix Fix repetition detection signature to survive ref churn between snapshots #430 already scoped.
3. Extend REPETITION_EXEMPT_ACTIONS to known paging controls. Cheapest, but label matching is locale-dependent (below) and it won't generalise.

Raising max_repeated_actions is a mitigation, not a fix — a date picker may need many more than 4 advances, and a higher threshold lets genuine loops burn more budget.

Also worth knowing (not a pilo bug)

3 of 7 sampled traces show the button labelled in Spanish (click:button:mes siquiente:), because the Bright Data exit IP geolocates outside the US on a substantial fraction of sessions. Relevant to option 3 — any label-based special-casing must be locale-agnostic.

Reproduction

Not yet run, and this one is cheap and decisive:

1. Run a Booking task with a check-in 3+ months out (e.g. webvoyagerx--Booking--32, Sydney, Dec 2026).
2. Watch for agent:status "Warning: Repeated action detected - click:button:next month:" then task:aborted.
3. Re-run with --max-repeated-actions 12. If it then progresses past the date picker, the guard is confirmed as the blocker rather than the site.

Step 3 is the discriminator and is worth running before picking a fix.

Regression guard: drive checkAndHandleRepeatedAction() with four identical successful click outputs where page state changes between each, and assert no abort. packages/core/test/webAgent.test.ts has repetition-abort tests at ~4470, ~4683, ~4902, ~5048 to model on.

---

Source claims cited against mozilla/pilo@7446077e. Evidence: 6 task artifacts from scheduled-bu-pilo-tab-hundred runs 1785337200 / 1785423600, participant pilo-cli — public, no auth: https://storage.googleapis.com/pilo-public-eval-reports/reports/<run>/<task-id>/result.json. Full analysis: Mozilla-Ocho/pilo-evals-judge#122.

🤖 Investigated with Claude Code

[lmorchard]

Repro done — confirmed, plus a quantitative prediction that holds

I ran this deterministically rather than against live Booking, because the contested claim ("the guard fires on legitimate state-changing repeats") is provable without a browser, and a live run drags in bot-detection and exit-IP variance that would only muddy it.

Two tests against origin/main (b5fc5f2a). Note packages/core/src/webAgent.ts is byte-identical between that and 7d0f381688f5394892eead1fe28fe4fd7ebf6a9f, the build the failures were measured on, so this exercises the cited code exactly.

The setup mirrors the real traces: the agent clicks a control whose identity is {role: "button", name: "Next month"}, every click returns success: true, and getTreeWithRefs returns a different snapshot each turn (the calendar advancing September → January).

✓ REPRO #640: aborts while paging a date picker, though every click succeeds and the page changes
✓ REPRO #640: raising max_repeated_actions lets the same paging run through

Full file still green: 85/85.

The sharp finding: the ceiling is 3, not 4

result.stats.actions is 3, not 4 — the 4th press trips the abort and returns actionExecuted: false, so it never executes. The agent can advance the calendar at most 3 months.

With maxRepeatedActions: 12, all 5 presses execute and no repetition abort fires.

That ceiling predicts the live pass/fail data exactly

If the cap is 3 month-advances, Booking tasks should pass when the check-in is ≤3 months out and fail beyond it. Across 4 runs on the pinned build:

task	pass rate	months ahead	target date
Booking--0	4/4	1	August 29, 2026
Booking--1	2/4	2	September 17, 2026
Booking--9	1/4	2	September 5, 2026
Booking--12	2/4	3	October 6, 2026
Booking--22	0/4	4	November 6, 2026
Booking--32	0/4	5	December 15, 2026

Monotonic, and the boundary lands exactly where the unit test says it should. Everything within the cap passes at least sometimes; both tasks beyond it fail 0/4. Two independent lines of evidence converging on the same number.

(The partial rates at 2–3 months are other failure modes intruding, not the guard — those tasks reach the picker fine.)

Why I'm not doing the live --max-repeated-actions 12 run

The issue proposed it as the discriminator, and I no longer think it adds anything. The unit test isolates the guard more cleanly than a live run could, and the dose-response above already rules out a site-side explanation — a bot-hostile picker wouldn't produce a clean monotonic cliff at exactly the threshold the code implies. A live run would also need a working browser backend (local foxcloud is down, in-container Chromium is broken on arm64) and would put Booking traffic on a residential IP, which reliably changes the page. Happy to run it if you want it for completeness, but it would be confirmation theatre.

Sibling test worth noting

Immediately below these in the same describe block:

should not flag legitimate repeated scroll actions — "Scroll repeats with the same direction are legitimate workflow (traversing an infinite-scroll feed). The detector must exempt them."

The codebase already holds the principle that legitimate repeats must not abort. It's implemented only for Scroll and Wait (webAgent.ts:345). Clicks on paging controls are the same category.

The patch

Tests are on branch repro/640-repetition-state-blind (not pushed — say the word and I will). They're written to be deleted or inverted when the fix lands: today they assert the buggy behaviour, so whoever fixes this should flip the first one's expectation rather than delete it, which then guards the fix.

I have not attempted a fix. Option 1 from the issue (reset the counter when the action changed the page) is the smallest change and has a precedent at webAgent.ts:2176-77, but which of the three options is right is a design call for whoever owns this code.

[lmorchard]

Repro branch is now pushed: repro/640-repetition-state-blind — one commit, packages/core/test/webAgent.test.ts only, no source changes. Rebased on current origin/main (b5fc5f2a), 85/85 green.

As noted above, the two tests assert the buggy behaviour today, so whoever fixes this should invert the first one's expectation rather than delete it — that turns the repro into a regression guard for the fix.