diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e1449e..b39db62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,25 @@ # Changelog +## [0.1.63] - 19-07-2026 + +### Fixed + +- **`browser_fetch`/`browser_fetch_batch` `browserFallback` never escalating chrome-heavy SPAs** — `isThinShell` gated escalation on raw-HTML visible-text length (`< 600` chars); client-rendered pages with a lot of nav/footer chrome (e.g. `digitec.ch`, `fr.skoda.ch/apps/stock`) cleared that threshold while their real content still only existed after JS hydration, so the fast-path extraction silently returned `wordCount: 0` with `escalated: false`. `browserFallback` now also escalates on the extraction RESULT: when it comes back empty/near-empty (`wordCount < 15`, and not already fixed by hollow-extraction recovery) on HTML carrying JS-rendering markers (SPA mount point or heavy ` + + + + General Discussion + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ + + + +
+
+
+
+ + + + + +
+ +
+
+
+ +
+
+ +
+
+
+

General Discussion

+

Discuss your own setups, firewalls, security and technology in general.

+
+
+ +
+ Go Down + Pages1 2 3 ... 471 +
+ +
+
+ +
+
+
+ + +
+
+
+
+ +
+
+ + + Please Make a Donation to OPNsense + +
+

+ Started by packet loss +

+ Pages1 2 3 ... 16 +
+
+
+

Replies: 228
Views: 512,940

+
+
+

May 29, 2026, 02:22:59 PM
by Monviech (Cedrik)

+
+
+
+
+ + +
+
+ +
+
+

Replies: 0
Views: 1

+
+
+

Today at 02:52:33 AM
by opnsense1

+
+
+
+
+ + +
+
+ +
+
+

Replies: 0
Views: 1

+
+
+

Today at 02:52:02 AM
by grevelle

+
+
+
+
+ + +
+
+
+
+
+
+ + + Recovery with a backup file + +
+

+ Started by tdalej +

+ +
+
+
+

Replies: 3
Views: 77

+
+
+

July 18, 2026, 06:59:31 PM
by tdalej

+
+
+
+
+ + +
+
+ +
+
+

Replies: 16
Views: 635

+
+
+

July 18, 2026, 05:36:10 PM
by Vectralis

+
+
+
+
+ + +
+
+
+
+
+
+ + + Captive Portal Not Working + +
+

+ Started by somanet +

+ +
+
+
+

Replies: 7
Views: 128

+
+
+

July 18, 2026, 02:53:50 PM
by viragomann

+
+
+
+
+ + +
+
+
+
+
+
+ + + Unbound DNS keeps crashing every 2-3 mins. + +
+

+ Started by apoorv569 +

+ Pages1 2 3 +
+
+
+

Replies: 32
Views: 1,085

+
+
+

July 18, 2026, 09:26:43 AM
by lmoore

+
+
+
+
+ + +
+
+
+
+
+
+ + + Periodic NIC issues (?) with Protectli Vault, Intel i226-V + +
+

+ Started by fornax +

+ Pages1 2 3 ... 5 +
+
+
+

Replies: 70
Views: 2,768

+
+
+

July 18, 2026, 09:08:23 AM
by franco

+
+
+
+
+ + +
+
+ +
+
+

Replies: 5
Views: 193

+
+
+

July 17, 2026, 07:46:31 PM
by BrandyWine

+
+
+
+
+ + +
+
+ +
+
+

Replies: 7
Views: 261

+
+
+

July 16, 2026, 07:58:44 PM
by franco

+
+
+
+
+ + +
+
+
+
+
+
+ + + Postfix/rspamd DKIM signing + +
+

+ Started by bubbel +

+ Pages1 2 +
+
+
+

Replies: 19
Views: 18,877

+
+
+

July 16, 2026, 06:43:02 PM
by putt1ck

+
+
+
+
+ + +
+
+
+
+
+
+ + + NUT is Broken After Udating to 26.1.8_5 + +
+

+ Started by kiekar +

+ Pages1 2 +
+
+
+

Replies: 19
Views: 2,231

+
+
+

July 16, 2026, 06:23:09 PM
by Patrick M. Hausen

+
+
+
+
+ + +
+
+
+
+
+
+ + + System memory cache is consistently at zero + +
+

+ Started by camellia +

+ +
+
+
+

Replies: 4
Views: 307

+
+
+

July 16, 2026, 04:35:53 PM
by sopex

+
+
+
+
+ + +
+
+ +
+
+

Replies: 7
Views: 230

+
+
+

July 16, 2026, 11:51:08 AM
by lmoore

+
+
+
+
+ + +
+
+
+
+
+
+ + + Discussion: Sanity checking URL aliases + +
+

+ Started by pfry +

+ +
+
+
+

Replies: 0
Views: 80

+
+
+

July 15, 2026, 11:06:51 PM
by pfry

+
+
+
+
+ + +
+
+ +
+
+

Replies: 4
Views: 176

+
+
+

July 14, 2026, 08:08:10 PM
by patient0

+
+
+
+
+ + +
+
+ +
+
+

Replies: 3
Views: 210

+
+
+

July 14, 2026, 08:54:24 AM
by bimbar

+
+
+
+
+ + +
+
+
+
+
+
+ + + Help with SSH and terminal modes for cli and vi + +
+

+ Started by ciaduck +

+ +
+
+
+

Replies: 8
Views: 330

+
+
+

July 13, 2026, 11:02:58 AM
by lmoore

+
+
+
+
+ + +
+
+
+
+
+
+ + + Unbound forwarding local FQDN + +
+

+ Started by tbk49 +

+ +
+
+
+

Replies: 3
Views: 192

+
+
+

July 12, 2026, 07:25:22 PM
by nero355

+
+
+
+
+ + +
+
+ +
+
+

Replies: 12
Views: 2,689

+
+
+

July 11, 2026, 11:35:21 PM
by nero355

+
+
+
+
+
+ + +
+ Go Up + Pages1 2 3 ... 471 +
+
+
+ +
+ +
+
+

+

+ Poll
+ Moved Topic
+

+

+ Locked Topic
+ Sticky Topic
+ Topic you are watching
+

+ +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/tests/fixtures/smf-thread.html b/tests/fixtures/smf-thread.html new file mode 100644 index 0000000..086b2e9 --- /dev/null +++ b/tests/fixtures/smf-thread.html @@ -0,0 +1,599 @@ + + + + + + + + + + + + + + + Recovery with a backup file + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ + + + +
+
+
+
+ + + + + +
+ +
+
+
+ +
+
+ +
+
+
+
+

+ Recovery with a backup file +

+

Started by tdalej, July 18, 2026, 05:51:14 PM

+ Previous topic - Next topic +
+ +
+ +
+ + Print +
+ +
+ Go Down + Pages1 +
+
+
+ User actions + +
+
+
+
+ +
+
+
+
+ +
+ +
+
+
+ +
+ + + + July 18, 2026, 05:51:14 PM + + + + + +
+
+
+
+
+ I'm doing some poking, and I have to identical servers for firewalls -- MAC addresses are different of course.
Last time I did a recovery from one to the other, restoring the backup caused all sort of fun because the MAC addresses don't align.

Is it an accepted practice to edit the backup to alter the MAC addresses first?
Is there a more elegant way to handle the MAC address changes that I'm just missing?

+
+
+
+
+
+
+
+
+
+
+
+ + +
+
+
+
+ +
+ +
+
+
+ +
+ + + + July 18, 2026, 06:09:53 PM + + #1 + + + +
+
+
+
+
+ Have a look at this topic : https://forum.opnsense.org/index.php?topic=51075.msg261489#msg261489

Maybe it applies to your situation :) +
+
+
+
+
+
+ Weird guy who likes everything Linux and *BSD on PC/Laptop/Tablet/Mobile and funny little ARM based boards :) +
+
+
+
+
+
+
+ + +
+
+
+
+ +
+ +
+
+
+ +
+ + + + July 18, 2026, 06:34:04 PM + + #2 + + + +
+
+
+
+
+ Sort of. 

I'm trying to the recovery with the importer script to see how it gets handled there compared to manual restore.

I can't get the importer to mount the second USB device -- both the boot USB device and the second fat formatted USB device show up as da0 and da1, an it fails to mount with an "invalid device" message. 



+
+
+
+
+
+
+
+
+
+
+
+ + +
+
+
+
+ +
+ +
+
+
+ +
+ + + + July 18, 2026, 06:59:31 PM + + #3 + + + +
+
+
+
+
+ Got the mount solved. 

I have never used the importer script before, always just bulled through manually transferring configurations when needed.
In this case I have two identical Supermicro X10SLH-N6-ST031F (that used to be used for separate locations) that I am going to use one as primary and one as backup.

+
+
+
+
+
+
+
+
+
+
+
+
+
+ +
+ + Print +
+ +
+ Go Up + Pages1 +
+
+ User actions + +
+
+
+ +
+
+ +
+
+ + +
+
+
+
+ + + + \ No newline at end of file diff --git a/tests/fixtures/sparse-login.html b/tests/fixtures/sparse-login.html new file mode 100644 index 0000000..aa88cfd --- /dev/null +++ b/tests/fixtures/sparse-login.html @@ -0,0 +1,37 @@ + + + + + Log in - ExampleApp + + +
+ +
+
+

Log in to your account

+
+ + + + + +
+

Don't have an account? Sign up

+

Forgot your password?

+

Continue with single sign-on

+
+ + + diff --git a/tests/integration/dismiss-overlay-selectors.test.ts b/tests/integration/dismiss-overlay-selectors.test.ts new file mode 100644 index 0000000..f0e0549 --- /dev/null +++ b/tests/integration/dismiss-overlay-selectors.test.ts @@ -0,0 +1,41 @@ +/** + * Real-browser proof that `OVERLAY_DISMISS_SELECTORS` (FIX 2) matches only a + * genuine exact-text consent control, never a same-substring decoy button + * ("Book"/"Cookie"/"Token"/an aria-label containing "close") that a prior + * `:has-text()`/`[attr*= i]` substring match would have hit. Runs under Node + * with a real headless Chromium. + */ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { resolveConfig } from "../../src/agent/config.js"; +import { OVERLAY_DISMISS_SELECTORS } from "../../src/actions/robust-click.js"; +import { SessionManager } from "../../src/session/manager.js"; + +const PAGE = + "" + + "" + + "" + + "" + + ""; +const URL = `data:text/html,${encodeURIComponent(PAGE)}`; + +test("overlay-dismiss selectors match only the real consent button, never a substring decoy", { timeout: 120_000 }, async () => { + const sessions = new SessionManager(); + const session = await sessions.open(resolveConfig({ headless: true, engine: "patchright" })); + try { + await session.page.goto(URL, { waitUntil: "domcontentloaded", timeout: 30_000 }); + // `:text-is()` is a Playwright-only pseudo-class (not real CSS), so this + // must be resolved through Playwright's own selector engine + // (`page.locator`), never the native `Element.matches()` — that throws + // "not a valid selector" on a real DOM node. + const matches = await session.page.locator(OVERLAY_DISMISS_SELECTORS).all(); + const texts = await Promise.all(matches.map((m) => m.textContent())); + assert.deepEqual( + texts.map((t) => t?.trim()), + ["Accept all"], + "must match exactly the real consent button — none of the Book/Cookie/Token/disclose decoys", + ); + } finally { + await sessions.close(session.id); + } +}); diff --git a/tests/integration/fetch-extract.test.ts b/tests/integration/fetch-extract.test.ts new file mode 100644 index 0000000..9fa3ba7 --- /dev/null +++ b/tests/integration/fetch-extract.test.ts @@ -0,0 +1,53 @@ +/** + * Regression proof for the SMF hollow-extraction bug: Defuddle's `removeExactSelectors` + * step deletes the bare `
` wrapping every forum post, silently dropping the + * post bodies. Runs the REAL markdown pipeline (`renderFetch`) against saved fixtures. + */ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import { test } from "node:test"; +import { renderFetch } from "../../src/agent/fetch-render.js"; +import { htmlToText } from "../../src/net/fetch-fast.js"; + +const FIXTURES = fileURLToPath(new URL("../fixtures/", import.meta.url)); +const smfThread = readFileSync(`${FIXTURES}smf-thread.html`, "utf-8"); +const smfBoard = readFileSync(`${FIXTURES}smf-board.html`, "utf-8"); +const sparseLogin = readFileSync(`${FIXTURES}sparse-login.html`, "utf-8"); + +function bodyOf(html: string, url: string) { + return { status: 200, url, html, text: htmlToText(html), isHtml: true, escalated: false }; +} + +test("SMF thread: hollow extraction is auto-recovered", async () => { + const rendered = await renderFetch(bodyOf(smfThread, "https://forum.opnsense.org/index.php?topic=52453.0")); + assert.equal(rendered.extraction, "recovered"); + for (const phrase of ["MAC addresses", "importer script", "Supermicro"]) { + assert.ok(rendered.text.includes(phrase), `expected recovered text to contain "${phrase}"`); + } + assert.ok((rendered.wordCount ?? 0) < 100, "defuddle's own wordCount stays low (it's what triggered recovery)"); +}); + +test("SMF board index: NOT recovered — no link-soup regression", async () => { + const rendered = await renderFetch(bodyOf(smfBoard, "https://forum.opnsense.org/index.php?board=1.0")); + assert.equal(rendered.extraction, "primary", "a link-heavy listing must never be dumped as raw link soup"); + assert.ok(!/\t{2,}/.test(rendered.text), "output should not balloon into raw tab-soup"); +}); + +test("Sparse login page: NOT recovered — short legit page stays primary", async () => { + const rendered = await renderFetch(bodyOf(sparseLogin, "https://example.com/login")); + assert.equal(rendered.extraction, "primary", "a genuinely sparse login page must never be recovered"); + assert.ok(!/\t{2,}/.test(rendered.text), "output should not balloon into raw tab-soup"); +}); + +test("synthetic article: primary extraction, byte-identical to pre-recovery output", async () => { + const html = + 'Hello World' + + "

Hello World

" + + '

This is the main content with a [special] "quote" inside.

' + + ""; + const rendered = await renderFetch(bodyOf(html, "https://x.com/post")); + assert.equal(rendered.extraction, "primary"); + assert.ok(rendered.text.includes("main content")); + assert.ok(!rendered.text.includes("Menu Home About")); +}); diff --git a/tests/integration/mcp.test.ts b/tests/integration/mcp.test.ts index 61c1e9d..75a587c 100644 --- a/tests/integration/mcp.test.ts +++ b/tests/integration/mcp.test.ts @@ -35,6 +35,7 @@ const EXPECTED = [ "browser_fill", "browser_scroll", "browser_press", + "browser_type", "browser_select", "browser_back", "browser_forward", diff --git a/tests/unit/act-by-ref-gating.test.ts b/tests/unit/act-by-ref-gating.test.ts new file mode 100644 index 0000000..ccee2bb --- /dev/null +++ b/tests/unit/act-by-ref-gating.test.ts @@ -0,0 +1,53 @@ +import { describe, expect, test } from "bun:test"; +import type { Locator, Page } from "playwright"; +import { actByRef } from "../../src/actions/act-by-ref.js"; + +/** A ref-resolved locator stub: fixed count, an `evaluate` result queue, and overrides. */ +function makeLocator(evalQueue: unknown[], extra: Record = {}): Locator { + let i = 0; + const self = { first: () => self, count: async () => 1, evaluate: async () => evalQueue[i++], ...extra }; + return self as unknown as Locator; +} + +function makePage(locator: Locator): Page { + return { frames: () => [{ locator: () => locator }] } as unknown as Page; +} + +describe("actByRef — regression guardrails + new gates", () => { + test("native or button (regression guardrail)", async () => { + const el = { getAttribute: () => null, parentElement: null }; + const locator = { evaluate: async (fn: (e: typeof el) => unknown) => fn(el) } as unknown as Locator; + expect(await isComboboxTrigger(locator)).toBe(false); + }); + + test("false when evaluate throws (detached element)", async () => { + const locator = { + evaluate: async () => { + throw new Error("detached"); + }, + } as unknown as Locator; + expect(await isComboboxTrigger(locator)).toBe(false); + }); +}); + +/** A locator stub recording click()/waitFor()/textContent() invocations. */ +function makeOptionLocator(text: string): Locator { + const self = { + first: () => self, + count: async () => 1, + waitFor: async () => {}, + textContent: async () => text, + click: async () => {}, + }; + return self as unknown as Locator; +} + +describe("openComboboxAndPick", () => { + test("opens the trigger via robustClick rung 1 only, waits for the listbox, and clicks the matching option", async () => { + const clicks: string[] = []; + const trigger = { + click: async () => void clicks.push("trigger"), + fill: async () => {}, + pressSequentially: async () => {}, + } as unknown as Locator; + const listbox = { first: () => listbox, waitFor: async () => {} }; + const option = makeOptionLocator("Zurich"); + const page = { + getByRole: (role: string) => (role === "listbox" ? listbox : option), + } as unknown as Page; + const result = await openComboboxAndPick(page, trigger, "Zurich"); + expect(result.ok).toBe(true); + expect(result.strategy).toBe("combobox"); + expect(result.rung).toBe("direct"); + expect(clicks).toEqual(["trigger"]); + }); + + test("reports a single-line error when the entire robust-click ladder is exhausted (sticky/obscured trigger)", async () => { + const fail = async () => { + throw new Error("Element is not visible\nstack"); + }; + const trigger = { click: fail, scrollIntoViewIfNeeded: fail, boundingBox: fail } as unknown as Locator; + const page = { locator: () => ({ first: () => ({ count: async () => 0 }) }) } as unknown as Page; + const result = await openComboboxAndPick(page, trigger, "Zurich"); + expect(result.ok).toBe(false); + expect(result.error).toBe("Error: Element is not visible"); + }); +}); diff --git a/tests/unit/fetch-escalate.test.ts b/tests/unit/fetch-escalate.test.ts new file mode 100644 index 0000000..9413ae9 --- /dev/null +++ b/tests/unit/fetch-escalate.test.ts @@ -0,0 +1,85 @@ +import { describe, expect, test } from "bun:test"; +import { isEmptyExtraction, recoverFromRawText, shouldEscalateEmptyExtraction } from "../../src/agent/fetch-escalate.js"; +import type { RenderedFetch } from "../../src/agent/fetch-render.js"; + +const SPA_SHELL = '
'; +const SCRIPT_HEAVY_SHELL = "loading…"; +const CONTENT_PAGE = `

${"Lorem ipsum dolor sit amet ".repeat(40)}

`; + +/** Minimal `RenderedFetch` fixture — only the fields the gate reads matter. */ +function rendered(wordCount: number | undefined, extraction?: "primary" | "recovered"): RenderedFetch { + return { status: 200, url: "https://x.test", format: "markdown", escalated: false, text: "", wordCount, extraction }; +} + +describe("shouldEscalateEmptyExtraction — post-extraction empty-SPA gate", () => { + test("script-heavy shell + empty extraction + browserFallback → escalate", () => { + expect(shouldEscalateEmptyExtraction(true, false, rendered(0, "primary"), SCRIPT_HEAVY_SHELL)).toBe(true); + }); + + test("SPA-root shell + near-empty (nonzero) extraction → escalate", () => { + expect(shouldEscalateEmptyExtraction(true, false, rendered(5, "primary"), SPA_SHELL)).toBe(true); + }); + + test("content page (no JS markers) with empty-ish text → do NOT escalate", () => { + expect(shouldEscalateEmptyExtraction(true, false, rendered(3, "primary"), CONTENT_PAGE)).toBe(false); + }); + + test("word count already above the floor → do NOT escalate", () => { + expect(shouldEscalateEmptyExtraction(true, false, rendered(200, "primary"), SPA_SHELL)).toBe(false); + }); + + test("browserFallback off → do NOT escalate even on a hollow shell", () => { + expect(shouldEscalateEmptyExtraction(false, false, rendered(0, "primary"), SPA_SHELL)).toBe(false); + }); + + test("already escalated (loop guard) → do NOT escalate twice", () => { + expect(shouldEscalateEmptyExtraction(true, true, rendered(0, "primary"), SPA_SHELL)).toBe(false); + }); + + test("raw-text recovery already found real content → do NOT escalate", () => { + expect(shouldEscalateEmptyExtraction(true, false, rendered(5, "recovered"), SPA_SHELL)).toBe(false); + }); + + test("text-format render (wordCount undefined) → do NOT escalate", () => { + expect(shouldEscalateEmptyExtraction(true, false, rendered(undefined), SPA_SHELL)).toBe(false); + }); +}); + +describe("isEmptyExtraction", () => { + test("zero word count, primary → empty", () => { + expect(isEmptyExtraction(rendered(0, "primary"))).toBe(true); + }); + + test("word count above floor → not empty", () => { + expect(isEmptyExtraction(rendered(200, "primary"))).toBe(false); + }); + + test("recovered extraction → not empty (already fixed)", () => { + expect(isEmptyExtraction(rendered(3, "recovered"))).toBe(false); + }); + + test("wordCount undefined (text format) → not empty", () => { + expect(isEmptyExtraction(rendered(undefined))).toBe(false); + }); +}); + +describe("recoverFromRawText — last-resort fallback after a real browser render is still empty", () => { + test("ships the raw text, computes a real word count, marks extraction recovered", () => { + const before = rendered(0, "primary"); + const after = recoverFromRawText(before, "Chargeur de secours CHF27 avant CHF45.70", 20_000); + expect(after.text).toBe("Chargeur de secours CHF27 avant CHF45.70"); + expect(after.wordCount).toBe(6); + expect(after.extraction).toBe("recovered"); + expect(after.escalated).toBe(before.escalated); // other fields untouched + }); + + test("truncates to maxChars like renderFetch does", () => { + const after = recoverFromRawText(rendered(0, "primary"), "a".repeat(100), 10); + expect(after.text).toHaveLength(10); + }); + + test("empty raw text → wordCount 0, not a crash", () => { + const after = recoverFromRawText(rendered(0, "primary"), " ", 20_000); + expect(after.wordCount).toBe(0); + }); +}); diff --git a/tests/unit/fetch-fast.test.ts b/tests/unit/fetch-fast.test.ts index 00e6feb..7a3ee83 100644 --- a/tests/unit/fetch-fast.test.ts +++ b/tests/unit/fetch-fast.test.ts @@ -27,6 +27,21 @@ describe("htmlToText", () => { test("fragment page with no / still yields its text", () => { expect(htmlToText("

Title

BODY CONTENT
")).toContain("BODY CONTENT"); }); + + test("strips ' + + "

Real page copy.

"; + const t = htmlToText(html); + expect(t).toContain("Real page copy."); + expect(t).not.toContain("color:#161718FF"); + expect(t).not.toContain("should not appear"); + }); }); describe("isHtmlContentType", () => { diff --git a/tests/unit/fill-range.test.ts b/tests/unit/fill-range.test.ts new file mode 100644 index 0000000..b516cf5 --- /dev/null +++ b/tests/unit/fill-range.test.ts @@ -0,0 +1,121 @@ +import { describe, expect, test } from "bun:test"; +import type { Locator, Page } from "playwright"; +import { clampToStep, fillRange, isRangeInput, sliderKind } from "../../src/actions/fill-range.js"; + +describe("isRangeInput", () => { + test("false for a plain element (regression guardrail: text inputs untouched)", async () => { + const locator = { evaluate: async (fn: (el: unknown) => unknown) => fn({}) } as unknown as Locator; + expect(await isRangeInput(locator)).toBe(false); + }); + + test("false when evaluate throws (detached element)", async () => { + const locator = { + evaluate: async () => { + throw new Error("detached"); + }, + } as unknown as Locator; + expect(await isRangeInput(locator)).toBe(false); + }); +}); + +describe("clampToStep (pure snap/clamp arithmetic, no DOM)", () => { + test("snaps to the nearest step within bounds", () => { + expect(clampToStep(0, 100, 10, 47)).toBe(50); + expect(clampToStep(0, 100, 10, 44)).toBe(40); + }); + + test("clamps above max down to max", () => { + expect(clampToStep(0, 50, 5, 999)).toBe(50); + }); + + test("clamps below min up to min", () => { + expect(clampToStep(10, 100, 5, -20)).toBe(10); + }); + + test("falls back to min for a non-finite requested value", () => { + expect(clampToStep(5, 100, 1, Number.NaN)).toBe(5); + }); + + test("defaults a zero/falsy step to 1", () => { + expect(clampToStep(0, 10, 0, 7)).toBe(7); + }); +}); + +describe("sliderKind (single detection pass, never re-detected by fillRange)", () => { + test("'range' for a native range input, without touching getAttribute", async () => { + let getAttributeCalled = false; + const locator = { + evaluate: async () => true, + getAttribute: async () => { + getAttributeCalled = true; + return null; + }, + } as unknown as Locator; + expect(await sliderKind(locator)).toBe("range"); + expect(getAttributeCalled).toBe(false); + }); + + test("'aria' for a role=slider widget", async () => { + const locator = { evaluate: async () => false, getAttribute: async () => "slider" } as unknown as Locator; + expect(await sliderKind(locator)).toBe("aria"); + }); + + test("null for a plain element (no getAttribute at all — regression guardrail)", async () => { + const locator = { evaluate: async () => false } as unknown as Locator; + expect(await sliderKind(locator)).toBeNull(); + }); +}); + +describe("fillRange — keyboard-first success path (no native-setter fallback needed)", () => { + test("native range: keyboard step moves the value, read back via evaluate", async () => { + const evalQueue = [{ min: 0, max: 100, step: 5, current: 40 }, "45"]; + let i = 0; + const presses: string[] = []; + const locator = { + focus: async () => {}, + evaluate: async () => evalQueue[i++], + } as unknown as Locator; + const page = { keyboard: { press: async (key: string) => void presses.push(key) } } as unknown as Page; + const result = await fillRange(page, locator, "47", "range"); + expect(presses).toEqual(["ArrowRight"]); + expect(result).toEqual({ value: "45", reached: true }); + }); + + test("role=slider: keyboard step moves aria-valuenow, read back via getAttribute", async () => { + let now = 5; + const presses: string[] = []; + const locator = { + focus: async () => {}, + getAttribute: async (name: string) => { + if (name === "aria-valuemin") return "0"; + if (name === "aria-valuemax") return "10"; + if (name === "aria-valuenow") { + const v = now; + now = 8; + return String(v); + } + return null; + }, + } as unknown as Locator; + const page = { keyboard: { press: async (key: string) => void presses.push(key) } } as unknown as Page; + const result = await fillRange(page, locator, "8", "aria"); + expect(presses).toEqual(["ArrowRight", "ArrowRight", "ArrowRight"]); + expect(result).toEqual({ value: "8", reached: true }); + }); +}); + +describe("fillRange — FIX 4-CLOSE: float-step rounding on the native-setter fallback path", () => { + test('step=0.1, requested "0.3" clamps to a float-noisy target; native-setter read-back "0.3" → reached:true (fails on the old `Number(finalValue) === target` strict-equality code)', async () => { + const evalQueue: unknown[] = [{ min: 0, max: 1, step: 0.1, current: 0.2 }, "0.3"]; + let i = 0; + const locator = { + focus: async () => { + throw new Error("no focus"); // forces stepSliderByKeyboard -> null -> native-setter fallback + }, + evaluate: async () => evalQueue[i++], + } as unknown as Locator; + const page = { keyboard: { press: async () => {} } } as unknown as Page; + const result = await fillRange(page, locator, "0.3", "range"); + expect(result).toEqual({ value: "0.3", reached: true }); + }); +}); diff --git a/tests/unit/hit-test.test.ts b/tests/unit/hit-test.test.ts new file mode 100644 index 0000000..9feb3d9 --- /dev/null +++ b/tests/unit/hit-test.test.ts @@ -0,0 +1,85 @@ +import { describe, expect, test } from "bun:test"; +import type { Locator } from "playwright"; +import { isTopElement } from "../../src/actions/hit-test.js"; + +/** Minimal fake DOM node matching hit-test.ts's duck-typed `TopHit` shape. */ +interface FakeNode { + nodeType: number; + host?: FakeNode; + contains(other: FakeNode | null): boolean; + getRootNode?(): FakeNode | null; +} + +/** A plain light-DOM node: `nodeType` 1 (`ELEMENT_NODE`), no ancestor/descendant relation by default. */ +function node(overrides: Partial = {}): FakeNode { + return { nodeType: 1, contains: () => false, ...overrides }; +} + +/** + * A locator stub whose `evaluate` genuinely RUNS `isTopElement`'s in-page + * callback against `fakeSelf`, with `globalThis.document` stubbed to serve + * `stack` via BOTH `elementsFromPoint` (current predicate) AND the singular + * `elementFromPoint` (the pre-fix predicate) — so a test run against the OLD + * source genuinely exercises its real (buggy) logic instead of just throwing + * on a missing API and defaulting "true" for the wrong reason. Exercises the + * real predicate, unlike `robust-click.test.ts`'s `evaluate: async () => + * true/false` stubs (those only test `robustClick`'s USE of the boolean). + */ +function makeLocator(fakeSelf: FakeNode, stack: FakeNode[]): Locator { + return { + evaluate: async (fn: (el: unknown, args: [number, number]) => unknown) => { + const g = globalThis as unknown as { document?: unknown }; + const prev = g.document; + g.document = { elementsFromPoint: () => stack, elementFromPoint: () => stack[0] ?? null }; + try { + return await fn(fakeSelf, [0, 0]); + } finally { + g.document = prev; + } + }, + } as unknown as Locator; +} + +describe("isTopElement — FIX 3-CLOSE (shadow-DOM / cross-frame false negatives)", () => { + test("open shadow-DOM element: elementsFromPoint retargets to the host — must ALLOW (fails on the old single-elementFromPoint + plain Node.contains predicate)", async () => { + const host = node(); + const shadowRoot: FakeNode = { nodeType: 11, host, contains: () => false }; + const button = node({ getRootNode: () => shadowRoot }); + const locator = makeLocator(button, [host]); + expect(await isTopElement(locator, 5, 5)).toBe(true); + }); + + test("nested shadow roots: the host chain is walked more than one level up", async () => { + const outerHost = node(); + const outerRoot: FakeNode = { nodeType: 11, host: outerHost, contains: () => false }; + const innerHost = node({ getRootNode: () => outerRoot }); + const innerRoot: FakeNode = { nodeType: 11, host: innerHost, contains: () => false }; + const button = node({ getRootNode: () => innerRoot }); + const locator = makeLocator(button, [outerHost]); + expect(await isTopElement(locator, 5, 5)).toBe(true); + }); + + test("genuine foreign overlay covering the point, same frame (no shadow root at all) — still REFUSE", async () => { + const overlay = node(); + const button = node({ getRootNode: () => node({ nodeType: 9 }) }); // Document, not a ShadowRoot + const locator = makeLocator(button, [overlay]); + expect(await isTopElement(locator, 5, 5)).toBe(false); + }); + + test("cross-frame locator (isMainFrame:false): hit-test is skipped entirely, evaluate never runs", async () => { + let evaluated = false; + const locator = { + evaluate: async () => { + evaluated = true; + return false; + }, + } as unknown as Locator; + expect(await isTopElement(locator, 5, 5, false)).toBe(true); + expect(evaluated).toBe(false); + }); + + test("empty hit stack (nothing at that point) defaults to allow, same as a probe failure", async () => { + const locator = makeLocator(node(), []); + expect(await isTopElement(locator, 5, 5)).toBe(true); + }); +}); diff --git a/tests/unit/navigation-type.test.ts b/tests/unit/navigation-type.test.ts new file mode 100644 index 0000000..736d5df --- /dev/null +++ b/tests/unit/navigation-type.test.ts @@ -0,0 +1,25 @@ +import { describe, expect, test } from "bun:test"; +import type { Page } from "playwright"; +import { typeText } from "../../src/actions/navigation.js"; + +describe("typeText", () => { + test("types into whatever currently has focus (ref-less) and tags the result", async () => { + const typed: Array<{ text: string; opts?: unknown }> = []; + const page = { + keyboard: { type: async (text: string, opts?: unknown) => void typed.push({ text, opts }) }, + } as unknown as Page; + expect(await typeText(page, "iphone")).toEqual({ type: "type", ok: true, text: "iphone" }); + expect(typed).toEqual([{ text: "iphone", opts: { delay: 20 } }]); + }); + + test("captures a single-line error on failure", async () => { + const page = { + keyboard: { + type: async () => { + throw new Error("Target closed\nstack"); + }, + }, + } as unknown as Page; + expect(await typeText(page, "x")).toEqual({ type: "type", ok: false, text: "x", error: "Error: Target closed" }); + }); +}); diff --git a/tests/unit/recover-choose.test.ts b/tests/unit/recover-choose.test.ts new file mode 100644 index 0000000..c1cb1da --- /dev/null +++ b/tests/unit/recover-choose.test.ts @@ -0,0 +1,50 @@ +import { describe, expect, test } from "bun:test"; +import { chooseRecovery, MIN_PROSE_CHARS, nonLinkProseLen } from "../../src/extraction/serialize/recover.js"; + +const PROSE_HTML = `

${"Real prose sentence about a topic. ".repeat(30)}

`; +const LINK_SOUP_HTML = `${Array.from({ length: 40 }, (_, i) => `Topic ${i}`).join(" ")}`; +const SHORT_SPARSE_HTML = + '

Log in.

'; + +describe("chooseRecovery", () => { + test("non-hollow input stays primary (byte-identical c1)", () => { + const result = chooseRecovery({ + html: SHORT_SPARSE_HTML, + c1Text: "Log in.", + wordCount: 5, + rawText: "A B Log in.", + }); + expect(result).toEqual({ text: "Log in.", extraction: "primary" }); + }); + + test("hollow + substantial recoverable prose recovers the raw text", () => { + const raw = "Real prose sentence about a topic. ".repeat(30); + const result = chooseRecovery({ html: PROSE_HTML, c1Text: "stub", wordCount: 2, rawText: raw }); + expect(result.extraction).toBe("recovered"); + expect(result.text).toContain("Real prose sentence about a topic."); + }); + + test("hollow + link-heavy refuses recovery (stays primary, no link soup)", () => { + const raw = Array.from({ length: 40 }, (_, i) => `Topic ${i}`).join(" ").repeat(10); + const result = chooseRecovery({ html: LINK_SOUP_HTML, c1Text: "stub", wordCount: 2, rawText: raw }); + expect(result).toEqual({ text: "stub", extraction: "primary" }); + }); + + test("zero word count with substantial prose still stays primary (chrome-heavy SPA shell)", () => { + // Regression proof: a heavy client-rendered page (e.g. a Skoda stock-listing + // SPA) can carry thousands of chars of persistent nav/footer chrome as + // non-link "prose" while Defuddle finds literally no content container + // (wordCount 0). Recovering raw text there dumped page chrome (and, before + // the htmlToText script/style fix, raw CSS) instead of real content. + const result = chooseRecovery({ html: PROSE_HTML, c1Text: "", wordCount: 0, rawText: "raw chrome text" }); + expect(result).toEqual({ text: "", extraction: "primary" }); + }); + + test("hollow but below the recoverable-prose floor stays primary (short/sparse page)", () => { + // proseLen(SHORT_SPARSE_HTML) is well under MIN_PROSE_CHARS — a login/404-style + // page must never be dumped as raw text just because defuddle's stub is tiny. + expect(nonLinkProseLen(SHORT_SPARSE_HTML)).toBeLessThan(MIN_PROSE_CHARS); + const result = chooseRecovery({ html: SHORT_SPARSE_HTML, c1Text: "stub", wordCount: 2, rawText: "A B Log in." }); + expect(result).toEqual({ text: "stub", extraction: "primary" }); + }); +}); diff --git a/tests/unit/recover.test.ts b/tests/unit/recover.test.ts new file mode 100644 index 0000000..1cc277c --- /dev/null +++ b/tests/unit/recover.test.ts @@ -0,0 +1,62 @@ +import { describe, expect, test } from "bun:test"; +import { + HOLLOW_MAX_WORDS, + isHollow, + nonLinkProseLen, + normalizeWhitespace, +} from "../../src/extraction/serialize/recover.js"; + +const PROSE_HTML = `

${"Real prose sentence about a topic. ".repeat(30)}

`; +const LINK_SOUP_HTML = `${Array.from({ length: 40 }, (_, i) => `Topic ${i}`).join(" ")}`; +// JS string literal containing a bare `` between two real anchors — must not +// make the anchor regex span across the whole script into unrelated markup. +const SCRIPT_WITH_FAKE_CLOSE_TAG_HTML = + 'X

Some short prose here.

'; + +describe("nonLinkProseLen", () => { + test("prose-only page: non-link prose ~= visible text (no anchors)", () => { + expect(nonLinkProseLen(PROSE_HTML)).toBeGreaterThan(900); + }); + + test("link-soup page has ~zero non-link prose (only inter-anchor whitespace)", () => { + expect(nonLinkProseLen(LINK_SOUP_HTML)).toBeLessThan(50); + }); + + test("empty html has zero non-link prose (no divide-by-zero)", () => { + expect(nonLinkProseLen("")).toBe(0); + }); + + test("a JS string literal '' does not corrupt the anchor scan", () => { + // Without stripping