Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Changelog

## [0.1.63] - 19-07-2026

### Fixed

- **`browser_fetch`/`browser_fetch_batch` `browserFallback` never escalating chrome-heavy SPAs** — `isThinShell` gated escalation on raw-HTML visible-text length (`< 600` chars); client-rendered pages with a lot of nav/footer chrome (e.g. `digitec.ch`, `fr.skoda.ch/apps/stock`) cleared that threshold while their real content still only existed after JS hydration, so the fast-path extraction silently returned `wordCount: 0` with `escalated: false`. `browserFallback` now also escalates on the extraction RESULT: when it comes back empty/near-empty (`wordCount < 15`, and not already fixed by hollow-extraction recovery) on HTML carrying JS-rendering markers (SPA mount point or heavy `<script>` count — `looksJsRendered`, `src/net/thin-shell.ts`), the fetch retries through a real browser render and re-extracts. Escalates at most once per fetch (loop guard). A further last-resort fallback (`recoverFromRawText`) covers the rare case where Defuddle's own content-scoring still finds nothing even on the real browser-rendered HTML (a known Defuddle limitation on dense, non-article layouts like an e-commerce grid homepage — no upstream config fixes it): the browser's own raw visible text is shipped instead of near-empty markdown, with a real word count and `extraction: "recovered"`, at no extra network/browser cost. New shared orchestrator (`src/agent/fetch-orchestrate.ts`) is now the single fetch-and-render path for both `browser_fetch` and `browser_fetch_batch`; server-rendered pages whose fast-path extraction already succeeds are unaffected (byte-identical, no new browser launch).
- **`browser_fetch` silent hollow extraction** — on some pages (e.g. SMF forum threads) Defuddle's own boilerplate-removal deleted the element wrapping every post body, silently returning near-empty markdown (`status:200`, no error, no signal). The markdown path now detects a "hollow" extraction (nonzero-but-low word count and a low prose-capture ratio against the page's real **non-link** prose) and recovers the raw page text instead — gated on an absolute floor of recoverable non-link prose (`MIN_PROSE_CHARS`, tuned against real pages incl. `github.com/login`'s 505-char chrome footprint, not just the saved SMF fixtures), so a genuinely sparse or link-heavy page (listing/index, login, 404) is never dumped as raw link/nav soup, regardless of its link density. A **zero**-word-count Defuddle result (no content container found at all — typical of a heavy client-rendered SPA whose real content never reaches the raw HTML) is never treated as "hollow": that failure mode belongs to `browserFallback`, not raw-text recovery.
- **`browser_fetch`/price extraction raw-CSS leak** — `htmlToText` (the raw-text fallback feeding both hollow-extraction recovery and price extraction) read `document.body.textContent`, which includes `<script>`/`<style>` tag source verbatim; a live CSS-in-JS SSR page injecting critical CSS into `<body>` leaked raw stylesheet source into the "recovered" text. `<script>`/`<style>` elements are now removed from the DOM before reading `textContent`.
- **`browser_act`/`browser_fill` "Malformed value" on `<input type="range">`** — Playwright's `locator.fill()` sets `.value` then round-trip-asserts the raw string against the post-clamp DOM value; a native range input snaps to `min`/`max`/`step`, so the assertion always threw. `smartFill`/`actByRef` now detect a range input (`isRangeInput`) and set it via the native value setter after clamping+snapping (`fillRange`, `src/actions/fill-range.ts`), dispatching `input`/`change` so framework listeners observe it. Plain text/label/placeholder fills are unaffected.
- **`browser_act`/`browser_fill` no-op on ARIA comboboxes** — a click-to-open combobox/listbox only renders its options after a click, so filling it directly silently no-oped. `smartFill`, `actByRef`, and `browser_act`'s `pick` kind now detect a combobox/listbox trigger (`isComboboxTrigger`, checked on the element or up to 2 ancestors) and open the trigger + click the matching option (`openComboboxAndPick`, `src/actions/combobox.ts`) instead.
- **`pick` throwing on human-readable targets** — `browser_act`'s `pick` kind and the `perform`/`pick` action resolved their target via a raw `page.locator(target)`, which throws `Unexpected token "?"` on text like `"Où allez-vous ?"`. `pick` now resolves through the same ordered selector→role→text→label strategies as `browser_click` (`resolveClickTarget`, extracted from `smart-click.ts` into `src/actions/resolve-click-target.ts`), falling back to the raw selector only when nothing else matches.
- **`detectChallenges` missed AWS WAF Bot Control** — `Challenges` now reports `awsWaf` (`chal_t=` query param or `window.awsWafCookieDomainList`), additive and optional in `challengesSchema`.
- **`browser_click`/`browser_act` failing on obscured or sticky elements** — a target that hit-tests as covered (booking.com's `obscured:true` destination field, skoda.ch's sticky combobox panel) or is otherwise unclickable now escalates through a bounded click ladder (`robustClick`, `src/actions/robust-click.ts`): plain click → scroll-into-view → dismiss a covering consent/cookie overlay → force-click → raw mouse click at coordinates, each rung attempted only after the previous one throws or times out. A normally-clickable element still resolves on rung 1, byte-identical to the previous plain `locator.click()`. Point-based hit-testing (`isTopElement`, `src/actions/hit-test.ts`) confirms the true top element before trusting a bypass-actionability click; a probe failure (no DOM access, detached element) always defaults to "clear" so it never blocks a real click.
- **Keyboard-stepped range/slider fills reporting false success** — `<input type=range>` and ARIA `role=slider` widgets that only re-render on a real trusted key event (not a synthetic setter+dispatch) now step via actual keyboard presses in bounded batches with a read-back after each batch (`stepSliderByKeyboard`, `src/actions/slider-keyboard.ts`), so a stalled widget reports an honest undershoot instead of a silent `ok:true` on an unset value. Also fixes a floating-point false negative where a clamped value (e.g. `0.30000000000000004`) read back as the equivalent parsed number (`0.3`) was reported as a mismatch.

### Added

- **`browser_fetch` `wordCount`/`extraction` signal** — every markdown result now reports the extractor's word count and which path produced the text (`"primary"` | `"recovered"`), both optional (unset on the `text`/non-HTML path).
- **`browser_fetch` `contentSelector`** — optional CSS selector to pin the main content container yourself, bypassing auto-detection, for pages where recovery still isn't the right call.
- **`browser_type`** (51st tool) — types text into whatever element currently has focus, no `ref`/`target` needed (drives `page.keyboard.type`, not a locator) — for elements a locator can't reach (e.g. a closed shadow-DOM input): `browser_click` the field first, then `browser_type`.

## [0.1.62] - 17-07-2026

### Fixed
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Shadow DOM + iframes), multi-step plans, structured extraction, visual diff, and
guardrails** for payments and bookings. It drives real Chromium, so it reads **Next.js / SPA**
pages after hydration — not just static HTML.

> 50 MCP tools · stealth + rotating proxies · HTTP fast-path (single, batch & crawl) · full-site content + screenshot snapshots · structured per-card product extraction · form fill + file upload · encrypted credential vault + TOTP (fill-by-reference, origin-bound, never exposed to the LLM) · hover + drag&drop · PDF export · cookies / permissions · network mocking · clipboard · virtualized-list scraping + autoscroll · tabs / dialogs / downloads · console + network logs · MCP screenshot resources · `FUSE_CAPS` tool-group filtering · named auth profiles (cookies + localStorage + IndexedDB, saved at login) · `blockResources` · HAR record/replay · pixel visual-diff · human handoff + live view · typed structured output (`outputSchema` + `structuredContent`).
> 51 MCP tools · stealth + rotating proxies · HTTP fast-path (single, batch & crawl) · full-site content + screenshot snapshots · structured per-card product extraction · form fill + file upload · encrypted credential vault + TOTP (fill-by-reference, origin-bound, never exposed to the LLM) · hover + drag&drop · PDF export · cookies / permissions · network mocking · clipboard · virtualized-list scraping + autoscroll · tabs / dialogs / downloads · console + network logs · MCP screenshot resources · `FUSE_CAPS` tool-group filtering · named auth profiles (cookies + localStorage + IndexedDB, saved at login) · `blockResources` · HAR record/replay · pixel visual-diff · human handoff + live view · typed structured output (`outputSchema` + `structuredContent`).

## Install

Expand Down Expand Up @@ -68,7 +68,7 @@ Full reference in **[`docs/`](./docs/README.md)**:

[Installation](./docs/installation.md) ·
[CLI](./docs/cli.md) ·
[MCP tools (50)](./docs/mcp-tools.md) ·
[MCP tools (51)](./docs/mcp-tools.md) ·
[Configuration](./docs/configuration.md) ·
[Sessions](./docs/sessions.md) ·
[Extraction](./docs/extraction.md) ·
Expand Down
2 changes: 1 addition & 1 deletion biome.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://biomejs.dev/schemas/2.4.16/schema.json",
"files": {
"includes": ["src/**", "tests/**", "!dist/**", "!node_modules/**"]
"includes": ["src/**", "tests/**", "!dist/**", "!node_modules/**", "!tests/fixtures/**"]
},
"formatter": {
"enabled": false
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ New here? Start with the root [README](../README.md), then dive in:
| --- | --- |
| [Installation](./installation.md) | Requirements, install, Chromium, MCP registration, the three ways to get a browser |
| [CLI](./cli.md) | `probe` / `fetch` / `fetch-batch` / `crawl` / `collect-batch` / `shots` / `shots-batch` / `site-shots` / `serp-batch` + one-shot page commands (`run` / `products` / `extract` / `snapshot` / `screenshot` / `inspect`) + every flag |
| [MCP tools](./mcp-tools.md) | All 50 tools with parameters and examples |
| [MCP tools](./mcp-tools.md) | All 51 tools with parameters and examples |
| [Configuration](./configuration.md) | `AgentOptions`, `FUSE_*` env vars, identity, retry, output location |
| [Sessions](./sessions.md) | Session lifecycle, auto crash recovery, `storageState` auto-save, HAR record/replay, CDP attach |
| [Extraction](./extraction.md) | `browser_extract` / `extract_schema` / `collect` + the clean→validate→dedupe→emit pipeline |
Expand Down
26 changes: 23 additions & 3 deletions docs/mcp-tools.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# MCP tools

Complete reference for the 50 `browser_*` tools exposed by the fuse-browser MCP server.
Complete reference for the 51 `browser_*` tools exposed by the fuse-browser MCP server.

Tools fall into two families:

Expand All @@ -18,11 +18,11 @@ Every tool declares an `outputSchema` (zod) and returns a **typed `structuredCon

## Capability groups (`FUSE_CAPS`)

By default all 50 tools are registered. Set the `FUSE_CAPS` env var (comma-separated group names) to expose fewer tools — a lighter context for the LLM client:
By default all 51 tools are registered. Set the `FUSE_CAPS` env var (comma-separated group names) to expose fewer tools — a lighter context for the LLM client:

| Group | Tools |
| --- | --- |
| `core` | Session lifecycle (`browser_open`/`browser_status`/`browser_close`/`browser_connect`), navigation (`browser_navigate`/`browser_back`/`browser_forward`), actions (`browser_click`/`browser_fill`/`browser_login`/`browser_scroll`/`browser_press`/`browser_select`), `browser_tabs`, `browser_dialog`/`browser_downloads`, `browser_snapshot`/`browser_act`, `browser_wait`/`browser_wait_for`, `browser_screenshot`, `browser_autoscroll`, `browser_vault`. |
| `core` | Session lifecycle (`browser_open`/`browser_status`/`browser_close`/`browser_connect`), navigation (`browser_navigate`/`browser_back`/`browser_forward`), actions (`browser_click`/`browser_fill`/`browser_login`/`browser_scroll`/`browser_press`/`browser_type`/`browser_select`), `browser_tabs`, `browser_dialog`/`browser_downloads`, `browser_snapshot`/`browser_act`, `browser_wait`/`browser_wait_for`, `browser_screenshot`, `browser_autoscroll`, `browser_vault`. |
| `batch` | `browser_probe`, `browser_probe_html`, `browser_fetch`, `browser_fetch_batch`, `browser_crawl`, `browser_collect_batch`, `browser_shots_batch`, `browser_site_shots`, `browser_serp_batch`. |
| `extract` | `browser_collect`, `browser_run`, `browser_extract`, `browser_extract_schema`, `browser_products`. |
| `debug` | `browser_inspect`, `browser_console`, `browser_network`, `browser_visual_diff`, `browser_metrics`, `browser_pdf`, `browser_cookies`. |
Expand Down Expand Up @@ -94,10 +94,15 @@ Accepts the full [agentOptionShape](#browser_open) plus the same probe flags as

HTTP fetch with browser TLS/HTTP2 impersonation — no browser launch, ~10x faster. For server-rendered HTML; for JS/SPA pages set `browserFallback: true` (or use `browser_probe`). Non-HTML responses (JSON APIs, `text/plain`) are returned **verbatim** — the markdown/HTML pipeline is skipped — so this also works as a fast JSON-API fetcher. The body download is capped at 10 MB.

With `browserFallback: true`, escalation to a real browser render fires on **either** of two independent signals: (1) the raw HTML itself looks like an unrendered SPA shell (thin visible text + SPA/script markers), or (2) the raw HTML looks JS-rendered (SPA hydration mount point or heavy `<script>` count) **and** the fast-path extraction still came back empty/near-empty (`wordCount < 15`, not already recovered by the hollow-extraction path) — this covers chrome-heavy pages (nav/footer markup clearing 600+ visible chars) whose real content only exists after client-side hydration. Either path sets `escalated: true` and re-renders from the browser's rendered HTML; a body that already escalated is never escalated twice. If the content extractor still finds nothing on the real browser-rendered HTML (rare — a dense, non-article layout like an e-commerce grid homepage can defeat its content-scoring even post-render), the browser's own raw visible text is shipped instead of near-empty markdown, with `extraction: "recovered"` and a real `wordCount` — no extra browser launch.

The markdown path auto-detects a **hollow extraction** — when the content extractor silently captures only a sliver of the page's real non-link prose (e.g. some forum threads whose posts sit inside a wrapper element the extractor drops as boilerplate) — and recovers the raw page text instead, gated so a genuinely sparse or link-heavy page (listing/index, login, 404) never gets dumped as raw link soup: recovery only fires when the page holds a substantial amount of reading content outside its `<a>` chrome that the extractor missed. The result carries `wordCount` and `extraction` (`"primary"` | `"recovered"`) so callers can see which path produced `text`. Pass `contentSelector` to pin the main content container yourself when auto-detection still picks the wrong element.

| Param | Type | Required | Description |
| --- | --- | --- | --- |
| `url` | string | yes | URL to fetch. |
| `format` | enum `markdown` \| `text` | no | Output format (default `markdown`: main content + YAML frontmatter). Forced to raw `text` for non-HTML bodies (JSON, plain text). |
| `contentSelector` | string | no | CSS selector pinning the main content element, bypassing auto-detection. Markdown path only. |
| `extractPrices` | boolean | no | Run the price extractor on the body. |
| `extractContacts` | boolean | no | Extract `{ emails, phones, hasContactForm }` from the fetched HTML (no browser). |
| `contactFilter` | enum `strict` \| `off` | no | Drop placeholder/template emails (default `strict`). |
Expand All @@ -110,6 +115,8 @@ HTTP fetch with browser TLS/HTTP2 impersonation — no browser launch, ~10x fast
{ "url": "https://example.com", "extractContacts": true }
```

Returns `{ status, url, format, escalated, text, wordCount?, extraction? }` — `wordCount`/`extraction` are set on the markdown path only.

---

### browser_fetch_batch
Expand Down Expand Up @@ -521,6 +528,19 @@ Press a key or shortcut (e.g. `Enter`, `ArrowDown`, `Control+a`).
{ "sessionId": "s_abc123", "key": "Enter" }
```

### browser_type

Type text into whatever element currently has focus — no `ref`/`target` (it drives `page.keyboard.type`, not a locator). Use for elements a ref can't reach, e.g. a closed shadow-DOM search input: `browser_click` the field first, then `browser_type`.

| Param | Type | Required | Description |
| --- | --- | --- | --- |
| `sessionId` | string | yes | Target session. |
| `text` | string | yes | Text to type into the focused element. |

```json
{ "sessionId": "s_abc123", "text": "iphone" }
```

### browser_select

Select an option in a `<select>` by value, label, or index.
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@fusengine/browser-mcp",
"version": "0.1.62",
"version": "0.1.63",
"description": "MCP server + CLI giving AI agents a real, stealth browser (Patchright/Playwright) — per-country identity, self-healing actions, snapshots, multi-step plans, structured extraction, CDP attach.",
"license": "MIT",
"author": "Fusengine",
Expand Down Expand Up @@ -55,7 +55,7 @@
"typecheck": "tsc -p tsconfig.json --noEmit",
"lint": "biome check src tests",
"test": "bun test tests/unit",
"test:integration": "node --test --import tsx tests/integration/mcp.test.ts tests/integration/probe.test.ts tests/integration/snapshot.test.ts tests/integration/snapshot-prune.test.ts tests/integration/snapshot-frames.test.ts tests/integration/collect.test.ts tests/integration/collect-batch.test.ts tests/integration/selectors.test.ts tests/integration/visual-diff.test.ts tests/integration/session-state.test.ts tests/integration/pipeline.test.ts tests/integration/run.test.ts tests/integration/extract-schema.test.ts tests/integration/recovery.test.ts tests/integration/live-view.test.ts tests/integration/vault.test.ts",
"test:integration": "node --test --import tsx tests/integration/mcp.test.ts tests/integration/probe.test.ts tests/integration/snapshot.test.ts tests/integration/snapshot-prune.test.ts tests/integration/snapshot-frames.test.ts tests/integration/collect.test.ts tests/integration/collect-batch.test.ts tests/integration/selectors.test.ts tests/integration/visual-diff.test.ts tests/integration/session-state.test.ts tests/integration/pipeline.test.ts tests/integration/run.test.ts tests/integration/extract-schema.test.ts tests/integration/recovery.test.ts tests/integration/live-view.test.ts tests/integration/vault.test.ts tests/integration/fetch-extract.test.ts tests/integration/dismiss-overlay-selectors.test.ts",
"browsers": "patchright install chromium",
"mcp": "node --import tsx src/bin/mcp.ts",
"cli": "node --import tsx src/bin/cli.ts"
Expand Down
Loading