fix(core): stop reading a form Turnstile loader as a bot wall

.changeset/turnstile-form-scope.md

@@ -0,0 +1,5 @@
+---
+"@forkpoint/agent-lighthouse-core": patch
+---
+
+Stop treating a Cloudflare Turnstile loader on a readable page as a bot wall. `agentic-commerce/cart-handoff-reachability` fails only a cart that is a challenge page or only a widget, `access-crawl-control/no-bot-detection` passes a loader on a page the scan could read, and `operability-safety/no-blocking-captcha` matches widget scripts and markup instead of any mention of the vendor name.

docs/evidence/audits/access-crawl-control/no-bot-detection.md

@@ -82,6 +82,18 @@ a pause is the only way to get a verdict from a rate-limited origin.
 
 ## Implementation deviations
 
+### A loader on a readable page passes (2026-10-06)
+
+The page-script branch warned at weight 1 on any page whose HTML contained a
+challenge host, including `challenges.cloudflare.com`. The counter-evidence
+grades that sub-signal D, and the required fix says to warn only when the
+widget gates the main content. A loader is now reported only on a page that
+served no readable text, using the per-page `renderedByPage` verdict from the
+scan evidence (`pageRendersText` when it is absent). A loader on a page the
+scan read is named in a pass, because no wall was observed and the page was
+read. The `wafProtection` branch, which carries the A grade, is unchanged.
+Script-`src` scoping and a UA probe are still open items from the review above.
+
 - 2026-10-06 — the shared WAF detector no longer treats the
   `challenges.cloudflare.com` hostname as proof of an access wall. A live fetch
   of a storefront returned HTTP 200 with the storefront HTML and a

docs/evidence/audits/agentic-commerce/cart-handoff-reachability.md

@@ -73,6 +73,22 @@ Tier per evidence policy: **scored** — grade B meets the A/B bar required for
 - **Without a platform fingerprint the audit is not applicable, not failing.** It still probes `/cart` and `/checkout`, but when neither answers it reports `notApplicable`: the paths were a guess. With a fingerprint, an unanswered cart path is a failure, because the platform's own convention says where it should be.
 - **The JS-only arm needs both halves.** A document warns only when its visible text is under 200 characters _and_ it carries no `<noscript>` content. A short page with a real fallback passes.
 
+### A challenge widget fails only when it is the document (2026-10-06)
+
+Sketch item (b) fails a cart document that references a challenge widget by
+URL substring. A large retail site served a readable cart with the Turnstile
+loader (`api.js?render=explicit`) included site-wide for its login and
+newsletter forms. The cart was readable, and the audit failed it.
+
+The claimed mechanism is a handoff that dead-ends on a challenge. The
+guidance asks a site to challenge the payment submission rather than the
+document, and a widget on a form of a readable cart does exactly that. The
+audit now fails in two cases. First, when the shared WAF detector
+(`detectWafProtection`, the logic the scan gate uses) reads the cart response as a
+challenge page. Second, when a widget is mounted on a document with under 200
+characters of visible text, the same threshold the JS-only arm uses, so the
+challenge is all there is to read. A widget on a readable cart passes.
+
 ## Deferred
 
 - **The ACP `continue_url` itself is never probed.** Reaching it needs a Cart API session, which needs credentials and a POST. The audit probes the storefront cart the same handoff lands on.

docs/evidence/audits/operability-safety/no-blocking-captcha.md

@@ -95,6 +95,26 @@ _No dedicated evidence signal was researched for this audit in the 2026-08-20 pa
   `text/html`, from the requested host. This audit's verdict on that state is
   unchanged: the wall branch above the guard reports the wall either way.
 
+### Detection reads widget markup, not body text (2026-10-06)
+
+The audit matched `recaptcha`, `hcaptcha` and `turnstile` anywhere in the
+body. A large retail site's inline config (`provider = "CloudflareTurnstile"`)
+matched as a CAPTCHA. Detection now uses the selectors the review asks for: a
+`<script src>` from the vendor's loader path, or the element the widget mounts
+on (`.g-recaptcha`, `.h-captcha`, `.cf-turnstile`). Prose, config and CSP text
+no longer match. A Turnstile, reCAPTCHA or hCaptcha widget on a form still
+warns.
+
+## Deferred
+
+- **Interactive versus invisible variants.** The counter-evidence shows that
+  reCAPTCHA v3, Turnstile's managed and invisible modes and hCaptcha's passive
+  mode do not stop an agent. The markup alone does not say which mode a widget
+  runs in: the mode is usually set in the vendor dashboard or in a script
+  call. Every detected widget still warns.
+- **Form correlation.** The finding names the page, not the form the widget
+  guards.
+
 ## Review history
 
 - 2026-08-20 — code review (11-agent workflow) + evidence research (12-domain workflow, 400 sources).

packages/core/src/audits/access-crawl-control/no-bot-detection.test.ts

@@ -54,6 +54,23 @@ describe("NoBotDetectionAudit", () => {
     expect(result.found).toContain("Cloudflare Turnstile");
   });
 
+  // The dossier grades a form-scoped widget D. A page the scan read was not
+  // walled, so a loader on it is not a finding.
+  it("passes a readable page that loads Turnstile for its forms", () => {
+    const prose = "Browse our collection of jackets, boots and bags. ".repeat(
+      10,
+    );
+    const pages = [
+      mockPageContext(
+        "https://example.com/",
+        `<html><head><script src="https://challenges.cloudflare.com/turnstile/v0/api.js?render=explicit"></script></head><body><main><h1>Shop</h1><p>${prose}</p></main><form action="/newsletter"><input name="email"></form></body></html>`,
+      ),
+    ];
+    const result = audit.audit(mockCheckContext(pages));
+    expect(result.status).toBe("pass");
+    expect(result.message).toContain("Cloudflare Turnstile");
+  });
+
   it("warns when no pages were scanned", () => {
     const ctx = mockCheckContext([]);
     const result = audit.audit(ctx);

packages/core/src/audits/access-crawl-control/no-bot-detection.ts

@@ -2,7 +2,11 @@ import type { AuditMeta, AuditResult } from "../../types";
 import { Audit } from "../../audit";
 import type { CheckContext } from "../../check-context";
 import { weightForGrade } from "../../scorer";
-import { scanReadPageText, unreadPageTextReason } from "../../scan-evidence";
+import {
+  pageRendersText,
+  scanReadPageText,
+  unreadPageTextReason,
+} from "../../scan-evidence";
 
 const BOT_DETECTION_PATTERNS: Array<{
   name: string;
@@ -85,17 +89,26 @@ export class NoBotDetectionAudit extends Audit {
     }
 
     const detectedServices: Map<string, string[]> = new Map();
+    // Loaders found on pages that still served readable text. The scan read
+    // those pages, so the loader gated nothing: the dossier grades a
+    // form-scoped widget D, and only a loader on an unreadable page is reported.
+    const readableServices = new Set<string>();
+    const rendered = ctx.evidence.renderedByPage;
 
     for (const page of ctx.pages) {
       const html = page.fetchResult.body.toLowerCase();
+      const readable = rendered[page.url] ?? pageRendersText(page);
 
       for (const { name: serviceName, pattern } of BOT_DETECTION_PATTERNS) {
-        if (html.includes(pattern.toLowerCase())) {
-          if (!detectedServices.has(serviceName)) {
-            detectedServices.set(serviceName, []);
-          }
-          detectedServices.get(serviceName)!.push(page.url);
+        if (!html.includes(pattern.toLowerCase())) continue;
+        if (readable) {
+          readableServices.add(serviceName);
+          continue;
         }
+        if (!detectedServices.has(serviceName)) {
+          detectedServices.set(serviceName, []);
+        }
+        detectedServices.get(serviceName)!.push(page.url);
       }
     }
 
@@ -115,6 +128,15 @@ export class NoBotDetectionAudit extends Audit {
         );
       }
 
+      if (readableServices.size > 0) {
+        const names = [...readableServices].join(", ");
+        return this.pass(
+          `${names} loads on scanned pages that still served readable text, so it guards forms rather than the content. No bot wall was observed.`,
+          "No JavaScript-based bot challenges that would block legitimate AI agents",
+          `Checked ${ctx.pages.length} page(s) — ${names} on readable pages only`,
+        );
+      }
+
       return this.pass(
         "No aggressive bot-detection scripts found on scanned pages.",
         "No JavaScript-based bot challenges that would block legitimate AI agents",

packages/core/src/audits/agentic-commerce/cart-handoff-reachability.test.ts

@@ -35,6 +35,7 @@ interface Answer {
   status?: number;
   body?: string;
   finalUrl?: string;
+  headers?: Record<string, string>;
 }
 
 interface Store {
@@ -77,6 +78,7 @@ function run(store: Store = {}) {
     );
     result.url = o.url;
     result.finalUrl = answer.finalUrl ?? o.url;
+    Object.assign(result.headers, answer.headers ?? {});
     return result;
   };
 
@@ -127,13 +129,31 @@ describe("CartHandoffReachabilityAudit", () => {
     expect(strings(r, "failures")[0]).toContain("account wall");
   });
 
-  it("fails a checkout document carrying a bot challenge", async () => {
+  // The false positive: a readable cart with the site-wide Turnstile loader
+  // and its inline config. The widget guards forms, not the document.
+  it("passes a readable cart that loads Turnstile for its forms", async () => {
+    const { result } = run({
+      markup: SHOPIFY,
+      paths: {
+        "/cart": {
+          headers: { server: "cloudflare", "cf-ray": "abc" },
+          body: `${CART_BODY}<script src="https://challenges.cloudflare.com/turnstile/v0/api.js?render=explicit"></script><script>app.captcha.provider = "CloudflareTurnstile"; app.captcha.tokenParam = "cf-turnstile-response";</script>`,
+        },
+      },
+    });
+    const r = await result;
+    expect(r.status).toBe("pass");
+    expect(strings(r, "failures")).toEqual([]);
+  });
+
+  // True positive: the widget is the whole document.
+  it("fails a checkout document that is only a bot challenge", async () => {
     const { result } = run({
       markup: '<link href="/wp-content/plugins/woocommerce/style.css">',
       paths: {
         "/cart": {},
         "/checkout": {
-          body: `${CART_BODY}<script src="https://challenges.cloudflare.com/turnstile/v0/api.js"></script>`,
+          body: '<html><body><form action="/verify"><div class="cf-turnstile" data-sitekey="0x0000"></div></form><script src="https://challenges.cloudflare.com/turnstile/v0/api.js"></script></body></html>',
         },
       },
     });
@@ -142,6 +162,22 @@ describe("CartHandoffReachabilityAudit", () => {
     expect(strings(r, "failures")[0]).toContain("Cloudflare Turnstile");
   });
 
+  // True positive: a managed challenge page served at 200 in place of the cart.
+  it("fails a cart answered by a managed challenge page", async () => {
+    const { result } = run({
+      markup: SHOPIFY,
+      paths: {
+        "/cart": {
+          headers: { server: "cloudflare" },
+          body: '<html><head><title>Just a moment...</title></head><body><script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1"></script></body></html>',
+        },
+      },
+    });
+    const r = await result;
+    expect(r.status).toBe("fail");
+    expect(strings(r, "failures")[0]).toContain("Managed Challenge");
+  });
+
   it("fails a 403 under ChatGPT-User and names the user agent", async () => {
     const { result } = run({
       markup: SHOPIFY,

packages/core/src/audits/agentic-commerce/cart-handoff-reachability.ts

@@ -13,6 +13,7 @@ import {
   sharedUaFetch,
 } from "../../gatherers/ua-parity";
 import { parseRobots, isPathAllowed } from "../../gatherers/robots";
+import { detectWafProtection } from "../../waf-detector";
 
 /** Where each storefront keeps its cart. */
 const CANDIDATES: Record<CommercePlatform, string[]> = {
@@ -179,10 +180,27 @@ export class CartHandoffReachabilityAudit extends Audit {
         continue;
       }
 
+      // A challenge page in place of the cart is a wall. The shared detector
+      // owns what a challenge page looks like, so this reads its verdict
+      // rather than keeping a second list of markers.
+      const wall =
+        document.status >= 200 && document.status < 300
+          ? detectWafProtection(url, document, {}, 1)
+          : null;
+      if (wall?.isBlocked && !wall.isRateLimit) {
+        failures.push(
+          `${path} answers with ${wall.name} instead of the cart (${wall.reason}), so an agent never reaches the document`,
+        );
+        continue;
+      }
+
+      // A widget on a readable cart guards a form on it, which is where the
+      // guidance asks a site to put its challenge. Only a widget on a document
+      // with nothing else to read stands between the agent and the cart.
       const challenge = challengeIn(document.body);
-      if (challenge) {
+      if (challenge && textLength(document.body) < MIN_TEXT) {
         failures.push(
-          `${path} mounts ${challenge} on the document an agent has to read`,
+          `${path} mounts ${challenge} on a document with no readable cart content, so the challenge is the page an agent has to read`,
         );
         continue;
       }

packages/core/src/audits/operability-safety/no-blocking-captcha.test.ts

@@ -53,6 +53,37 @@ describe("NoBlockingCaptchaAudit", () => {
     expect(result.status).toBe("warn");
     expect(result.message).toContain("turnstile");
   });
+
+  // Inline config naming the vendor is not a widget.
+  it("passes a page whose only Turnstile mention is inline config", () => {
+    const page = mockPageContext(
+      "https://example.com",
+      `<html><body>
+        <script>app.captcha.provider = "CloudflareTurnstile"; app.captcha.tokenParam = "cf-turnstile-response";</script>
+        <p>This site is protected by reCAPTCHA and hCaptcha policies.</p>
+        <form action="/contact" method="POST"><input name="email" /></form>
+      </body></html>`,
+    );
+    const result = audit.audit(mockCheckContext([page]));
+    expect(result.status).toBe("pass");
+  });
+
+  // True positive: a Turnstile widget mounted on a form, with no script tag in
+  // the served HTML (the loader comes from a bundle).
+  it("warns on a Turnstile widget mounted on a form", () => {
+    const page = mockPageContext(
+      "https://example.com",
+      `<html><body>
+        <form action="/contact" method="POST">
+          <input name="email" />
+          <div class="cf-turnstile" data-sitekey="0x0000"></div>
+        </form>
+      </body></html>`,
+    );
+    const result = audit.audit(mockCheckContext([page]));
+    expect(result.status).toBe("warn");
+    expect(result.message).toContain("turnstile");
+  });
 });
 
 describe("NoBlockingCaptchaAudit — the wall the scanner met", () => {

packages/core/src/audits/operability-safety/no-blocking-captcha.ts

@@ -4,13 +4,19 @@ import { weightForGrade } from "../../scorer";
 import type { CheckContext } from "../../check-context";
 import { scanReadPageText, unreadPageTextReason } from "../../scan-evidence";
 
-const CAPTCHA_PATTERNS = [
-  "recaptcha",
-  "hcaptcha",
-  "turnstile",
-  "google.com/recaptcha",
-  "hcaptcha.com",
-  "challenges.cloudflare.com/turnstile",
+/**
+ * CAPTCHA inclusions, matched on the script that loads the widget or the
+ * element it mounts on. A body substring also matched prose and inline
+ * config, such as a `provider = "CloudflareTurnstile"` setting.
+ */
+const CAPTCHA_SELECTORS: Array<{ type: string; selector: string }> = [
+  { type: "recaptcha", selector: 'script[src*="/recaptcha/"], .g-recaptcha' },
+  { type: "hcaptcha", selector: 'script[src*="hcaptcha.com"], .h-captcha' },
+  {
+    type: "turnstile",
+    selector:
+      'script[src*="challenges.cloudflare.com/turnstile"], .cf-turnstile',
+  },
 ];
 
 export class NoBlockingCaptchaAudit extends Audit {
@@ -83,10 +89,9 @@ export class NoBlockingCaptchaAudit extends Audit {
     const detectedCaptchas: Array<{ page: string; type: string }> = [];
 
     for (const page of ctx.pages) {
-      const html = page.fetchResult.body.toLowerCase();
-      for (const pattern of CAPTCHA_PATTERNS) {
-        if (html.includes(pattern)) {
-          detectedCaptchas.push({ page: page.url, type: pattern });
+      for (const { type, selector } of CAPTCHA_SELECTORS) {
+        if (page.$(selector).length > 0) {
+          detectedCaptchas.push({ page: page.url, type });
         }
       }
     }