From 180e6553cc91996161a05c7c07557ffe07f3dd25 Mon Sep 17 00:00:00 2001 From: Jake Writer Date: Sat, 26 Sep 2026 20:13:35 +0000 Subject: [PATCH] Docs that match the code, one AGENTS.md, dead code removed, and the audit's bug fixes (#787) * chore: remove build tooling nothing uses - The developer UI (scripts/developer.py, `make edits`). It depended on easygui, which no requirements file declares, and every action it offered is a Makefile target: patch, unpatch, workspace, revert, diff. Its two helpers in scripts/_mixin.py (is_bootstrap_patch, patch) had no other callers. - legacy/, the Go launcher deprecated in 2024-11. Nothing built or shipped it. Its Makefile targets and scripts/run-pw.py go with it, and so does Go from every dependency list and workflow. - jsonvv/ and settings/camoucfg.jvv. Nothing read the .jvv schema: config is validated against settings/properties.json, and the two had already drifted. The jsonvv package stays on PyPI. - Scripts with no caller: bootstrap.py, moztree, setup-wasi-linux.sh, package-helper.sh, install-local-build.sh, mozfetch.sh (copied into lw/ but never packaged), examples/. - The pre-ESM Juggler copies JugglerFrameParent.jsm and JugglerFrameChild.jsm, and hidden-scrollbars.css. Juggler loads the .sys.mjs actors and deliberately no stylesheet, but jar.mn still packaged all three. - patches/librewolf/*.opt, which list_patches() never picks up; the roverfox second pass in patch.py, whose directory no longer exists; the unread --no-settings-pane option. - The CAMOUFOX_PASSWD secret passed to `make fetch` and closedsrc_rev in upstream.sh, which nothing reads. Co-Authored-By: Claude Opus 5.5 * chore(python): remove dead helpers and a stale dependency None of these had a caller: - pkgman: is_supported_path, extract_zip, cleanup and set_version, left over from the single-directory install. cleanup() would have deleted every installed browser version. - multiversion.get_cached_repo_names, CONSTRAINTS.as_range, fingerprints._load_os_voices, utils._clean_locals, and unused imports. Also: - The "Apify Fingerprints" row in `camoufox version`, which has read "?" since fpgen replaced BrowserForge. - lxml is no longer a dependency; nothing imports it. - The geoip extra now names maxminddb, the module geolocation.py actually imports, rather than getting it transitively through geoip2. Co-Authored-By: Claude Opus 5.5 * docs: show the cursor paths humanize=True actually produces The README's cursor video showed the Bezier generator Camoufox replaced with Cursory's recorded trajectories. scripts/cursor-demo.py drives a real build with humanize=True and records every mousemove event the page receives. It writes assets/humanize-cursor.svg, an animated replay at the recorded speed, so what the figure shows is what a site sees. The script cannot change the binary, so ci/browser_inputs.py lists it as non-native and editing it does not invalidate the cached browser. Co-Authored-By: Claude Opus 5.5 * chore(python): stop naming BrowserForge in user-facing text fpgen replaced BrowserForge, but two LeakWarnings, the NonFirefoxFingerprint message and the fingerprint_preset docstring still named it. One warning also linked to a README anchor that no longer exists. Co-Authored-By: Claude Opus 5.5 * docs: one AGENTS.md for every agent, a roadmap, and docs that match the code - AGENTS.md holds the engineering rules for any coding agent, plus the repo map, build, patch and test commands that CLAUDE.md used to carry. CLAUDE.md now only imports it, so there is one set of rules. ci/tribal-rules.yml is the record of settled decisions it points to. - ROADMAP.md lists planned work, each item linked to its issue. - README: - fpgen and the coherence check replace BrowserForge; - the patch workflow uses the make targets instead of the removed developer UI; - letter-spacing noise is described as off by default, as it is. - docs/: - beta-testing-ff146.md removed; - patch-upgrading-guide rewritten around the make targets; - per-context-patches without the canvas patch that no longer exists, and with measured preset counts; - playwright-maintenance without the JSM wrapper that does not exist; - smaller fixes in MEDIA-DEVICES, input-dispatch and FONTS. - ci/README: every job, and the real shard, skiplist and entry-point lists. - pythonlib, tester and patch-dependency READMEs corrected against the code. Co-Authored-By: Claude Opus 5.5 * fix(pythonlib): handle headless='virtual' in launch_server launch_server() is documented to take the same arguments as Camoufox(), but passed headless='virtual' straight to launch_options(), so the server launched with no Xvfb display. Start a VirtualDisplay the way Camoufox() does, launch headful on it, and kill it when the server process exits or the launch fails. Co-Authored-By: Claude Opus 5.5 * chore(python): remove fontprobe, which nothing called fontprobe listed the fonts installed on the host, for a `camoufox fonts` command that was never added. It has nothing to do with the font bundle Camoufox serves to pages. Co-Authored-By: Claude Opus 5.5 * chore(license): the Python launcher is MIT; the browser stays MPL-2.0 The Python package has always been published to PyPI as MIT (#727), but pythonlib/ shipped no licence file, and the repo's LICENSE is the browser's MPL-2.0. MPL is copyleft per file. It covers the modified Firefox sources, not a separate launcher that drives the browser over Playwright. So pythonlib/LICENSE now carries the MIT text its metadata already declares, and a Licensing section in the README says which part is which. Closes #727. Co-Authored-By: Claude Opus 5.5 * fix(python): fingerprint_preset=False no longer turns presets on launch_options checked `fingerprint_preset is not None`, so passing False drew a random bundled preset, the opposite of what was asked. It now uses a truthiness check, and a test proves that None and False never draw a preset. Co-Authored-By: Claude Opus 5.5 * ci: bring the release workflow in line with the tests build job build.yml had drifted from tests.yml. It ran actions at v1/v2 on a retired Node runtime, prepared the source tree with bare make calls that fail the whole release on one dropped connection, and built with a different Python than every pull request is tested with. - Pin every action by commit SHA, at the major versions tests.yml uses (checkout v4, setup-python v5, upload/download-artifact v4, the same remove-unwanted-software SHA), and action-gh-release v2. The release job holds contents: write, so it should not follow a movable tag. - Prepare the tree with `python3 -m ci.run_prepare`, as the tests build job does. BUILD_TARGET is set from the matrix so `make dir` writes the right mozconfig and Rust targets; multibuild.py then finds _READY and builds without re-patching. mach's toolchain bootstrap ignores the mozconfig, so running it after `dir` bootstraps the same toolchains. - Build with Python 3.12, the version the tests build job compiles with. - Default the workflow to no permissions; the build job gets contents: read and the release job keeps contents: write. Co-Authored-By: Claude Opus 5.5 * fix(python): NewContext looks up a proxy's exit IP through the right URL, or fails NewContext derives the context's WebRTC IP and timezone from the proxy's exit IP. That lookup had two defects, and both left the context showing the host's values while its traffic went through the proxy: - It built its own proxy URL with urlparse, which reads a scheme-less server such as "1.2.3.4:8080" (a form Playwright accepts) as scheme "1.2.3.4" with no host. urllib could not use a SOCKS proxy at all. - Any failure was swallowed, and the context opened without the values. The URL is now built with Proxy.as_string(), which the geoip launch path already uses (scheme-less means http). The lookup goes through requests, which handles SOCKS, and a failed lookup raises InvalidIP, naming the two options that skip it. The tests cover scheme-less, http and socks5 servers with credentials, both failure modes, and the case where no lookup is needed, for NewContext and AsyncNewContext. Co-Authored-By: Claude Opus 5.5 * fix: stop generating a canvas seed, and drop config keys nothing reads The browser has not noised the canvas since #528, and no patch reads canvas:seed (#721). The launcher still drew one on every launch and sent it through CAMOU_CONFIG, and NewContext called a setCanvasSeed that does not exist. They no longer do. For users this changes nothing on any browser since #528: the value was ignored. A config that still passes canvas:seed gets the usual "Skipping unknown patch" notice instead of silence. On a browser from before #528, the launcher no longer turns canvas noise on, which is the behaviour #528 chose. The same audit found more keys declared in settings/properties.json that no patch or Juggler file reads, so setting them did nothing: - canvas:aaOffset, canvas:aaCapOffset - memorysaver, pdfViewerEnabled, webrtc:localipv4/6 - navigator.onLine, navigator.cookieEnabled, navigator.languages - navigator.appCodeName, appName, product, productSub. Firefox reports these constants itself, so fpgen.yml no longer maps them. - webGl:parameters:blockIfNotDefined and its WebGL2 twin test_config_schema now checks this direction too: every declared key must be read by the browser, unless it is listed with a reason. Three are listed: locale:script and navigator.doNotTrack, which the launcher applies itself, and navigator.buildID (#780). The build-tester grading followed the same wrong premise. It tracked canvas collisions as an unfixed per-context leak. A canvas that is rendered rather than noised follows the fonts and GPU, as it does on real machines, so canvas collisions are now counted with the other device-level values. The tribal rule that recorded it as an open question is now a settled one, canvas-is-not-noised, with an automated check. Closes #721. Co-Authored-By: Claude Opus 5.5 * fix(python): repair devicePixelRatio the same way on every launch The DPR repair snaps an off-grid ratio to the nearest real scaling step and keeps the first of two equally near steps. The steps were frozenset literals, and a frozenset literal iterates in one order when the module is compiled from source and another when it is loaded back from a .pyc. So a midpoint such as 1.125 became 1.25 on the first launch after an install and 1 on every launch after it: the same pinned identity presented two different devicePixelRatio values. The steps are now ascending tuples, so a tie always goes to the lower step. The test runs the repair in two fresh interpreters that share a bytecode cache, compiling in the first and loading in the second. It failed before this change. Co-Authored-By: Claude Opus 5.5 * fix: remove the glyph-spacing seed from the browser and the launcher anti-font-fingerprinting.patch added a seeded amount to every glyph advance, so that text widths differed per context. No real machine produces those widths: the same font on the same OS measures the same everywhere. So the noise was itself a fingerprint, measured in #779 at +1 px per ~100 glyphs plus fractional deltas on every measureText. #779 defaulted the seed to 0 and kept it as an opt-in, but an opt-in whose only effect is to become detectable is not worth carrying. Removed: - The browser side: - FontSpacingSeedManager and window.setFontSpacingSeed; - the HarfBuzz hook; - the plumbing that existed only to carry the context id down to the shaper: the userContextId on gfxTextRun, gfxShapedWord and the word-cache key, and the extra MakeTextRun argument in nsTextFrame, nsFontMetrics, MathML and canvas. The font group keeps its userContextId, which font-list-spoofing.patch uses to apply the per-context font list. Text is now shaped exactly as stock Firefox shapes it. - The fonts:spacing_seed key. The launcher had been sending 0 on every launch, plus a setFontSpacingSeed(0) call in every context's init script. - tests/patches/config-overrides.py, which tested only the spacing override. A pythonlib test now covers config_overrides with another key. timezone-spoofing, webrtc-ip-spoofing and window-setter-seal change only in context lines and the setter seal list. Every patch applies cleanly to a fresh tree, and the result builds. The settled decision is recorded as no-glyph-spacing-noise, with an automated check. Co-Authored-By: Claude Opus 5.5 * fix: stock animations and speech by default; drop config keys that freeze live values Three behaviours a page could detect, changed in one breaking release: - **Animations run on stock timing.** no-css-animations.patch finished every finite animation at once by default, and any page could read it: `el.animate(frames, 1000).effect.getComputedTiming().duration` was 0, and a 500ms transition reported 0. Measured on v152.0.4-beta.31. The speedup is now an opt-in, `instantAnimations: True`, which raises a LeakWarning. disableInstantAnimations is gone. - **speak() on a spoofed voice works like a real voice.** It fired `error` after 3ms unless voices:fakeCompletion was set, and then start and end in the same tick. It now starts and ends after the text's duration at ~150 words per minute. Both voices:fakeCompletion keys are gone, and so is a debug line printed to stderr on every call. - **Keys removed:** - battery:* and window.scrollMinX/Y: Firefox keeps getBattery() and scrollMin* chrome-only, so no page could read them. - window.scrollMaxX/Y, screen.pageXOffset/pageYOffset, window.history.length and document.body.client*: each pinned a live value to a constant, so scrolling, navigating or re-laying out never changed it. fpgen.yml mapped pageYOffset, so about 15% of identities froze window.scrollY at a non-zero value. - The body keys' role as an undocumented alias for window.innerWidth/Height in browser-init and in the launcher. - MaskConfig::GetInt32Rect, which only the body keys used. New guards, both of which fail on v152.0.4-beta.31: tests/patches/animation-timing.py and tests/patches/spoofed-voice-speaks.py. The decisions are recorded as animations-run-on-stock-timing and spoofed-voices-speak. Every patch applies cleanly to a fresh tree, and the result builds. Co-Authored-By: Claude Opus 5.5 * fix(python)!: remove dead public API and make `list all --path` work Breaking changes: - Remove the exceptions UnknownProperty, InvalidDebugPort and MissingDebugPort. Nothing in the package raises them, so code catching them was catching nothing. - Remove the legacy `allow_webgl` keyword of launch_options(). Use `block_webgl=True`. The keyword now reaches Playwright as an unknown launch option and fails there instead of being silently consumed. `camoufox list all --path` accepted the flag and ignored it. It now prints the install path beside each installed build, as `camoufox list --path` already does for the installed tree. Co-Authored-By: Claude Opus 5.5 * chore(python)!: drop data the package never draws from voices.json shipped in the wheel, but no code in the package reads it: the voice draw uses voice-manifests.json and voice-uris.json. Its only readers are the TypeScript port's golden-fixture generator and data-sync script (typescript/scripts/golden/identity_golden.py, typescript/scripts/sync-identity-data.py), which live on another branch and will need a new source; the last copy is at 676fb3f:pythonlib/camoufox/voices.json. docs/per-context-patches.md described it as runtime data and now describes the files that are. webgl_data.db held two rows with zero weight on every OS ("Intel(R) HD Graphics 400, or similar" from "Intel Inc." and "Radeon R9 200 Series, or similar" from "ATI Technologies Inc."), left behind when their impossible macOS weights were zeroed. No draw can reach them. They are deleted with secure_delete so their blobs do not linger in free pages; the file is not vacuumed, so the other pages are unchanged. A new test requires every row to be drawable on at least one OS. Co-Authored-By: Claude Opus 5.5 * fix(python): warn whenever an identity falls back to a substitute value Several draws swallowed their failure and used something else, so an identity could ship with values the rest of it was not drawn to match and nobody would hear about it: - from_preset(): a failed font or voice draw used the preset's recorded list, or nothing, on any exception. - generate_context_fingerprint(): a failed font, voice or WebGL draw was `except Exception: pass`, leaving the browser's launch-time values. - _load_font_groups() / _load_font_bases(): an unreadable file became {}, i.e. no font additions or no OS-version base. - launch_options(): a failed font draw used every font in fonts.json, a failed voice draw used no voices, and a preset GPU missing from webgl_data.db was silently swapped for a drawn one (36 of the 397 bundled presets). Each site now catches only the errors its data can raise (OSError and ValueError for an unreadable or corrupt file, KeyError for a manifest with no entry for the OS, sqlite3.Error for the WebGL database) and emits a FallbackWarning. The text names what failed and what the identity uses instead, then gives a block to paste into an issue (camoufox, browser, OS and Python versions, the error, and the identity's user agent or GPU), asking the user to report it on GitHub. It shares LeakWarning's caller-frame attribution and its template lives in warnings.yml. The broad excepts had also been hiding a broken fixture: test_launch_environment's font and voice stubs did not accept `seed`, so every draw there raised and was swallowed. Co-Authored-By: Claude Opus 5.5 * fix(python): give NewContext identities the browser's Firefox version NewContext() and AsyncNewContext() passed ff_version=None through to generate_context_fingerprint(), so a context's user agent kept the version fpgen drew (e.g. Firefox/146) while the browser underneath was 152. They now default ff_version to the major version of Playwright's Browser.version, which Juggler reports from MOZ_APP_VERSION_DISPLAY, so the UA always names the browser the page is actually talking to. An explicit ff_version still wins. The docstrings said each context gets "its own real fingerprint preset"; the default has been an fpgen draw, with a preset only when one is passed. Co-Authored-By: Claude Opus 5.5 * fix(python): send an IPv6 WebRTC address to setWebRTCIPv6 The per-context init script passed every webrtc_ip, IPv6 included, to window.setWebRTCIPv4(), and never called setWebRTCIPv6(). An IPv6 address (given directly, or resolved as a proxy's exit IP) was stored as the context's IPv4 value and the IPv6 slot stayed empty. The script now picks the setter by address family, and an address that is neither raises InvalidIP instead of being passed through. Co-Authored-By: Claude Opus 5.5 * fix(python): stop pinning the page's scroll offset from fpgen fpgen.yml mapped the drawn window.pageYOffset (e.g. 528) to screen.pageYOffset, and the browser returns that value from scrollY on every read, so a page saw one scroll position forever whatever the user did. Real scroll offsets are live page state, not part of a device's fingerprint, so neither offset is mapped any more. Co-Authored-By: Claude Opus 5.5 * chore(python): stop checking a config key that no longer exists warn_manual_config() looked for navigator.languages, which was removed from settings/properties.json; validate_config() rejects it before the check could matter. Co-Authored-By: Claude Opus 5.5 * fix(python): close the WebGL database connection on every path sample_webgl raised its not-found and wrong-OS errors before reaching conn.close(), leaking a sqlite connection each time a preset named a GPU the database does not hold. Co-Authored-By: Claude Opus 5.5 * chore(data): drop the 23 presets whose GPU has no WebGL data A preset records only its GPU's name. The WebGL parameters, extensions and shader precision behind it have to come from somewhere, and for these 23 nothing Camoufox has describes the GPU: fpgen has never seen Firefox report it on that OS. So each launch paired the name with another device's parameters, a mismatch any WebGL fingerprinter can see. They were: - Windows on ARM (Adreno 650); - Direct3D 10-level GPUs (vs_4_0/vs_4_1); - "Generic Renderer"; - 945GM and GTX 480 on macOS; - nouveau/Mesa buckets on Linux; - one Linux preset pairing NVIDIA's proprietary vendor string with the nouveau renderer name. scripts/clean-fingerprint-data.py now applies the rule, via a shared fingerprints.firefox_gpus(), and test_shipped_data asserts it. 374 presets remain, and every OS keeps its presets. ROADMAP.md lists capturing WebGL data for these GPUs, which would bring them back. Co-Authored-By: Claude Opus 5.5 * feat(python): draw every identity's WebGL from fpgen WebGL vendor, renderer, context attributes, extensions, parameters and shader precisions, for WebGL1 and WebGL2, now come from fpgen's recorded Firefox devices instead of webgl_data.db, which is deleted with the camoufox/webgl/ package. camoufox/webgl.py: - webgl_for_gpu() traces `webgl` given Firefox, the OS and the GPU, then `webgl2` given the chosen `webgl` too, and draws each with one seeded random.Random. The GPU and the webgl value are pinned by their fpgen lookup index: a dict condition is flattened into leaves that overwrite each other, so only the renderer applied and Linux "Mesa" and "AMD" Radeon HD 3200 devices came back mixed. - sample_webgl_for_screen() draws the GPU of a generated identity from fpgen's per-OS weights, filtering out software rasterisers, GPUs the OS cannot report, discrete GPUs behind a netbook screen and the resistFingerprinting "Mozilla" mask before the weighted choice, so there is no rejection loop. An empty pool raises. - The draft/host-dependent extension filter moves over unchanged. A preset's GPU and a caller's webgl_config pair are looked up as given; a pair fpgen has never seen from Firefox on that OS raises instead of falling back to another GPU. generate_context_fingerprint no longer falls back to the host GPU when the draw fails. For 10 of the 15 (GPU, OS) pairs the two sources share, one of fpgen's records converts to exactly the database row on every value the browser reads. The other five rows (Linux R9 200 and Radeon HD 3200, macOS Intel HD, and two software rasterisers) are devices fpgen does not carry; those GPUs now present fpgen's recorded devices instead. The Linux GTX 980 row is kept as a test fixture. Co-Authored-By: Claude Opus 5.5 * docs: say where WebGL comes from now that the database is gone The per-context guide, the fpgen.yml header and coherence's comments still named webgl_data.db and sample_webgl(). They now point at camoufox/webgl.py and fpgen. The guide also claimed presets carry WebGL parameters; they record only the vendor and renderer. Co-Authored-By: Claude Opus 5.5 * fix(patches): a host's missing speech daemon no longer errors spoofed speech On a Linux host where speech-dispatcher cannot start, Firefox broadcasts synth-voices-error, and SpeechSynthesis answers it by firing `error` on every queued utterance. So a spoofed Windows voice errored about 11ms into speak() on any host without the daemon: the CI runners, and most servers. It passed only where the daemon runs. While Camoufox manages the voice list, the registry no longer forwards a host backend's error. The spoofed voices do not depend on the host's engine, and a Windows or macOS identity never raises one. The guard now makes the daemon unreachable itself, so it tests this case on every machine; on the previous build it fails every time. Co-Authored-By: Claude Opus 5.5 * ci: stop skipping the two click tests that stock animation timing fixed test_wait_for_stable_position and test_timeout_waiting_for_stable_position were skipped with humanized travel time as the reason. The real cause was instant animations. Every finite animation finished at once, so the button Playwright waits on to stop moving never moved, and the click landed where upstream does not expect. With animations on stock timing both pass, and the skiplist audit flagged them as no longer failing. The entries go, and the counts in ci/README.md drop from 14 to 12. Co-Authored-By: Claude Opus 5.5 * fix(build-tester): accept 18 and 22 cores, as real hardware reports plausibleHWC's list of common core counts lacked 18 and 22 -- Intel Meteor Lake laptops (Core Ultra 5 125H, Core Ultra 7 155H), and 22 is in 8 recorded presets. build-tester draws random presets, so a run that picked one of the two Linux presets reporting 22 failed: about one run in eleven, on any pull request. A CI self-test now fails if the list rejects any core count pythonlib can present (the presets and PLAUSIBLE_CORE_COUNTS). Co-Authored-By: Claude Opus 5.5 * test(guards): judge the query-cost probes on a median, not one sample stock-parity-probes timed each getter once. On a shared runner one GC pause or CPU-steal spike decided the verdict: navigator.hardwareConcurrency took 77 ms against a 50 ms allowance on the same restored build that passed the run before. Each pair is now timed five times, interleaved, and compared by median. The regressions these catch (a sync IPC per read, ~240 ms over the loop) cost extra on every read, so they move the median; verified by giving the getter a constant ~4 us of extra work per read -- 86 ms median, FAIL -- while the healthy build passes. Co-Authored-By: Claude Opus 5.5 * test(native): compare the whole fingerprint when two launches must differ test_two_browsers_get_different_fingerprints compared seven coarse values: UA, platform, screen size, core count, timezone and language. CI pins the timezone and language, and real machines share the rest: two draws of a common Mac (Firefox 152, MacIntel, 2560x1440, 8 cores) matched, and the test failed on a correct browser. It now reads the whole fingerprint a site computes, from a script in the page: - navigator values, screen and window geometry, device pixel ratio, timezone; - WebGL vendor, renderer, limits and extensions; - installed fonts, measured by width against the generic fallbacks; - voices, media-device counts, and an OfflineAudioContext hash. The page is served from an https URL Playwright fulfils locally, because mediaDevices exists only in a secure context. The page computes the result itself because the isolated world may not read audio sample data. The test then requires the fingerprints to differ, and the audio hash to differ on its own, since its noise is seeded per identity. Co-Authored-By: Claude Opus 5.5 * Update README to remove warning, camoufox is now actively maintained Camoufox will now be actively maintained and improved for the foreseeable future --------- Co-authored-by: Claude Opus 5.5 --- .dockerignore | 2 +- .github/workflows/build.yml | 54 +- .github/workflows/tests.yml | 8 +- .gitignore | 5 - AGENTS.md | 187 +++ CLAUDE.md | 107 +- CONTRIBUTING.md | 12 +- Dockerfile | 2 +- Makefile | 43 +- README.md | 83 +- ROADMAP.md | 66 ++ additions/camoucfg/MaskConfig.hpp | 12 - additions/juggler/JugglerFrameParent.jsm | 40 - .../juggler/content/JugglerFrameChild.jsm | 85 -- .../juggler/content/hidden-scrollbars.css | 7 - additions/juggler/jar.mn | 3 - assets/humanize-cursor.svg | 571 ++++++++++ build-tester/README.md | 12 +- build-tester/run_tests.sh | 2 +- build-tester/scripts/generate-presets.py | 2 - build-tester/scripts/presets.py | 2 - build-tester/src/lib/checks/extended.ts | 9 +- build-tester/src/lib/checks/index.ts | 1 - build-tester/src/lib/types.ts | 2 - ci/README.md | 77 +- ci/browser_inputs.py | 1 + ci/run_build_tester.py | 54 +- ci/run_prepare.py | 2 +- ci/skiplist.yml | 23 +- ci/tests/test_ci.py | 65 +- ci/tribal-rules.yml | 78 +- docs/FONTS.md | 2 +- docs/MEDIA-DEVICES.md | 6 +- docs/beta-testing-ff146.md | 64 -- docs/input-dispatch.md | 2 +- docs/patch-upgrading-guide.md | 318 ++---- docs/per-context-patches.md | 151 +-- docs/playwright-maintenance.md | 125 +- jsonvv/README.md | 728 ------------ jsonvv/jsonvv/__init__.py | 21 - jsonvv/jsonvv/__main__.py | 70 -- jsonvv/jsonvv/exceptions.py | 33 - jsonvv/jsonvv/parser.py | 309 ----- jsonvv/jsonvv/strings.py | 64 -- jsonvv/jsonvv/types.py | 262 ----- jsonvv/jsonvv/validator.py | 170 --- jsonvv/publish.sh | 12 - jsonvv/pyproject.toml | 25 - legacy/README.md | 6 - legacy/launcher/build.sh | 34 - legacy/launcher/constants.go | 47 - legacy/launcher/exec.go | 166 --- legacy/launcher/go.mod | 7 - legacy/launcher/go.sum | 4 - legacy/launcher/load-addons.go | 157 --- legacy/launcher/main.go | 219 ---- legacy/launcher/procgroup-unix.go | 17 - legacy/launcher/procgroup-win.go | 16 - legacy/launcher/validate.go | 75 -- legacy/launcher/xpi-dl.go | 146 --- legacy/scripts/generate-locales.sh | 49 - native-tests/_churn.py | 2 +- native-tests/test_contexts_vs_browsers.py | 131 ++- native-tests/test_tribal_rules.py | 42 + patches/anti-font-fingerprinting.patch | 1012 +---------------- patches/browser-init.patch | 22 +- patches/fingerprint-injection.patch | 212 +--- patches/librewolf/1550_1549.diff.opt | 33 - patches/librewolf/arm.patch.opt | 12 - .../librewolf/bootstrap-without-vcs.patch.opt | 179 --- patches/no-css-animations.patch | 19 +- patches/patch-dependencies.md | 63 +- patches/playwright/README.md | 2 +- patches/timezone-spoofing.patch | 31 +- patches/voice-spoofing.patch | 68 +- patches/webrtc-ip-spoofing.patch | 20 +- patches/window-setter-seal.patch | 35 +- pythonlib/LICENSE | 21 + pythonlib/README.md | 23 +- pythonlib/camoufox/__main__.py | 8 +- pythonlib/camoufox/__version__.py | 6 - pythonlib/camoufox/_warnings.py | 83 +- pythonlib/camoufox/async_api.py | 60 +- pythonlib/camoufox/coherence.py | 25 +- pythonlib/camoufox/exceptions.py | 28 +- .../camoufox/fingerprint-presets-v150.json | 618 ---------- pythonlib/camoufox/fingerprint-presets.json | 168 --- pythonlib/camoufox/fingerprints.py | 203 ++-- pythonlib/camoufox/fontprobe.py | 297 ----- pythonlib/camoufox/fpgen.yml | 12 +- pythonlib/camoufox/geolocation.py | 2 +- pythonlib/camoufox/ip.py | 28 + pythonlib/camoufox/multiversion.py | 8 - pythonlib/camoufox/pkgman.py | 33 - pythonlib/camoufox/server.py | 16 +- pythonlib/camoufox/sync_api.py | 56 +- pythonlib/camoufox/utils.py | 123 +- pythonlib/camoufox/voices.json | 382 ------- pythonlib/camoufox/warnings.yml | 17 +- pythonlib/camoufox/webgl.py | 194 ++++ pythonlib/camoufox/webgl/__init__.py | 3 - pythonlib/camoufox/webgl/sample.py | 166 --- pythonlib/camoufox/webgl/webgl_data.db | Bin 270336 -> 0 bytes pythonlib/pyproject.toml | 5 +- pythonlib/tests/data/webgl-gtx980-linux.json | 567 +++++++++ pythonlib/tests/test_cli_list.py | 25 + pythonlib/tests/test_coherence.py | 39 +- pythonlib/tests/test_config_schema.py | 40 + pythonlib/tests/test_fallback_warnings.py | 82 ++ pythonlib/tests/test_identity_salt.py | 67 +- pythonlib/tests/test_launch_environment.py | 4 +- pythonlib/tests/test_new_context_version.py | 36 + pythonlib/tests/test_proxy_geo.py | 96 ++ .../tests/test_scroll_offset_unmapped.py | 16 + pythonlib/tests/test_server.py | 76 ++ pythonlib/tests/test_shipped_data.py | 46 +- pythonlib/tests/test_viewport_default.py | 1 - pythonlib/tests/test_voices.py | 7 +- pythonlib/tests/test_webgl.py | 186 +++ .../tests/test_webgl_extension_filter.py | 60 - .../tests/test_webgl_screen_consistency.py | 69 +- pythonlib/tests/test_webrtc_ip_setter.py | 23 + scripts/_mixin.py | 22 - scripts/bootstrap.py | 430 ------- scripts/clean-fingerprint-data.py | 45 +- scripts/copy-additions.sh | 1 - scripts/cursor-demo.py | 89 ++ scripts/developer.py | 362 ------ scripts/examples/buttonclick.html | 63 - scripts/examples/serve.sh | 1 - scripts/examples/webgl.html | 341 ------ scripts/install-deps.sh | 13 +- scripts/install-local-build.sh | 141 --- scripts/mozfetch.sh | 7 - scripts/moztree | 3 - scripts/package-helper.sh | 22 - scripts/patch.py | 28 +- scripts/run-pw.py | 82 -- scripts/setup-wasi-linux.sh | 41 - service-tester/README.md | 16 +- service-tester/run_tests.ps1 | 4 +- service-tester/run_tests.py | 2 +- settings/camoucfg.jvv | 319 ------ settings/properties.json | 36 +- tests/camoufox/README.md | 6 +- tests/patches/animation-timing.py | 66 ++ tests/patches/config-overrides.py | 152 --- tests/patches/fingerprint-setter-seal.py | 1 - tests/patches/spoofed-voice-speaks.py | 87 ++ tests/patches/stock-parity-probes.py | 30 +- upstream.sh | 1 - 151 files changed, 3755 insertions(+), 9486 deletions(-) create mode 100644 AGENTS.md create mode 100644 ROADMAP.md delete mode 100644 additions/juggler/JugglerFrameParent.jsm delete mode 100644 additions/juggler/content/JugglerFrameChild.jsm delete mode 100644 additions/juggler/content/hidden-scrollbars.css create mode 100644 assets/humanize-cursor.svg delete mode 100644 docs/beta-testing-ff146.md delete mode 100644 jsonvv/README.md delete mode 100644 jsonvv/jsonvv/__init__.py delete mode 100644 jsonvv/jsonvv/__main__.py delete mode 100644 jsonvv/jsonvv/exceptions.py delete mode 100644 jsonvv/jsonvv/parser.py delete mode 100644 jsonvv/jsonvv/strings.py delete mode 100644 jsonvv/jsonvv/types.py delete mode 100644 jsonvv/jsonvv/validator.py delete mode 100644 jsonvv/publish.sh delete mode 100644 jsonvv/pyproject.toml delete mode 100644 legacy/README.md delete mode 100644 legacy/launcher/build.sh delete mode 100644 legacy/launcher/constants.go delete mode 100644 legacy/launcher/exec.go delete mode 100644 legacy/launcher/go.mod delete mode 100644 legacy/launcher/go.sum delete mode 100644 legacy/launcher/load-addons.go delete mode 100644 legacy/launcher/main.go delete mode 100644 legacy/launcher/procgroup-unix.go delete mode 100644 legacy/launcher/procgroup-win.go delete mode 100644 legacy/launcher/validate.go delete mode 100644 legacy/launcher/xpi-dl.go delete mode 100644 legacy/scripts/generate-locales.sh delete mode 100644 patches/librewolf/1550_1549.diff.opt delete mode 100644 patches/librewolf/arm.patch.opt delete mode 100644 patches/librewolf/bootstrap-without-vcs.patch.opt create mode 100644 pythonlib/LICENSE delete mode 100644 pythonlib/camoufox/fontprobe.py delete mode 100644 pythonlib/camoufox/voices.json create mode 100644 pythonlib/camoufox/webgl.py delete mode 100644 pythonlib/camoufox/webgl/__init__.py delete mode 100644 pythonlib/camoufox/webgl/sample.py delete mode 100644 pythonlib/camoufox/webgl/webgl_data.db create mode 100644 pythonlib/tests/data/webgl-gtx980-linux.json create mode 100644 pythonlib/tests/test_cli_list.py create mode 100644 pythonlib/tests/test_fallback_warnings.py create mode 100644 pythonlib/tests/test_new_context_version.py create mode 100644 pythonlib/tests/test_proxy_geo.py create mode 100644 pythonlib/tests/test_scroll_offset_unmapped.py create mode 100644 pythonlib/tests/test_webgl.py delete mode 100644 pythonlib/tests/test_webgl_extension_filter.py create mode 100644 pythonlib/tests/test_webrtc_ip_setter.py delete mode 100644 scripts/bootstrap.py create mode 100644 scripts/cursor-demo.py delete mode 100644 scripts/developer.py delete mode 100644 scripts/examples/buttonclick.html delete mode 100644 scripts/examples/serve.sh delete mode 100644 scripts/examples/webgl.html delete mode 100755 scripts/install-local-build.sh delete mode 100644 scripts/mozfetch.sh delete mode 100644 scripts/moztree delete mode 100644 scripts/package-helper.sh delete mode 100644 scripts/run-pw.py delete mode 100644 scripts/setup-wasi-linux.sh delete mode 100644 settings/camoucfg.jvv create mode 100644 tests/patches/animation-timing.py delete mode 100644 tests/patches/config-overrides.py create mode 100644 tests/patches/spoofed-voice-speaks.py diff --git a/.dockerignore b/.dockerignore index a53cdc37f..b5168046b 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,7 +1,7 @@ # The Dockerfile does `COPY . /app` and then runs `make setup-minimal` inside # the image, so everything the build system itself needs (Makefile, upstream.sh, # multibuild.py, scripts/, patches/, additions/, settings/, assets/, bundle/, -# pythonlib/, jsonvv/, legacy/) must stay in the context. This file only drops +# pythonlib/) must stay in the context. This file only drops # things the in-image build never reads. See #698. # Git history -- by far the largest single item, and the build only needs the diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 2a630760f..fb43c76bd 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -10,9 +10,23 @@ on: # trigger a browser build (see scripts/fetch-fonts.py). - "!font-bundle-*" +permissions: {} + +env: + # The version tests.yml builds and tests with, so a release is compiled by the + # same interpreter every pull request's build job already exercised. + PYTHON_VERSION: "3.12" + jobs: build: runs-on: ubuntu-24.04 + permissions: + contents: read + env: + # `make dir` (inside ci.run_prepare) writes the mozconfig and adds the Rust + # targets from BUILD_TARGET, defaulting to macos,arm64 when it is unset. + # multibuild.py sets the same value again before building. + BUILD_TARGET: ${{ matrix.target }},${{ matrix.arch }} strategy: matrix: target: [linux, windows, macos] @@ -29,7 +43,7 @@ jobs: steps: - name: Maximize build space - uses: AdityaGarg8/remove-unwanted-software@v4.1 + uses: AdityaGarg8/remove-unwanted-software@8831c82abf29b34eb2caac48d5f999ecfc0d8eef # v4.1 with: remove-dotnet: "true" remove-android: "true" @@ -57,19 +71,12 @@ jobs: sudo apt-get autoremove -y > /dev/null sudo apt-get clean > /dev/null - - uses: actions/checkout@v2 - with: - fetch-depth: 1 - - - name: Set up Go - uses: actions/setup-go@v2 - with: - go-version: "1.23" + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - name: Set up Python - uses: actions/setup-python@v2 + uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0 with: - python-version: "3.11" + python-version: ${{ env.PYTHON_VERSION }} - name: Set up LLVM run: | @@ -110,17 +117,12 @@ jobs: # reports that the packaged fontconfig cannot resolve. run: python3 scripts/verify-fonts.py - - name: Fetch source - env: - CAMOUFOX_PASSWD: ${{ secrets.CAMOUFOX_PASSWD }} - run: | - make fetch - - - name: Setup and bootstrap - run: | - make setup-minimal - make mozbootstrap - mkdir -p dist + - name: Prepare the source tree + # setup-minimal (which fetches the tarball) -> dir -> mozbootstrap, with + # the two network-bound steps retried on transient failures, exactly as + # the tests.yml build job does. multibuild.py then finds _READY and + # builds without re-patching. + run: python3 -m ci.run_prepare - name: Create swap space run: | @@ -139,7 +141,7 @@ jobs: run: python3 ./multibuild.py --target ${{ matrix.target }} --arch ${{ matrix.arch }} - name: Upload artifacts - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: CamoufoxBuilds-${{ matrix.target }}-${{ matrix.arch }} path: dist/* @@ -152,17 +154,15 @@ jobs: steps: - name: Download all artifacts - uses: actions/download-artifact@v4 + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: path: artifacts - name: Create Release - uses: softprops/action-gh-release@v1 + uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2.6.2 if: startsWith(github.ref, 'refs/tags/') with: files: artifacts/**/* generate_release_notes: true draft: true prerelease: true - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index c42f14eb2..3b9c2f9b1 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -390,10 +390,6 @@ jobs: if: steps.prebuilt.outputs.cache-hit != 'true' with: python-version: ${{ env.PYTHON_VERSION }} - - uses: actions/setup-go@v5 - if: steps.prebuilt.outputs.cache-hit != 'true' - with: - go-version: "1.23" - name: Install build dependencies if: steps.prebuilt.outputs.cache-hit != 'true' @@ -432,8 +428,6 @@ jobs: - name: Prepare the source tree if: steps.prebuilt.outputs.cache-hit != 'true' - env: - CAMOUFOX_PASSWD: ${{ secrets.CAMOUFOX_PASSWD }} run: | ccache -M 8G && ccache -z echo "CCACHE_DIR=$HOME/.ccache" >> "$GITHUB_ENV" @@ -476,7 +470,7 @@ jobs: # FileNotFoundError without it. That breaks # patch-guards, the leak suite and sundial. make stage-fonts - for f in properties.json chrome.css camoucfg.jvv; do + for f in properties.json chrome.css; do [ -f "$src/$f" ] || cp -v "settings/$f" "$src/$f" done diff --git a/.gitignore b/.gitignore index c0ac6560d..63dba2e17 100644 --- a/.gitignore +++ b/.gitignore @@ -4,15 +4,12 @@ /mozilla-unified dist/ bin/ -launch -launch.exe # Internal testing /extra-docs pythonlib/test* !pythonlib/tests/ -jsonvv/test* /.vscode /bundle/fonts/extra @@ -52,7 +49,6 @@ __pycache__/ *.pyc *.mmdb pythonlib-dev/ -run-pw-dev.py # Closed source patches private @@ -72,7 +68,6 @@ closedsrc /application.ini /platform.ini /camoufox.cfg -/camoucfg.jvv /chrome.css /properties.json /removed-files diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..6e58c6c59 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,187 @@ +# AGENTS.md + +Instructions for any coding agent working in this repository: Claude Code, +Codex, Cursor, Copilot, Gemini or anything else. `CLAUDE.md` only imports this +file, so there is one set of rules. Edit them here. + +--- + +## Before you touch anything + +1. **This repository is public.** The professionalism rules below govern every + line, commit message, PR title and comment you write. +2. **Read [`ci/tribal-rules.yml`](ci/tribal-rules.yml).** It lists the decisions + this repo has already made, each with its evidence. Many are checked by + `native-tests/test_tribal_rules.py`. Do not reopen one without new evidence. + When a new decision is argued and settled, add it there with its citation. +3. **Branch off `main`. Never commit or push to `main`.** Every change is a pull + request tied to a GitHub issue ([`CONTRIBUTING.md`](CONTRIBUTING.md)). + +## Before you write a function + +Search for an existing implementation first: `pythonlib/camoufox/`, `ci/`, +`scripts/` and `additions/`. If something close +exists, extend it rather than forking it. Duplicated logic is a serious defect +here, because the copies drift and a fingerprint built from two drifting copies +is detectable. + +## Professionalism + +A public repository is a finished product. People judge the project by what +is in it, and they will not ask what a file was for. + +- **No garbage.** No scratch files, diagnostic scripts, saved test output, + logs, debug logging, commented-out code, dead code, placeholder text, stub + docs, or `TODO`s left as notes to self. +- **Nothing private reaches this repo.** That includes proprietary results, + private repo names, and per-vector stealth detail. `ci/run_sundial.py` reports + a grade and a count; the vectors never leave that module. +- Write commits and PR titles for a stranger reading them in a year. +- **Documentation is never stale.** Update the README, `docs/` and the package + READMEs in the same pull request as the code that changed them, never + "later". Every snippet must run as written. + +## Code + +- **Less code.** More code is a cost, not an achievement. Make minimal, general + changes, and delete dead code when you find it. +- **No band-aids.** Fix the main flow. A hard-coded value or a special case at + the call site is a bug relocated, not fixed. +- **Fallbacks are a last resort.** They turn a loud failure into a silent wrong + answer, and in an anti-detect browser a silent wrong answer is a fingerprint. + Fail loudly instead. +- **Check real data before inventing a value.** Every spoofed value must be + something a real device reports. Take it from the recorded distributions + (fpgen, `pythonlib/camoufox/*.json`), not from memory. +- **Read the provider's docs** before writing against a third-party API or a + Firefox internal. Never infer an endpoint, pref or field from memory. +- **Comments say why, in a sentence or two.** Simple code needs none. A longer + explanation is a decision: put it in `ci/tribal-rules.yml` or `docs/`. + +## Tests + +- **Run the failing test, not the whole suite.** CI runs the full pipeline on + every pull request. +- **Write test output to a log file and grep the file.** Piping a run straight + into `grep` throws away output you will need. Delete the log afterwards. +- **Every bug gets a regression test, in this order:** reproduce it with a test + and confirm the test fails, fix the bug, confirm the test passes, then land + both. +- **No flakes.** An intermittent failure means something is non-deterministic. + Fix that behaviour. Never retry, loosen a threshold or skip the test to get + green. +- **A missing prerequisite fails in CI; it never passes as a skip.** A skip + reads as green, so a job that forgot to install something passes having + tested nothing. +- **When local and CI disagree, name the mechanism and fix it in the repo.** + Typical causes are git-ignored inputs, skipped prerequisites and cold-cache + races. A fresh clone is a sanity check, not a fix. +- **Never write a test just to pass, or code just to pass a test.** + +## Security + +- **Never commit a credential.** That covers code, config, fixtures, logs and + commit messages. CI secrets live in GitHub Actions secrets. +- **Least privilege** for workflows and tokens. Grant `permissions:` per job, + and only what that job needs. +- **Adding a dependency is a decision.** Say why in the pull request, and commit + the lockfile. + +## Working with the maintainers + +- Say when a direction is wrong, before starting, and give the reason. +- Review your own diff the way a strict senior reviewer would. You are biased + toward what you just wrote. +- Explain plainly and briefly. + +## Parallel work + +- **Separate pull requests:** one agent per task, each in its own git worktree, + running at the same time. +- **One pull request with independent slow parts:** use subagents in worktrees. + Merge each part into your branch as it finishes, then delete the worktrees. +- **Shared files or an ordering requirement:** use one agent. + +--- + +# Repository + +## What this is + +Camoufox is an anti-detect Firefox for web scraping and automation. This repo +is **not the Firefox source**. It is a build system that fetches upstream +Firefox, applies `patches/` and copies in `additions/`, and produces the +browser. Fingerprint spoofing happens in C++ and in Juggler, not in injected +JavaScript, so a page cannot see it. + +`upstream.sh` pins `version` and `release`. The `Makefile` sources and exports +it, so every script sees them. The Firefox tree lives in +`camoufox--/`. It is generated: persist a change there as a +patch, never as an edit to that tree. + +| Path | What it is | +|---|---| +| `patches/` | Diffs applied to Firefox (44 top level, plus `playwright/`, `librewolf/`, `ghostery/`). Browser behaviour changes here. | +| `additions/camoucfg/` | The C++ config layer. `MaskConfig.hpp` reads `CAMOU_CONFIG`, which the patches consult. | +| `additions/juggler/` | Camoufox's Juggler, Playwright's Firefox protocol. The page agent runs in an isolated world. `input/` holds the Cursory cursor trajectories. | +| `settings/` | `camoufox.cfg` (prefs), `properties.json` (every config key and its type), `chrome.css`, policies. | +| `scripts/` | `patch.py` applies patches and writes the mozconfig; `copy-additions.sh`, `package.py`, `install-deps.sh`, font tooling. | +| `pythonlib/` | The `camoufox` PyPI package, the reference launcher. It draws identities with [fpgen](https://github.com/scrapfly/fingerprint-generator), checks them with `coherence.py`, and launches the binary. | +| `ci/` | The test pipeline (below). `ci/tribal-rules.yml` holds the settled decisions. | +| `build-tester/`, `tests/`, `native-tests/`, `service-tester/` | Test suites (below). | +| `bundle/` | Font bundle manifests. The fonts themselves are a release asset: `make fonts-extract`. | + +## Building + +The build runs on **Linux**. Windows and macOS binaries are cross-compiled from +it. `mach` needs Python 3.11 or newer. + +```bash +bash scripts/install-deps.sh # host build dependencies +make dir # fetch Firefox, copy additions, apply every patch +make bootstrap # one time: system packages + mach bootstrap +make build # ./mach build +make run # run the build (wipes ~/.camoufox) +make package-linux arch=x86_64 # or package-macos / package-windows +python3 multibuild.py --target linux windows macos --arch x86_64 arm64 +``` + +Install `ccache`. A cold build takes about 40 minutes, and an incremental one +about 5. + +## Changing a patch + +Never hand-edit a `.patch` file. Edit the tree, then write the diff: + +```bash +make dir && make first-checkpoint # a new patch: checkpoint, edit, then +make diff > patches/my-change.patch # (git add -N new files first) + +make dir && make workspace ./patches/x.patch # an existing patch: edit, then +make diff > patches/x.patch +``` + +`make patch` and `make unpatch` apply or reverse one patch. `make revert` resets +the tree to unpatched Firefox. Keep the `Makefile` diff minimal: host +dependencies belong in `scripts/install-deps.sh`. + +## Testing + +`ci/` is the whole pipeline. It runs the same way locally and on a pull request +([`ci/README.md`](ci/README.md)). Branch protection requires one check, **`All +tests passed`**. Run the suite that covers your change while you work: + +| You changed | Run | +|---|---| +| Patches, C++, Juggler | `python3 -m ci.run_build_tester --binary ` (the anti-detect suite) and `python3 -m ci.run_patch_guards --binary ` (one guard per spoofing behaviour) | +| Automation behaviour | `make tests`, the upstream Playwright suite with `ci/skiplist.yml` applied | +| `pythonlib/` | `python3 -m ci.run_pythonlib` | +| `ci/` itself | `python3 -m pytest ci/tests -q` | + +- **`tests/` is not a fork of Playwright's tests.** It holds only `patches/` + and `camoufox/`. A deliberate difference from upstream goes in + `ci/skiplist.yml`, with its reason. +- **Tests against an unpackaged build** need `make stage-fonts` first. Without + it the browser has no content fonts. +- **The stealth grade** (`ci/run_sundial.py`) reports a letter and a count, never + the individual vectors. diff --git a/CLAUDE.md b/CLAUDE.md index 936eb513e..43c994c2d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,106 +1 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## What this is - -Camoufox is an anti-detect fork of Firefox for web scraping and automation. This repo is **not the Firefox source** — it is a *build system* that fetches upstream Firefox, applies a stack of patches + code additions, and produces a hardened, fingerprint-spoofing browser. The distinguishing design choice is that fingerprint spoofing happens at the **C++/Juggler implementation level**, not via injected JavaScript, so it is invisible to page-side inspection. - -The actual Firefox tree lives in `camoufox--/` (e.g. `camoufox-150.0.2-beta.25/`), created by the build. That directory is generated — never edit it directly to make lasting changes; changes there are captured as patches (see "Making patches" below). - -`upstream.sh` pins `version` / `release`, and is sourced+exported by the `Makefile`, so those variables flow into every script. - -## Build commands - -The build system is designed for **Linux**. Windows and macOS binaries are **cross-compiled from Linux** — they are never built natively. (`scripts/install-deps.sh` covers macOS/Linux host dependencies for local `make dir` + bootstrap experimentation; a full production build path is Linux/Docker.) - -```bash -bash scripts/install-deps.sh # install host build deps (Python ≥3.11, Rust, aria2, p7zip, go, msitools, wget, sqlite) -make dir # fetch Firefox source, extract, copy additions/settings, apply all patches → touches _READY -make bootstrap # install system deps (apt/dnf/pacman) + run `mach bootstrap` (one-time) -make build # ./mach build in the source dir -make run # run the built browser (wipes ~/.camoufox profile) -make run args="--headless https://test.com" -python3 multibuild.py --target linux windows macos --arch x86_64 arm64 i686 # full cross-platform build + package -``` - -`make dir` is the pipeline that matters: `setup` (fetch tarball via `aria2c` → extract → `copy-additions.sh`) → `python3 scripts/patch.py` (applies every patch, writes `mozconfig`) → `_READY`. `mach` requires **Python ≥ 3.11** (stdlib `tomllib`); older `python3` crashes with `ModuleNotFoundError: No module named 'tomllib'`. - -Docker is the portable path: `docker build -t camoufox-builder .` then `docker run -v "$(pwd)/dist:/app/dist" camoufox-builder --target --arch `. - -Packaging: `make package-linux|package-macos|package-windows arch=` (wraps `scripts/package.py`). Launcher (Go): `make build-launcher arch= os=`. - -## Working with patches (the core workflow) - -Almost all browser-behavior changes are `patches/*.patch` (~49 patches: `fingerprint-injection.patch`, `webgl-spoofing.patch`, `navigator-spoofing.patch`, `webrtc-ip-spoofing.patch`, the `playwright/` and `librewolf/` and `ghostery/` subdirs, etc.). Do not hand-edit patch files. - -Use the developer UI instead: - -```bash -make edits # launches scripts/developer.py — apply/undo/create/manage patches -``` - -- **New patch:** in the UI "Reset workspace" → edit files in `camoufox-*/` → `make build` / `make run` to test → "Write workspace to patch". -- **Edit existing patch:** "Edit a patch" (resets workspace to that patch's state) → edit → "Write workspace to patch" to overwrite. - -Low-level equivalents: `make patch ./patches/x.patch`, `make unpatch ./patches/x.patch`, `make workspace ./patches/x.patch`, `make revert` (reset to `unpatched` tag), `make diff` (diff against `first-checkpoint`). The source dir is a git repo with `unpatched` / `first-checkpoint` / `checkpoint` tags used by these targets. - -## Repository layout (the parts that require cross-file understanding) - -- **`patches/`** — the diffs applied to Firefox source. This is where browser behavior is changed. -- **`additions/`** — whole files copied *into* the source tree (not diffs) by `scripts/copy-additions.sh`: - - `additions/camoucfg/` — the C++ config layer. `MaskConfig.hpp` reads the spoofing config (from `CAMOU_CONFIG` env var / `camoufox.cfg`) that the patches consult at the C++ level. (The human-cursor algorithm used to live here too; it is now `additions/juggler/input/CursorTrajectory.js` and the vendored Cursory beside it.) - - `additions/juggler/` — Camoufox's patched **Juggler** (Firefox's Playwright automation protocol, the Firefox analog of CDP). This is where Playwright is made undetectable — the page agent runs in an isolated scope so injected automation JS is not visible to the page. -- **`settings/`** — `camoufox.cfg`, `chrome.css`, `properties.json`, `camoucfg.jvv`, prefs/policies. Copied into the source's `lw/` dir by `copy-additions.sh`. Edit the built config with `make edit-cfg`. -- **`scripts/`** — `patch.py` (the patcher, LibreWolf-derived), `developer.py` (the `make edits` UI), `package.py`, `copy-additions.sh`, `install-deps.sh`. -- **`pythonlib/`** — the `camoufox` PyPI package: the Playwright-compatible Python interface that generates + injects fingerprints via [fpgen](https://github.com/scrapfly/fingerprint-generator) and launches the binary. `fingerprint-presets-v150.json` holds real scraped fingerprints; `coherence.py` checks the assembled identity (the pools are sampled independently, so an impossible machine can be built from individually plausible parts), and `scripts/clean-fingerprint-data.py` applies the same rules to the shipped data files. This is the user-facing API; the browser binary is the backend. -- **`jsonvv/`** — JSON-with-validation format library used for `camoucfg.jvv` (config schema). -- **`legacy/launcher/`** — Go launcher binary. -- **`assets/`** — `base.mozconfig` and other build inputs. - -## Testing - -`ci/` is the whole pipeline, and it runs identically locally and on a pull -request — see [`ci/README.md`](ci/README.md). Every gate below must pass before a -PR can merge; the workflow is `.github/workflows/tests.yml`. - -- **`build-tester/`** — the raw binary directly, bypassing the Python package: - eight fingerprint profiles, injected via `generate_context_fingerprint` + - `addInitScript` and `CAMOU_CONFIG`. **This is the anti-detect suite** — run it - when changing patches, C++, or the JS browser layer. - ```bash - python3 -m ci.run_build_tester --binary /path/to/camoufox-bin - ``` -- **`tests/patches/`** — one standalone guard per shipped spoofing behaviour - (isolated evaluate, trusted events, fonts, mouse trajectories, touchscreen). - The most direct evidence a Firefox bump did not neuter a patch that still - *applies* cleanly. - ```bash - python3 -m ci.run_patch_guards --binary /path/to/camoufox-bin - ``` -- **Playwright** — upstream playwright-python, fetched fresh at the tag - `ci/versions.py` resolves for the browser, with `ci/skiplist.yml` applied and - `tests/camoufox/` overlaid. This is the automation-contract check, not the - stealth check. It runs **isolated-world first** (what users ship) and re-runs - only the failures with isolation off; those are counted and named as - main-world fallbacks rather than hidden, so the size of the isolated-world - gap is visible per run. - ```bash - make tests # or: python3 -m ci.run_playwright --binary ... - ``` -- **`native-tests/`** — leaks, context lifetime, and the repo's own conventions. -- **`pythonlib/`**, **`service-tester/`** — the Python package and service layer. -- **stealth grade** — `ci/run_sundial.py` reports a letter grade and a count. - Its per-vector detail never leaves that module, because this repo is public. - -The Playwright suite is **not** a fork: `tests/` holds only `patches/` and -`camoufox/`. Do not vendor upstream tests back into it — a deliberate difference -from upstream belongs in `ci/skiplist.yml` with a stated reason. - -`ccache` is enabled in the build config — install it for fast incremental rebuilds (cold ~40 min, incremental ~5 min). - -## Constraints when editing this repo - -- The `camoufox-*/` source directory is regenerated — persist changes as patches, never as edits committed to that tree. -- Keep the `Makefile` diff clean against `main` unless a change genuinely belongs there — dependency setup lives in `scripts/install-deps.sh`, not the Makefile. -- Every PR must be tied to a GitHub issue and pass the full pipeline (see `CONTRIBUTING.md` and `ci/README.md`). +@AGENTS.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 48cd46b7c..f5ead3133 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,6 +9,8 @@ Thanks for your interest in contributing! Here's how to get started. - **Code contributions** — Fork the repo, make your changes, and open a pull request. - **Documentation** — Fixes and improvements to docs are always welcome. +Planned work is in [`ROADMAP.md`](ROADMAP.md). Comment on an item's issue before you start on it. + ## Development Setup See README.md for general setup. For iterative development with frequent rebuilds, install [ccache](https://ccache.dev/) to cache compiled objects: @@ -26,6 +28,8 @@ ccache is already enabled in the build config. A cold build takes the usual ~40 ## Pull Request Rules +The engineering rules in [`AGENTS.md`](AGENTS.md) apply to every change, whether a person or an agent wrote it. They cover less code, no band-aids, no flakes, and docs that ship with the code. + 1. Each pull request must be associated with a Github issue 2. Follow the pull request template 3. Keep commits focused — one logical change per commit. @@ -34,7 +38,7 @@ ccache is already enabled in the build config. A cold build takes the usual ~40 ## Testing Requirements -**CI runs everything, on every pull request.** [`.github/workflows/tests.yml`](.github/workflows/tests.yml) builds the browser from your branch when you touch browser sources (and tests against the published release when you do not), then runs the patch guards, build-tester, the upstream Playwright suite, the leak suite and the stealth check. Branch protection requires exactly one check, **`All tests passed`**, which is green only when every applicable suite is. +**CI runs everything, on every pull request.** [`.github/workflows/tests.yml`](.github/workflows/tests.yml) builds the browser from your branch when you touch browser sources (and tests against the published release when you do not), then runs the Python package tests, the patch guards, build-tester, the upstream Playwright suite, the leak suite and the stealth check. Branch protection requires exactly one check, **`All tests passed`**, which is green only when every applicable suite is. So there is nothing to attach to the pull request by hand. The old process — run the suites locally, screenshot the output, paste it in — was unenforceable: nothing checked that the browser in the screenshot was built from the branch under review. If you want a report in the description anyway, CI leaves one as a comment on the pull request. @@ -45,6 +49,7 @@ python3 -m ci.run_patch_guards --binary /path/to/camoufox-bin python3 -m ci.run_build_tester --binary /path/to/camoufox-bin python3 -m ci.run_playwright --binary /path/to/camoufox-bin # or --shard 3/6 python3 -m ci.run_skiplist_audit --binary /path/to/camoufox-bin +python3 -m ci.run_pythonlib # pythonlib/ python3 -m pytest ci/tests -q # the pipeline's own tests ``` @@ -77,8 +82,7 @@ Tests the **full stack** — the binary and the Python package together — usin ```bash cd service-tester -# Add proxies (one per line, format: user:pass@domain:port) -cp proxies.txt.example proxies.txt # or create manually +# Create proxies.txt: one user:pass@domain:port per line (# comments allowed) ./run_tests.sh ``` @@ -102,7 +106,7 @@ See [`service-tester/README.md`](service-tester/README.md) for full details. Please search existing issues before opening a new one. Include: - Camoufox version -- OS and Python version +- OS, and your Python or Node version - A minimal reproducible example ## Questions diff --git a/Dockerfile b/Dockerfile index 661d47666..0e2230cf6 100644 --- a/Dockerfile +++ b/Dockerfile @@ -12,7 +12,7 @@ RUN apt-get update && apt-get install -y \ # Python python3 python3-dev python3-pip \ # Camoufox build system tools - git p7zip-full golang-go aria2 curl rsync \ + git p7zip-full aria2 curl rsync \ # assets/base.mozconfig only enables ccache `if command -v ccache`, so # without this every container build is a cold build (#698) ccache \ diff --git a/Makefile b/Makefile index e0590465e..d4fb58482 100644 --- a/Makefile +++ b/Makefile @@ -4,13 +4,13 @@ export cf_source_dir := camoufox-$(version)-$(release) ff_source_tarball := firefox-$(version).source.tar.xz -debs := python3 python3-dev python3-pip p7zip-full golang-go msitools wget aria2 libsqlite3-dev -rpms := python3 python3-devel p7zip golang msitools wget aria2 sqlite-devel -pacman := python python-pip p7zip go msitools wget aria2 sqlite +debs := python3 python3-dev python3-pip p7zip-full msitools wget aria2 libsqlite3-dev +rpms := python3 python3-devel p7zip msitools wget aria2 sqlite-devel +pacman := python python-pip p7zip msitools wget aria2 sqlite .PHONY: help fetch fetch-fonts fonts-extract fonts-check fonts-clean setup setup-minimal clean set-target distclean build package \ - build-launcher check-arch revert edits run bootstrap mozbootstrap dir \ - package-linux package-macos package-windows vcredist_arch patch unpatch \ + revert run bootstrap mozbootstrap dir \ + package-linux package-macos package-windows patch unpatch diff \ workspace check-arg edit-cfg ff-dbg tests update-ubo-assets generate-assets-car \ setup-macos-sdk @@ -22,8 +22,6 @@ help: @echo " mozbootstrap - Sets up mach" @echo " dir - Prepare Camoufox source directory with BUILD_TARGET" @echo " revert - Kill all working changes" - @echo " edits - Camoufox developer UI" - @echo " build-launcher - Build launcher" @echo " clean - Remove build artifacts" @echo " distclean - Remove everything including downloads" @echo " build - Build Camoufox" @@ -159,23 +157,10 @@ build: unbusy fi cd $(cf_source_dir) && ./mach build $(_ARGS) -edits: - python3 ./scripts/developer.py $(version) $(release) - -check-arch: - @if ! echo "x86_64 i686 arm64" | grep -qw "$(arch)"; then \ - echo "Error: Invalid arch value. Must be x86_64, i686, or arm64."; \ - exit 1; \ - fi - -build-launcher: check-arch - cd legacy/launcher && bash build.sh $(arch) $(os) - package-linux: fonts-extract python3 scripts/package.py linux \ --includes \ settings/chrome.css \ - settings/camoucfg.jvv \ settings/properties.json \ bundle/fontconfig \ --version $(version) \ @@ -187,7 +172,6 @@ package-macos: fonts-extract python3 scripts/package.py macos \ --includes \ settings/chrome.css \ - settings/camoucfg.jvv \ settings/properties.json \ --version $(version) \ --release $(release) \ @@ -198,7 +182,6 @@ package-windows: fonts-extract python3 scripts/package.py windows \ --includes \ settings/chrome.css \ - settings/camoucfg.jvv \ settings/properties.json \ ~/.mozbuild/vs/VC/Redist/MSVC/*/$(vcredist_arch)/Microsoft.VC*.CRT/*.dll \ --version $(version) \ @@ -206,19 +189,6 @@ package-windows: fonts-extract --arch $(arch) \ --fonts windows macos linux -run-launcher: - rm -rf $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/launch; - make build-launcher arch=x86_64 os=linux; - cp legacy/launcher/dist/launch $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/launch; - $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/launch - -run-pw: - rm -rf $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/launch; - make build-launcher arch=x86_64 os=linux; - python3 scripts/run-pw.py \ - --version $(version) \ - --release $(release) - run: cd $(cf_source_dir) \ && rm -rf ~/.camoufox obj-x86_64-pc-linux-gnu/tmp/profile-default \ @@ -281,8 +251,7 @@ stage-fonts: fonts-extract unbusy: rm -rf $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/camoufox-bin \ - $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/camoufox \ - $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/launch + $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/camoufox path: @realpath $(cf_source_dir)/obj-x86_64-pc-linux-gnu/dist/bin/camoufox-bin diff --git a/README.md b/README.md index be32a7f83..080ee0df1 100644 --- a/README.md +++ b/README.md @@ -359,10 +359,10 @@ Camoufox is a Firefox fork engineered for web scraping and AI agents. It is head * **Optimized for automation** * Human-like mouse movement 🖱️ * Blocks & circumvents ads 🛡️ - * No CSS animations 💨 + * Optional instant animations (`instantAnimations`), so Playwright never waits on one 💨 - Debloated & optimized for memory efficiency ⚡ -- [PyPi package](https://pypi.org/project/camoufox/) for updates & auto fingerprint injection 📦 +- [PyPI package](https://pypi.org/project/camoufox/) for updates & auto fingerprint injection 📦 - Stays up to date with the latest Firefox version 🕓 --- @@ -371,13 +371,13 @@ Camoufox is a Firefox fork engineered for web scraping and AI agents. It is head In Camoufox, data is intercepted at the C++ implementation level, making the changes undetectable through JavaScript inspection. -To spoof individual fingerprint properties, pass a JSON containing properties to spoof to the [Python interface](https://github.com/daijro/camoufox/tree/main/pythonlib#camoufox-python-interface): +To spoof individual fingerprint properties, pass a JSON containing properties to spoof to the [Python interface](pythonlib/): ```py >>> with Camoufox(config={"property": "value"}) as browser: ``` -Config data not set by the user will be automatically populated using [BrowserForge](https://github.com/daijro/browserforge) fingerprints, which mimic the statistical distribution of device characteristics in real-world traffic. +Config data not set by the user is populated from [fpgen](https://github.com/scrapfly/fingerprint-generator), a model of the statistical distribution of device characteristics in real-world traffic. The assembled identity is then checked for coherence, so parts that are each plausible cannot combine into a machine that does not exist. [[See implemented properties](https://camoufox.com/fingerprint/)] @@ -427,7 +427,6 @@ Below is a list of patches and features implemented in Camoufox. - Network headers (Accept-Languages and User-Agent) are spoofed to match the navigator properties - WebRTC IP spoofing at the protocol level - Geolocation, timezone, and locale spoofing -- Battery API spoofing - etc. ### Stealth patches @@ -447,7 +446,7 @@ Below is a list of patches and features implemented in Camoufox. - Automatically uses the correct system fonts for your User Agent - Bundled with Windows, Mac, and Linux system fonts -- Prevents font metrics fingerprinting by randomly offsetting letter spacing +- No glyph-spacing noise: measured text widths are the ones the same font gives on a real machine ### Playwright support @@ -460,7 +459,7 @@ Below is a list of patches and features implemented in Camoufox. - Patches from LibreWolf & Ghostery to help remove telemetry & bloat - Debloat config from PeskyFox, LibreWolf, and others - Speed & network optimizations from FastFox -- Removed all CSS animations +- Animations run on stock timing; `instantAnimations: True` finishes them at once, at the cost of being detectable - Minimalistic theming - etc. @@ -494,10 +493,6 @@ Below is a list of patches and features implemented in Camoufox. ## How Camoufox hides its automation library -> [!WARNING] -> **Current status as of 2026**: -> There has been a year gap in maintenance due to a personal situation. Camoufox has gone down in performance due to the base Firefox version and newly discovered fingerprint inconsistencies. **Camoufox is currently under active development.** - In Camoufox, all of Playwright's internal Page Agent's code is sandboxed and isolated. This makes it impossible for a page to detect the presence of Playwright through Javascript inspection. Normally, Playwright injects some JavaScript into the page such as `window.__playwright__binding__` and to perform actions like querying elements, evaluating javascript, or running init scripts, which can be detected by websites. In Camoufox, these actions are handled in an isolated scope outside of the page. In other words, websites can no longer "see" any JavaScript that Playwright would typically inject. This prevents traces of Playwright altogether. @@ -508,13 +503,15 @@ However, even with hiding its automation library, Camoufox is not immune to inco Anti-bot systems also run client-side scripts to monitor your behavior. For example, they look for patterns in mouse movements, clicks, scrolling, and the timing between actions. - +Cursor paths Camoufox produced with humanize=True, replayed at their recorded speed + +Every dot above is a `mousemove` event the page received from six `page.mouse.move()` calls, replayed at the speed it arrived. Close dots mean the hand slowed down. `scripts/cursor-demo.py` regenerates the figure from a build. Camoufox does not draw its cursor paths. With `humanize=True` it uses [**Cursory**](https://github.com/Vinyzu/cursory) by [Vinyzu](https://github.com/Vinyzu), which holds 2357 mouse movements recorded from real people: it picks a recording whose direction, distance and wander suit the move being made, morphs it onto the requested start and end points, and replays it with that recording's own timing — pauses, overshoots and all. That last part matters as much as the shape. Camoufox previously walked a Bézier curve through two random knots and emitted a point every 10ms. Both halves of that are tells: an analytic curve sampled at a fixed rate has velocity and jerk profiles that separate cleanly from a hand's, and the acceleration came entirely from one easing function, so every movement Camoufox ever made sped up and slowed down the same way. A replayed recording has neither property. -Camoufox ships [cursory-js](https://github.com/JWriter20/cursory-js), a TypeScript port of Cursory, vendored into Juggler at `additions/juggler/input/cursory/`. It reproduces the Python original bit for bit, so a path can be reproduced against `pip install cursory`. **Cursory is LGPLv3-or-later, not MPL-2.0 like the rest of Camoufox**; its licence and full provenance are in `additions/juggler/input/cursory/NOTICE`. +Camoufox ships [cursory-js](https://github.com/JWriter20/cursory-js), a TypeScript port of Cursory, vendored into Juggler at `additions/juggler/input/cursory/`. It reproduces the Python original bit for bit, so a path can be reproduced against `pip install cursory`. **Cursory is LGPLv3-or-later, not MPL-2.0 like the rest of the browser**; its licence and full provenance are in `additions/juggler/input/cursory/NOTICE`. However, this isn't perfect. It may still be detected with sophisticated enough analysis. (WIP for the future) @@ -528,13 +525,13 @@ AI agents need to operate across many sessions without getting flagged or rate-l Even if you are rotating your IP for each running bot instance, web access firewalls can still use machine learning to analyze incoming web traffic to detect if it's abnormal. If the Linux market share was 5%, then suddenly it's 20%, it's a red flag. They will unconditionally require all Linux users to complete a captcha. -Camoufox uses [BrowserForge](https://github.com/daijro/browserforge)'s fingerprint generator to mimic the statistical distribution of device data in real-world traffic. For example, Camoufox will make your browser look like a Linux user 5% of the time. Of that 5%, it will spoof a 2560x1440 screen resolution 9.5% of the time and an Intel HD GPU 27.5% of the time. +Camoufox draws identities from [fpgen](https://github.com/scrapfly/fingerprint-generator), a Bayesian network trained on live traffic, so each device characteristic appears about as often as it does in the real world, and in the combinations real devices produce. ### How can Camoufox be detected? Camoufox can spoof fingerprints with a correct market share. However, **fingerprints must also be internally consistent.** A Windows user agent with an Apple M1 GPU, a MacOS user agent with a Windows DirectX renderer, and a mobile device with a desktop screen resolution are all impossible, and will be flagged for being suspicious. -Of the thousands of possible datapoints that must be changed to create a believable spoofed fingerprint, where each change must be consistent with the others, Camoufox doesn't always succeed. Anti-bot providers test Camoufox over and over again to find even 1 unique inconsistency, then they immediately update their background scripts to test for it. +Every drawn identity passes a coherence check (`pythonlib/camoufox/coherence.py`) that rejects impossible combinations before launch. But of the thousands of datapoints that must agree with each other, Camoufox doesn't always get every one right. Anti-bot providers test Camoufox over and over again to find even 1 unique inconsistency, then they immediately update their background scripts to test for it. --- @@ -550,7 +547,7 @@ Additionally, all injected JavaScript is detectable in some way. Anti-bot system Since Camoufox intercepts calls in the browser's C++ implementation level, all of the hijacked objects and properties appear native. There is no JavaScript hijacking to be detected. -Camoufox also attempts to generate consistent and believable fingerprints with Browserforge as well. However, this can still be detected by complex fingerprint detection methods like mismatching data (as described earlier). +Camoufox also generates consistent and believable fingerprints with fpgen and its coherence check. However, this can still be detected by complex fingerprint detection methods like mismatching data (as described earlier).
@@ -571,7 +568,7 @@ Additionally, Juggler sends its inputs directly through the Firefox's original u

Build System

> [!WARNING] -> The content below is intended for those interested in building & debugging Camoufox. For Playwright usage instructions, see [here](https://github.com/daijro/camoufox/tree/main/pythonlib#camoufox-python-interface). +> The content below is intended for those interested in building & debugging Camoufox. For usage instructions, see [pythonlib](pythonlib/). ### Overview @@ -583,7 +580,7 @@ graph TD subgraph REPO[Camoufox Repository] PATCHES[Fingerprint masking patches] - ADDONS[uBlock & B.P.C.] + ADDONS[uBlock Origin] DEBLOAT[Debloat/optimizations] SYSTEM_FONTS[Win, Mac, Linux fonts] JUGGLER[Patched Juggler] @@ -620,7 +617,7 @@ make dir Before bootstrapping, install the system build dependencies with the helper script. It detects your platform and installs everything the build needs -(Python ≥ 3.11, Rust, `aria2`, `p7zip`, `go`, `msitools`, `wget`, `sqlite`, and +(Python ≥ 3.11, Rust, `aria2`, `p7zip`, `msitools`, `wget`, `sqlite`, and the core build tools) using the appropriate package manager — Homebrew on macOS, or `apt`/`dnf`/`pacman` on Linux: @@ -725,29 +722,27 @@ Build artifacts will now appear written under the `dist/` folder. --- -## Development Tools - -This repo comes with a developer UI under scripts/developer.py: - -``` -make edits -``` - -Patches can be edited, created, removed, and managed through here. +## Working on patches - +`make dir` leaves `camoufox--/` as a git repository with every patch applied. A patch is a diff against a checkpoint in that repository: -### How to make a patch +```bash +# A new patch +make dir # apply every existing patch +make first-checkpoint # mark the starting point +# ...edit files in camoufox-*/, test with `make build` and `make run`... +make diff > patches/my-change.patch -1. In the developer UI, click **Reset workspace**. -2. Make changes in the `camoufox-*/` folder as needed. You can test your changes with `make build` and `make run`. -3. After you're done making changes, click **Write workspace to patch** and save the patch file. +# An existing patch +make dir +make workspace ./patches/x.patch # unapply x, checkpoint, reapply x +# ...edit... +make diff > patches/x.patch +``` -### How to work on an existing patch +`make diff` shows only tracked files, so `git add -N ` any new file first. `make workspace` needs every later patch to leave the hunks of `x.patch` alone. `make patch` and `make unpatch` apply or reverse one patch, and `make revert` resets the tree to unpatched Firefox. -1. In the developer UI, click **Edit a patch**. -2. Select the patch you'd like to edit. Your workspace will be reset to the state of the selected patch. -3. After you're done making changes, hit **Write workspace to patch** and overwrite the existing patch file. +Then run the suites that cover what you changed. [`CONTRIBUTING.md`](CONTRIBUTING.md) says which ones, and [`ci/README.md`](ci/README.md) has the whole pipeline. --- @@ -768,12 +763,12 @@ flowchart TD B -->|Yes| C[Likely bad IP/rate-limiting. If the website fails on both headless and headful mode on the official Firefox distribution, the issue is not with the browser.] B -->|No| D["Run make ff-dbg(1) and build(2) a clean distribution of Firefox. Does the website flag in Firefox **headless** mode(4)?"] D -->|Yes| E["Does the website flag in headful mode(3) AND headless mode(4)?"] - D -->|No| F["Open the developer UI(5), apply config.patch, then rebuild(2). Does the website still flag(3)?"] + D -->|No| F["Apply config.patch(5), then rebuild(2). Does the website still flag(3)?"] E -->|No| G["Enable privacy.resistFingerprinting in the config(6). Does the website still flag(3)?"] E -->|Yes| C G -->|No| H["In the config(6), enable FPP and start omitting overrides until you find the one that fixed the leak."] G -->|Yes| I[If you get to this point, you may need to deobfuscate the Javascript behind the website to identify what it's testing.] - F -->|Yes| K["Open the developer UI, apply the playwright bootstrap patch, then rebuild. Does it still flag?"] + F -->|Yes| K["Apply playwright/0-playwright.patch(5), then rebuild. Does it still flag?"] F -->|No| J["Omit options from camoufox.cfg(6) and rerun(3) until you find the one causing the leak."] K -->|No| M[Juggler needs to be debugged to locate the leak.] K -->|Yes| L[The issue has nothing to do with Playwright. Apply the rest of the Camoufox patches one by one until the one causing the leak is found.] @@ -788,13 +783,20 @@ flowchart TD | (2) | `make build` | Build the source code. | | (3) | `make run` | Runs the built browser. | | (4) | `make run args="--headless https://test.com"` | Run a URL in headless mode. All redirects will be printed to the console to determine if the test passed. | -| (5) | `make edits` | Opens the developer UI. Allows the user to apply/undo patches, and see which patches are currently applied. | +| (5) | `make patch ./patches/.patch` | Apply one patch. `make unpatch` reverses it. | | (6) | `make edit-cfg` | Edit camoufox.cfg in the default system editor. | --- +## Licensing + +- **The browser** (`patches/`, `additions/`, `settings/`, and the build system) is [MPL-2.0](LICENSE), the licence of the Firefox source it modifies. The vendored Cursory trajectories are LGPLv3-or-later (`additions/juggler/input/cursory/NOTICE`). +- **The Python launcher** is MIT ([`pythonlib/LICENSE`](pythonlib/LICENSE)), as it has always been declared on PyPI. + +--- + ## Thanks Debloating & references: @@ -807,6 +809,7 @@ Web scraping & testing: - [Vinyzu/cursory](https://github.com/Vinyzu/cursory): The recorded human mouse trajectories behind `humanize=True`, vendored via [cursory-js](https://github.com/JWriter20/cursory-js) (LGPLv3-or-later — see `additions/juggler/input/cursory/NOTICE`) - [riflosnake/HumanCursor](https://github.com/riflosnake/HumanCursor): The Bézier cursor algorithm Camoufox used before Cursory +- [scrapfly/fingerprint-generator](https://github.com/scrapfly/fingerprint-generator) (fpgen): The device distribution identities are drawn from - [CreepJS](https://github.com/abrahamjuliot/creepjs), [Browserleaks](https://browserleaks.com), [BrowserScan](https://www.browserscan.net/) - Valuable leak testing sites UI theming: diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 000000000..55ecf3334 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,66 @@ +# Roadmap + +What is planned, grouped by area. Items link to their issue. There are no +dates: an item ships when its tests prove it. To pick one up, comment on its +issue first. + +## In progress + +- **TypeScript/JavaScript package on npm**, at parity with `pythonlib` + ([#784](https://github.com/daijro/camoufox/issues/784)). +- **Firefox 155** ([#764](https://github.com/daijro/camoufox/issues/764)). + +## Identity + +- **Draw more of the identity from fpgen.** Navigator, screen, window and + headers come from fpgen today. Its fonts, voices, WebGL parameters, + permissions, WebRTC capabilities and audio hashes are still drawn elsewhere. +- **Capture WebGL data for GPUs Camoufox cannot present yet**: Windows on ARM + (Adreno), Direct3D 10-level hardware, and the Mesa/nouveau and older Intel + Linux drivers. Presets naming them were dropped because nothing recorded + their WebGL parameters. They come back as normal identities once real + parameters for them exist. +- **Reproducible identities across relaunches**: the same seed gives the same + device, including its canvas and audio output + ([#442](https://github.com/daijro/camoufox/issues/442), + [#765](https://github.com/daijro/camoufox/issues/765)). +- **Per-context fonts everywhere**: the default context + ([#757](https://github.com/daijro/camoufox/issues/757)), and `measureText` + agreeing with `@font-face local()` + ([#759](https://github.com/daijro/camoufox/issues/759)). +- **Platform-consistent APIs**: + - speech voices ([#717](https://github.com/daijro/camoufox/issues/717)); + - WebAuthn platform-authenticator availability + ([#718](https://github.com/daijro/camoufox/issues/718)); + - audio output devices ([#768](https://github.com/daijro/camoufox/issues/768)); + - favicon caching ([#577](https://github.com/daijro/camoufox/issues/577)). + +## Automation + +- **Routing in the isolated world**: `route_web_socket` + ([#775](https://github.com/daijro/camoufox/issues/775)) and server-sent events + ([#786](https://github.com/daijro/camoufox/issues/786)). +- **Protocol resilience**: interrupting a runaway `evaluate` + ([#720](https://github.com/daijro/camoufox/issues/720)), and recovering from a + dead Juggler pipe ([#719](https://github.com/daijro/camoufox/issues/719)). + +## Platforms + +- Sandboxes without `ARCH_SET_GS`, such as gVisor + ([#740](https://github.com/daijro/camoufox/issues/740)), and read-only + filesystems ([#572](https://github.com/daijro/camoufox/issues/572)). +- **macOS**: headed windows must not take focus when shown + ([#739](https://github.com/daijro/camoufox/issues/739)). +- **Windows**: + - deterministic generic-font resolution at startup + ([#783](https://github.com/daijro/camoufox/issues/783)); + - clean repaint while resizing + ([#734](https://github.com/daijro/camoufox/issues/734)). + +## Testing + +- **Test the packaged browser in CI, langpacks included.** CI tests an + unpackaged build that carries only `en-US`, so no suite can check another + locale end to end. +- **Run the stealth grade on pull requests from forks.** Forks get no secrets + today, so that check is skipped for them. diff --git a/additions/camoucfg/MaskConfig.hpp b/additions/camoucfg/MaskConfig.hpp index 813323fbd..efc90f9dc 100644 --- a/additions/camoucfg/MaskConfig.hpp +++ b/additions/camoucfg/MaskConfig.hpp @@ -212,18 +212,6 @@ inline std::optional> GetRect( return result; } -inline std::optional> GetInt32Rect( - const std::string& left, const std::string& top, const std::string& width, - const std::string& height) { - if (auto optValue = GetRect(left, top, width, height)) { - std::array result; - std::transform(optValue->begin(), optValue->end(), result.begin(), - [](const auto& val) { return static_cast(val); }); - return result; - } - return std::nullopt; -} - // Helpers for WebGL inline std::optional GetNested(const std::string& domain, diff --git a/additions/juggler/JugglerFrameParent.jsm b/additions/juggler/JugglerFrameParent.jsm deleted file mode 100644 index 9d41842a3..000000000 --- a/additions/juggler/JugglerFrameParent.jsm +++ /dev/null @@ -1,40 +0,0 @@ -"use strict"; - -const { TargetRegistry } = ChromeUtils.importESModule('chrome://juggler/content/TargetRegistry.js'); -const { Helper } = ChromeUtils.importESModule('chrome://juggler/content/Helper.js'); - -const helper = new Helper(); - -export class JugglerFrameParent extends JSWindowActorParent { - constructor() { - super(); - } - - receiveMessage() { } - - async actorCreated() { - // Actors are registered per the WindowGlobalParent / WindowGlobalChild pair. We are only - // interested in those WindowGlobalParent actors that are matching current browsingContext - // window global. - // See https://github.com/mozilla-firefox/firefox/blob/35e22180b0b61413dd8eccf6c00b1c6fac073eee/testing/mochitest/BrowserTestUtils/BrowserTestUtilsParent.sys.mjs#L15 - if (!this.manager?.isCurrentGlobal) - return; - - // Only interested in main frames for now. - if (this.browsingContext.parent) - return; - - this._target = TargetRegistry.instance()?.targetForBrowserId(this.browsingContext.browserId); - if (!this._target) - return; - - this.actorName = `browser::page[${this._target.id()}]/${this.browsingContext.browserId}/${this.browsingContext.id}/${this._target.nextActorSequenceNumber()}`; - this._target.setActor(this); - } - - didDestroy() { - if (!this._target) - return; - this._target.removeActor(this); - } -} diff --git a/additions/juggler/content/JugglerFrameChild.jsm b/additions/juggler/content/JugglerFrameChild.jsm deleted file mode 100644 index af060c57a..000000000 --- a/additions/juggler/content/JugglerFrameChild.jsm +++ /dev/null @@ -1,85 +0,0 @@ -"use strict"; - -const { Helper } = ChromeUtils.importESModule('chrome://juggler/content/Helper.js'); -const { initialize } = ChromeUtils.importESModule('chrome://juggler/content/content/main.js'); - -const Ci = Components.interfaces; -const helper = new Helper(); - -let sameProcessInstanceNumber = 0; - -const topBrowingContextToAgents = new Map(); - -export class JugglerFrameChild extends JSWindowActorChild { - constructor() { - super(); - - this._eventListeners = []; - } - - handleEvent(aEvent) { - const agents = this._agents(); - if (!agents) - return; - if (aEvent.type === 'DOMWillOpenModalDialog') { - agents.channel.pause(); - return; - } - if (aEvent.type === 'DOMModalDialogClosed') { - agents.channel.resumeSoon(); - return; - } - if (aEvent.target === this.document) { - agents.pageAgent.onWindowEvent(aEvent); - agents.frameTree.onWindowEvent(aEvent); - } - } - - _agents() { - return topBrowingContextToAgents.get(this.browsingContext.top); - } - - actorCreated() { - this.actorName = `content::${this.browsingContext.browserId}/${this.browsingContext.id}/${++sameProcessInstanceNumber}`; - - this._eventListeners.push(helper.addEventListener(this.contentWindow, 'load', event => { - this._agents()?.pageAgent.onWindowEvent(event); - })); - - if (this.document.documentURI.startsWith('moz-extension://')) - return; - - // Child frame events will be forwarded to related top-level agents. - if (this.browsingContext.parent) - return; - - let agents = topBrowingContextToAgents.get(this.browsingContext); - if (!agents) { - agents = initialize(this.browsingContext, this.docShell); - topBrowingContextToAgents.set(this.browsingContext, agents); - } - agents.channel.bindToActor(this); - agents.actor = this; - } - - didDestroy() { - helper.removeListeners(this._eventListeners); - - if (this.browsingContext.parent) - return; - - const agents = topBrowingContextToAgents.get(this.browsingContext); - // The agents are already re-bound to a new actor. - if (agents?.actor !== this) - return; - - topBrowingContextToAgents.delete(this.browsingContext); - - agents.channel.resetTransport(); - agents.pageAgent.dispose(); - agents.frameTree.dispose(); - } - - receiveMessage() { } -} - diff --git a/additions/juggler/content/hidden-scrollbars.css b/additions/juggler/content/hidden-scrollbars.css deleted file mode 100644 index 26fc0db76..000000000 --- a/additions/juggler/content/hidden-scrollbars.css +++ /dev/null @@ -1,7 +0,0 @@ -/* This Source Code Form is subject to the terms of the Mozilla Public - * License, v. 2.0. If a copy of the MPL was not distributed with this file, - * You can obtain one at http://mozilla.org/MPL/2.0/. */ - -* { - scrollbar-width: none !important; -} diff --git a/additions/juggler/jar.mn b/additions/juggler/jar.mn index ca8d72edb..44fbaf1b2 100644 --- a/additions/juggler/jar.mn +++ b/additions/juggler/jar.mn @@ -27,19 +27,16 @@ juggler.jar: content/ChannelEventSink.sys.mjs (ChannelEventSink.sys.mjs) content/TargetRegistry.js (TargetRegistry.js) content/SimpleChannel.js (SimpleChannel.js) - content/JugglerFrameParent.jsm (JugglerFrameParent.jsm) content/JugglerFrameParent.sys.mjs (JugglerFrameParent.sys.mjs) content/protocol/PrimitiveTypes.js (protocol/PrimitiveTypes.js) content/protocol/Protocol.js (protocol/Protocol.js) content/protocol/Dispatcher.js (protocol/Dispatcher.js) content/protocol/PageHandler.js (protocol/PageHandler.js) content/protocol/BrowserHandler.js (protocol/BrowserHandler.js) - content/JugglerFrameChild.jsm (content/JugglerFrameChild.jsm) content/JugglerFrameChild.sys.mjs (content/JugglerFrameChild.sys.mjs) content/content/main.js (content/main.js) content/content/FrameTree.js (content/FrameTree.js) content/content/PageAgent.js (content/PageAgent.js) content/content/Runtime.js (content/Runtime.js) content/content/WorkerMain.js (content/WorkerMain.js) - content/content/hidden-scrollbars.css (content/hidden-scrollbars.css) diff --git a/assets/humanize-cursor.svg b/assets/humanize-cursor.svg new file mode 100644 index 000000000..8441aeda7 --- /dev/null +++ b/assets/humanize-cursor.svg @@ -0,0 +1,571 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +554 mousemove events from 6 page.mouse.move() calls, humanize=True, replayed at recorded speed + diff --git a/build-tester/README.md b/build-tester/README.md index 960eda131..26aa06165 100644 --- a/build-tester/README.md +++ b/build-tester/README.md @@ -4,7 +4,7 @@ Tests a raw Camoufox binary (Firefox) directly against the same antibot-detectio ## Prerequisites -- Python 3.9+ +- Python 3.10+ (what `pythonlib/` requires) - Node.js (for building the TypeScript checks bundle via `esbuild`, first run only) ## Setup @@ -28,6 +28,13 @@ python scripts/run_tests.py [options] python scripts/run_tests.py /path/to/camoufox-bin/camoufox ``` +`./run_tests.sh [options]` does the setup for you: it installs the +npm dependencies and creates `.venv/` with this tree's `pythonlib/` on first +run, then calls `run_tests.py`. It forwards every option below except `--json`. + +In CI the suite runs through `python3 -m ci.run_build_tester --binary ` +from the repository root (see [`ci/README.md`](../ci/README.md)). + ## Options ``` @@ -36,6 +43,7 @@ python scripts/run_tests.py /path/to/camoufox-bin/camoufox --secret KEY HMAC signing key for certificate --save-cert PATH Save certificate text to a file --no-cert Skip certificate generation + --json PATH Write the full machine-readable result tree to PATH ``` ## What It Tests @@ -56,7 +64,7 @@ Each profile is scored across: | Firefox APIs | Firefox-specific API presence | | Cross-Signal | Consistency across navigator, screen, etc. | | CSS Fingerprint | CSS rendering fingerprint | -| Canvas Noise | Canvas hash uniqueness and stability | +| Canvas Noise | Canvas output is identical across renders (no random noise) | | WebGL Render | WebGL rendering hash | | Audio Integrity | AudioContext fingerprint | | Font Platform | OS-consistent font availability | diff --git a/build-tester/run_tests.sh b/build-tester/run_tests.sh index fee02fdb5..01bd4301c 100755 --- a/build-tester/run_tests.sh +++ b/build-tester/run_tests.sh @@ -19,7 +19,7 @@ while [[ $# -gt 0 ]]; do ;; -h|--help) echo "Usage: $0 [--profile-count N] [--secret KEY] [--save-cert PATH] [--no-cert]" - echo " e.g. $0 ../camoufox-146.0.1-beta.25/obj-aarch64-apple-darwin/dist/Camoufox.app" + echo " e.g. $0 ../camoufox-152.0.4-beta.31/obj-x86_64-pc-linux-gnu/dist/bin/camoufox-bin" exit 0 ;; -*) diff --git a/build-tester/scripts/generate-presets.py b/build-tester/scripts/generate-presets.py index 9f155584d..2aaf072e7 100644 --- a/build-tester/scripts/generate-presets.py +++ b/build-tester/scripts/generate-presets.py @@ -29,9 +29,7 @@ def convert_preset(ctx): }, 'camouConfig': config, 'profileConfig': { - 'fontSpacingSeed': config.get('fonts:spacing_seed', 0), 'audioSeed': config.get('audio:seed', 0), - 'canvasSeed': config.get('canvas:seed', 0), 'screenWidth': screen.get('width', 1920), 'screenHeight': screen.get('height', 1080), 'screenColorDepth': screen.get('colorDepth', 24), diff --git a/build-tester/scripts/presets.py b/build-tester/scripts/presets.py index 34b474981..f00ae72e2 100644 --- a/build-tester/scripts/presets.py +++ b/build-tester/scripts/presets.py @@ -30,9 +30,7 @@ def convert_preset(ctx: dict) -> dict: }, "camouConfig": config, "profileConfig": { - "fontSpacingSeed": config.get("fonts:spacing_seed", 0), "audioSeed": config.get("audio:seed", 0), - "canvasSeed": config.get("canvas:seed", 0), "screenWidth": screen.get("width", 1920), "screenHeight": screen.get("height", 1080), "screenColorDepth": screen.get("colorDepth", 24), diff --git a/build-tester/src/lib/checks/extended.ts b/build-tester/src/lib/checks/extended.ts index 35e11c1d4..ed59dca85 100644 --- a/build-tester/src/lib/checks/extended.ts +++ b/build-tester/src/lib/checks/extended.ts @@ -1118,12 +1118,15 @@ export async function runExtendedChecks(): Promise< }; })(); - // hardwareConcurrency should be a common value + // hardwareConcurrency should be a common value. 18 and 22 are real: + // Intel Meteor Lake laptops (Core Ultra 5 125H = 18 threads, Core Ultra 7 + // 155H = 22), and both are in the recorded presets. Every value pythonlib + // can present must be here -- ci/tests/test_ci.py checks it. result.trashDetection.plausibleHWC = (() => { const hwc = navigator.hardwareConcurrency; const common = [ - 1, 2, 4, 6, 8, 10, 12, 14, 16, 20, 24, 28, 32, 36, 40, 48, 56, 64, - 96, 128, 256, + 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 28, 32, 36, 40, 48, 56, + 64, 96, 128, 256, ]; const isCommon = common.indexOf(hwc) !== -1; return { diff --git a/build-tester/src/lib/checks/index.ts b/build-tester/src/lib/checks/index.ts index 0d61e9fac..56705d08b 100644 --- a/build-tester/src/lib/checks/index.ts +++ b/build-tester/src/lib/checks/index.ts @@ -7,7 +7,6 @@ export interface PhaseResult { } const SELF_DESTRUCT_FUNCTIONS = [ - "setFontSpacingSeed", "setAudioFingerprintSeed", "setTimezone", "setScreenDimensions", diff --git a/build-tester/src/lib/types.ts b/build-tester/src/lib/types.ts index 6c0c21d4e..0564e2ad8 100644 --- a/build-tester/src/lib/types.ts +++ b/build-tester/src/lib/types.ts @@ -95,8 +95,6 @@ export interface ProfileConfig { webglVendor: string; webglRenderer: string; audioSeed: number; - canvasSeed: number; - fontSpacingSeed: number; fontList: string[]; speechVoices?: string[]; } diff --git a/ci/README.md b/ci/README.md index c17cfcfd0..269de0d0c 100644 --- a/ci/README.md +++ b/ci/README.md @@ -6,16 +6,17 @@ to test a specific browser version, so there is one definition of "the tests pass", not two. ``` -resolve ──┬─ static ────────── tribal rules, skiplist, self-tests (seconds) - ├─ pythonlib ─────── the package's own tests (a minute) - └─ build ──┬─ playwright × 6 shards (conformance + our own) - ├─ skiplist audit ───── every skip must still fail - ├─ native ───────────── leaks, contexts (ours) - ├─ patch guards ─────── one per spoofing patch - ├─ build-tester ─────── 8 fingerprint profiles - └─ sundial ──────────── stealth grade (off: see below) - │ - summary ──► one comment on the PR +resolve ── static ─────────────── lint, tribal rules, skiplist, self-tests (seconds) + └─ pythonlib ─────── the package's own tests (a minute) + └─ build or fetch ─┬─ patch guards ─────── one per spoofing patch, + skiplist audit + ├─ build-tester ─────── 8 fingerprint profiles + └─ once guards and build-tester pass: + ├─ playwright × 6 shards (conformance + our own) + ├─ native ───────── leaks, contexts, crash recovery + ├─ sundial ──────── stealth grade + └─ growth ───────── memory growth (scheduled only) + │ + summary ──► one comment on the PR ``` ## Which browser, which suite @@ -260,7 +261,7 @@ isolation itself regresses. entry has to claim a test cannot pass in *either* world, or the suite would have counted it as a fallback rather than a failure. -Ten tests are deselected outright by [`ci/skiplist.yml`](skiplist.yml), which +Fourteen tests are deselected outright by [`ci/skiplist.yml`](skiplist.yml), which requires a stated reason per entry — `ci/summarize.py` fails the run on an unreasoned one. @@ -273,19 +274,21 @@ leaving them bare. `ci/run_skiplist_audit.py` now runs every entry with the skiplist disabled and **fails the build if a skipped test passes**. It is cheap precisely because a -correct skiplist is short — ten tests, a few seconds — and it is what keeps the +correct skiplist is short — twelve tests, a few seconds — and it is what keeps the list from drifting back into a place failing tests go to disappear. ```bash python3 -m ci.run_skiplist_audit --binary /path/to/camoufox-bin ``` -What remains after the audit, 10 tests: two `test_click.py` tests where -Playwright's stable-position wait races the humanized travel time; six -client-certificate tests (async and sync) that need the **browser** to present a -certificate during the TLS handshake — the two that go through the Node driver's -own request context instead pass, and are not skipped; and the two upstream -expectations that encode a stock-Firefox quirk, replaced by `tests/camoufox/`. +What remains, 12 tests: two `test_keyboard.py` +tests that assert a shifted character arrives without Shift, which Camoufox +presses as a real keyboard would; six client-certificate tests (async and sync) +that need the **browser** to present a certificate during the TLS handshake — +the two that go through the Node driver's own request context instead pass, and +are not skipped; two upstream expectations that encode a stock-Firefox quirk, +replaced by `tests/camoufox/`; and two popup tests that rely on Playwright +shipping Firefox's popup blocker off, which Camoufox keeps on. That client-certificate split is the audit earning its place. The entry was first written as a whole module, because on a local machine all five fail — @@ -308,6 +311,14 @@ authority for what fails; a local run is a hypothesis.** and per-context injection silently degrades to process-global — which passes every single-context test there is. It has happened here before (commit `d17c887`, "fix screen size leak in contexts"). +- **Crashes.** Kill the browser, the X server, a content process or the driver + mid-run, then check that teardown does not hang, nothing leaks, and a fresh + launch still works (`test_crash_recovery.py`). +- **Memory growth.** Drive one mechanism (iframes, canvas readback, WebGL + contexts, workers, script compilation, font measurement) N and 4N times and + compare the growth: a bounded cost stays flat, a per-iteration leak scales + (`test_memory_growth.py`). It takes over half an hour, so it runs on the + schedule and on demand (the `growth` job, `--subset growth`), not in the gate. - **Settled decisions.** `ci/tribal-rules.yml` lists choices this project already made, each with the issue or PR that made it, and `native-tests/test_tribal_rules.py` asserts them. A comment explaining a @@ -529,8 +540,8 @@ Each tier gates the next, so a two-second lint failure never reaches the build: 1 unit pythonlib ~1 min 2 browser build (patches/additions/settings/assets/upstream.sh/Makefile/scripts changed) fetch (anything else -- driver changes test against the published release) -3a smoke patch guards, build-tester ~15 min -3b full Playwright x2, leaks, stealth ~40 min +3a smoke patch guards, skiplist audit, build-tester ~15 min +3b full Playwright x6, leaks, stealth ~40 min 4 gate the required check ``` @@ -598,7 +609,8 @@ whole pull request. `ci.run_prepare` runs `setup-minimal` → `dir` → text reads as transient. A failed patch hunk or a compile error still fails on the first attempt — retrying a broken tree only spends a runner to reach the same answer, and a retry loop that swallows a real breakage turns a red build -into a slow red build. +into a slow red build. The release workflow (`build.yml`) prepares its tree the same +way, so a tagged build gets the same hardening. > One consequence of `cancel-in-progress`: pushing to a branch cancels its > running build. That is right while iterating, but a 70-minute build will not @@ -607,16 +619,23 @@ into a slow red build. ## Running a piece by hand ```bash -python3 -m ci.run_playwright --binary path/to/camoufox-bin -python3 -m ci.run_playwright --binary path/to/camoufox-bin --shard 3/6 -python3 -m ci.run_native --subset rules # no browser needed -python3 -m ci.run_native --subset browser --binary path/to/camoufox-bin -python3 -m ci.run_sundial --binary path/to/camoufox-bin -python3 -m ci.summarize --results-dir .ci-work/results +python3 -m ci.run_prepare # make setup-minimal, dir, mozbootstrap +python3 -m ci.run_build +python3 -m ci.run_pythonlib # no browser needed +python3 -m ci.run_patch_guards --binary path/to/camoufox-bin +python3 -m ci.run_build_tester --binary path/to/camoufox-bin +python3 -m ci.run_skiplist_audit --binary path/to/camoufox-bin +python3 -m ci.run_playwright --binary path/to/camoufox-bin +python3 -m ci.run_playwright --binary path/to/camoufox-bin --shard 3/6 +python3 -m ci.run_native --subset rules # no browser needed +python3 -m ci.run_native --subset browser --binary path/to/camoufox-bin +python3 -m ci.run_native --subset growth --binary path/to/camoufox-bin +python3 -m ci.run_sundial --binary path/to/camoufox-bin +python3 -m ci.summarize --results-dir .ci-work/results ``` -Each writes one result file to `.ci-work/results/`. `ci/summarize.py` folds the -shards, decides, and renders the table. A required suite that produced no result +Each suite runner writes one result file to `.ci-work/results/` (`run_prepare` +writes none). `ci/summarize.py` folds the shards, decides, and renders the table. A required suite that produced no result file is a **failure**, never a skip — otherwise deleting a job would be the cheapest way to a green tick. diff --git a/ci/browser_inputs.py b/ci/browser_inputs.py index 1ee10087d..dc448fdfa 100644 --- a/ci/browser_inputs.py +++ b/ci/browser_inputs.py @@ -82,6 +82,7 @@ NON_NATIVE_SCRIPTS = frozenset( { "scripts/clean-fingerprint-data.py", + "scripts/cursor-demo.py", } ) diff --git a/ci/run_build_tester.py b/ci/run_build_tester.py index 9e01846ac..894f3f8fc 100644 --- a/ci/run_build_tester.py +++ b/ci/run_build_tester.py @@ -33,30 +33,12 @@ # this whole suite exists to catch, so a single collision here is fatal. _MUST_VARY = ("uniqueAudio", "uniqueTimezones") -# Canvas belongs in _MUST_VARY and is not there yet, because it does not -# currently hold. Measured across 24 profiles in 3 runs against -# v152.0.4-beta.30, once the canvas fingerprint was actually being hashed -# (it had been a 100-character prefix of the data URL, which compared equal -# for almost anything): -# -# audio 24 distinct / 24 samples every one unique -# canvas 16 distinct / 24 samples values recurring across runs -# macOS 7/12, Linux 9/12 one value seen three times -# -# Same hash, same harness, same runs -- so this is not the measurement. Two -# contexts share a canvas fingerprint roughly a third of the time, which matches -# the rate at which CI flagged it. Gating on it would fail about one run in -# three for a real reason nobody has fixed yet, so it is reported every run and -# tracked here rather than quietly dropped or quietly tolerated. -# -# See ci/tribal-rules.yml: canvas-noise-entropy-is-lower-than-audio. -_TRACKED_LOW_ENTROPY = ("uniqueCanvas",) - -# Values drawn from the preset pool. Three draws from a pool of a dozen collide -# regularly -- that is the birthday paradox, not a leak, and the pools are -# deliberately small because they hold real devices. Counted and reported, -# never fatal on their own. -_MAY_COLLIDE = ("uniqueFonts", "uniqueScreens", "uniqueVoices", "uniqueWebGL") +# Values that follow the device rather than the context. fonts, screens, voices +# and WebGL are drawn from a pool of real devices, and three draws from a pool +# of a dozen collide regularly -- the birthday paradox, not a leak. The canvas +# is rendered, not noised (ci/tribal-rules.yml: canvas-is-not-noised), so it +# follows the fonts and GPU the same way. Counted and reported, never fatal. +_MAY_COLLIDE = ("uniqueCanvas", "uniqueFonts", "uniqueScreens", "uniqueVoices", "uniqueWebGL") # Properties of the operating system. Every macOS context reports MacIntel and # every Linux one reports Linux x86_64, because that is what those systems @@ -169,7 +151,7 @@ def uniqueness(full: dict) -> Dict[str, List[str]]: Only `leaks` and `not_constant` should fail a build. """ out: Dict[str, List[str]] = { - "leaks": [], "noise": [], "low_entropy": [], "absent": [], "not_constant": [] + "leaks": [], "noise": [], "absent": [], "not_constant": [] } for group, stats in (full.get("crossProfile") or {}).items(): @@ -180,7 +162,7 @@ def uniqueness(full: dict) -> Dict[str, List[str]]: def describe(key: str) -> str: return f"{group}.{key} ({stats.get(key)}/{total} distinct)" - for key in _MUST_VARY + _MAY_COLLIDE + _TRACKED_LOW_ENTROPY: + for key in _MUST_VARY + _MAY_COLLIDE: value = stats.get(key) if not isinstance(value, int): continue @@ -190,13 +172,7 @@ def describe(key: str) -> str: # collision reports a leak where there is no data at all. out["absent"].append(describe(key)) elif value < total: - if key in _MUST_VARY: - bucket = "leaks" - elif key in _TRACKED_LOW_ENTROPY: - bucket = "low_entropy" - else: - bucket = "noise" - out[bucket].append(describe(key)) + out["leaks" if key in _MUST_VARY else "noise"].append(describe(key)) for key in _MUST_MATCH: value = stats.get(key) @@ -278,19 +254,9 @@ def main(argv: Optional[List[str]] = None) -> int: allowed = int(cfg.get("allow_uniqueness_collisions", 0)) result.metrics["uniqueness"] = slots - if slots["low_entropy"]: - result.note( - "KNOWN, UNFIXED -- per-context values that collide more often than they should: " - + ", ".join(slots["low_entropy"]) - + ". Measured 16 distinct canvas fingerprints in 24 samples where audio gave " - "24/24, so two contexts are linkable by canvas roughly a third of the time. " - "Reported every run, not gated, because it is real and unfixed. See " - "ci/tribal-rules.yml: canvas-noise-entropy-is-lower-than-audio." - ) - if slots["noise"]: result.note( - f"{len(slots['noise'])} preset-pool collision(s), not gated: " + f"{len(slots['noise'])} device-level collision(s), not gated: " + ", ".join(slots["noise"]) ) if slots["absent"]: diff --git a/ci/run_prepare.py b/ci/run_prepare.py index 3608b1a93..9d8925b77 100644 --- a/ci/run_prepare.py +++ b/ci/run_prepare.py @@ -16,7 +16,7 @@ must still fail on the first try, because retrying those only wastes a runner. The retry lives here rather than in the Makefile so the Makefile diff stays -clean against upstream (see CLAUDE.md) and so the auto-update harness gets the +clean against upstream (see AGENTS.md) and so the auto-update harness gets the same behaviour without duplicating it in a workflow. Run: diff --git a/ci/skiplist.yml b/ci/skiplist.yml index e919c7e13..1f80432d7 100644 --- a/ci/skiplist.yml +++ b/ci/skiplist.yml @@ -42,28 +42,15 @@ skip: # # That breaks far less than this section used to claim. test_click.py, # test_check.py, test_fill.py, test_focus.py, test_dispatch_event.py and - # test_element_handle.py were skipped wholesale -- 160 tests -- and 158 of - # them pass. Only the two below actually fail, and they fail for one specific - # reason rather than for "humanized input" in general. + # test_element_handle.py were skipped wholesale -- 160 tests -- and all of + # them pass. The last two, test_click.py's stable-position tests, were blamed + # on humanized travel time; they failed because every animation finished at + # once, so the element Playwright waited on never moved. With animations on + # stock timing (instantAnimations is now opt-in) they pass too. # # See docs/input-dispatch.md. tests/patches/humanize-mouse-trajectory.py and # tests/patches/input-ack-backstop.py assert the behaviour that replaces them. - - test: tests/async/test_click.py::test_wait_for_stable_position - reason: >- - Upstream animates a button over 500ms and asserts click() waits for it to - stop moving, landing at offset 300. Camoufox clicks at 100 -- mid-flight. - Playwright's stable-position wait polls the bounding box and then dispatches - instantly; the humanized path spends real time travelling to the target, so - the element has moved on by the time the press lands. Narrow and real: the - other 68 tests in this module pass. - - - test: tests/async/test_click.py::test_timeout_waiting_for_stable_position - reason: >- - The same stable-position wait, asserted from the other side -- upstream - expects a timeout error when the element never settles, and Camoufox does - not raise one. - - test: tests/async/test_keyboard.py::test_should_send_proper_codes_while_typing reason: >- Typing "!" on a US layout takes Shift, and upstream asserts the three events diff --git a/ci/tests/test_ci.py b/ci/tests/test_ci.py index b6a56e6e1..91e2f7d68 100644 --- a/ci/tests/test_ci.py +++ b/ci/tests/test_ci.py @@ -237,11 +237,11 @@ def test_version_parsing(): def test_upstream_sh_roundtrip_preserves_comments(tmp_path): path = tmp_path / "upstream.sh" - path.write_text("# a comment\nversion=152.0.4\nrelease=beta.31\nclosedsrc_rev=1.0.0\n") + path.write_text("# a comment\nversion=152.0.4\nrelease=beta.31\nextra=1\n") write_upstream_sh({"version": "153.0.4", "release": "beta.32"}, path) text = path.read_text() assert "# a comment" in text - assert "closedsrc_rev=1.0.0" in text + assert "extra=1" in text assert read_upstream_sh(path)["version"] == "153.0.4" @@ -602,7 +602,7 @@ def _cross(**kw): def test_a_shared_per_context_value_is_a_leak(): - """audio, canvas and timezone are derived per context. + """audio and timezone are derived per context. Two contexts sharing one is the failure this whole suite exists to catch. """ @@ -614,19 +614,14 @@ def test_a_shared_per_context_value_is_a_leak(): assert not out["noise"] -def test_canvas_collisions_are_tracked_but_do_not_gate(): - """Canvas belongs in must-vary and does not hold there yet. - - Measured 16 distinct canvas fingerprints in 24 samples where audio gave - 24/24 -- so two contexts collide about a third of the time. Gating would - fail one run in three for a real, unfixed reason; silence would lose the - finding. It gets its own bucket and is reported every run. - """ +def test_a_shared_canvas_is_noise_not_a_leak(): + """The canvas is rendered, not noised (#528), so contexts with the same + fonts and GPU draw the same image, as two real machines would.""" from ci.run_build_tester import uniqueness out = uniqueness(_cross(uniqueCanvas=2)) - assert out["low_entropy"] == ["macPerContext.uniqueCanvas (2/3 distinct)"] - assert not out["leaks"] and not out["noise"] + assert out["noise"] == ["macPerContext.uniqueCanvas (2/3 distinct)"] + assert not out["leaks"] def test_a_shared_preset_value_is_noise_not_a_leak(): @@ -2584,3 +2579,47 @@ def test_group_timeout_is_shorter_than_the_job_timeout(): # Four times the slowest healthy invocation measured (296s); below that it # starts cutting slow-but-working groups short. assert default >= 900 + + +# --------------------------------------------------------------------------- +# build-tester agrees with the identities pythonlib can present +# --------------------------------------------------------------------------- + + +def test_build_tester_accepts_every_core_count_pythonlib_presents(): + """A real identity must not fail build-tester's plausibility check. + + build-tester's plausibleHWC list lacked 18 and 22 -- both real (Intel Meteor + Lake laptops) and both in the recorded presets -- so a run that drew one of + the two Linux presets reporting 22 failed. About one run in eleven, on any + pull request. The list follows the data, not the other way round. + """ + repo = pathlib.Path(__file__).resolve().parents[2] + source = (repo / "build-tester/src/lib/checks/extended.ts").read_text(encoding="utf-8") + block = source[source.index("plausibleHWC"):] + listed = re.search(r"const common = \[([^\]]*)\]", block) + assert listed, "plausibleHWC's list of common core counts was not found" + accepted = {int(n) for n in re.findall(r"\d+", listed.group(1))} + + presented = set() + lib = repo / "pythonlib/camoufox" + for name in ("fingerprint-presets.json", "fingerprint-presets-v150.json"): + data = json.loads((lib / name).read_text(encoding="utf-8")) + for rows in data.get("presets", {}).values(): + for row in rows: + hwc = row.get("navigator", {}).get("hardwareConcurrency") + if isinstance(hwc, int): + presented.add(hwc) + table = re.search( + r"^PLAUSIBLE_CORE_COUNTS = \(([^)]*)\)", + (lib / "fingerprints.py").read_text(encoding="utf-8"), + re.M, + ) + assert table, "PLAUSIBLE_CORE_COUNTS was not found in fingerprints.py" + presented |= {int(n) for n in re.findall(r"\d+", table.group(1))} + + missing = sorted(presented - accepted) + assert not missing, ( + f"build-tester's plausibleHWC rejects core counts pythonlib presents: {missing}. " + "Add them to the list in build-tester/src/lib/checks/extended.ts." + ) diff --git a/ci/tribal-rules.yml b/ci/tribal-rules.yml index 864c9a5d2..73cc5b176 100644 --- a/ci/tribal-rules.yml +++ b/ci/tribal-rules.yml @@ -159,28 +159,63 @@ rules: # Measurement # ------------------------------------------------------------------------- - - id: canvas-noise-entropy-is-lower-than-audio - title: Two contexts share a canvas fingerprint far more often than they should - check: manual # an open finding, not a settled decision + - id: animations-run-on-stock-timing + title: Animations run on stock timing unless the caller opts into instantAnimations + check: automated + evidence: + - "measured on v152.0.4-beta.31: el.animate(frames, 1000).effect.getComputedTiming().duration returned 0, a 500ms transition 0" + - "the finished promise still resolved after ~1000ms, so the only saving was Playwright's stability wait" + rationale: >- + Instant animations were the default so Playwright never waited on one, + but any page reads the zero duration back in one line. Reporting the + real duration while not rendering the animation does not hide it + either: getComputedStyle or getBoundingClientRect sampled mid-animation + returns the end state. So the saving costs stealth, and it is the + caller's choice: `instantAnimations: True` turns it on and warns. + + - id: spoofed-voices-speak + title: speak() on a spoofed voice starts, then ends after the text's duration + check: automated + evidence: + - "measured on v152.0.4-beta.31: speak() on a spoofed voice fired `error` after 3ms" + - "voices:fakeCompletion, the opt-out, fired start and end in the same tick" + rationale: >- + A spoofed voice has no engine behind it, and a real voice never errors + on a plain utterance or ends the instant it starts. So the browser + speaks it silently for as long as the text takes at ~150 words per + minute. The two config keys that chose between the two tells are gone. + + - id: no-glyph-spacing-noise + title: Text is shaped exactly as stock Firefox shapes it; there is no spacing seed + check: automated + evidence: + - "#779 (6daae88): measured +1 px per ~100 glyphs and fractional deltas on every measureText; defaulted the seed to 0" + - "issue #741 / ad697a8: the perturbation detached combining marks in Thai, Lao, Arabic, Devanagari and Hebrew" + rationale: >- + The feature added a seeded amount to every glyph advance so that text + widths differed per context. No real machine produces those widths: + the same font on the same OS measures the same everywhere, so a width + that matches no real installation is a fingerprint, not a disguise. It + was defaulted off in #779 and kept as an opt-in, but an opt-in whose + only effect is to become detectable is not a feature, so the manager, + the window setter, the shaper hook and the config key are gone. + + - id: canvas-is-not-noised + title: The canvas is rendered, not noised, and there is no canvas seed + check: automated evidence: - - "24 profiles across 3 runs on v152.0.4-beta.30, once the canvas was actually hashed" - - "audio 24 distinct / 24 samples; canvas 16 / 24, values recurring across independent runs" - - "macOS 7 distinct of 12, Linux 9 of 12; one value seen three times" - - "matches the rate at which CI flagged cross-profile canvas collisions" + - "PR #528 (e4528a2): 'Disable Canvas Noise' -- removed the canvas noise patch" + - "issue #721: canvas:seed declared and passed through, but nothing consumed it" + - "24 profiles across 3 runs on v152.0.4-beta.30: canvas 16 distinct of 24, audio 24 of 24" rationale: >- - Audio and canvas are both noised per context, by the same machinery, and - were measured in the same runs with the same hash. Audio came back - completely unique; canvas did not, and its values repeat across runs that - share nothing. So two contexts in one browser are linkable by canvas - roughly a third of the time, which is the property per-context spoofing - exists to prevent. - This was invisible until the fingerprint stopped being a 100-character - prefix of a data URL, which compared equal for almost anything. It is - recorded as an open question rather than a decision: it may be that canvas - output is deliberately tied to the device preset rather than to - canvas:seed, in which case the seed is not doing what its name suggests. - Not gated, because failing one run in three helps nobody, and not dropped, - because it is real. + No stock Firefox perturbs its canvas output, so noise is itself a tell: + a page that renders the same image twice and gets two answers has found + a spoofing browser. A context's canvas therefore follows what really + determines it on a real machine, the GPU and the fonts, and two contexts + that share those share a canvas. That is why build-tester counts canvas + collisions with the device-level values and does not gate on them. The + launchers generated a canvas:seed for three releases after the patch + that read it was gone; it is not generated or declared any more. - id: canvas-fingerprint-is-hashed-not-truncated title: The canvas check hashes pixels, not a prefix of the data URL @@ -197,8 +232,7 @@ rules: contexts whose canvas differed everywhere else read as identical. Worse, the stability check compared the same prefix between two collections, so it passed on PNG headers and could not have noticed non-deterministic - noise. getImageData is the surface Camoufox noises and the one a - fingerprinter reads; hash that. + output. getImageData is the surface a fingerprinter reads; hash that. # ------------------------------------------------------------------------- # Teardown diff --git a/docs/FONTS.md b/docs/FONTS.md index dbb65ded7..1a30ac09c 100644 --- a/docs/FONTS.md +++ b/docs/FONTS.md @@ -214,7 +214,7 @@ and `font-hijacker.patch` does not activate the bundle. On a Windows host the Win11 marker families are *subtracted* when the host cannot render them, rather than added when it can; claiming a marker the host lacks is the leak. -`pythonlib/tests/test_font_distribution.py` (29 tests) covers this: base +`pythonlib/tests/test_font_distribution.py` covers this: base completeness and weights, per-unit probability, bundle atomicity, à-la-carte sizing, locale gating, determinism, renderable-only, and that the draw actually varies (distinct lists, no single list dominating). Tolerances are binomial, at diff --git a/docs/MEDIA-DEVICES.md b/docs/MEDIA-DEVICES.md index e82bd3284..ebdddb14e 100644 --- a/docs/MEDIA-DEVICES.md +++ b/docs/MEDIA-DEVICES.md @@ -24,8 +24,10 @@ Camoufox reproduces exactly that for a spoofed machine: - `getUserMedia()` therefore succeeds iff the identity has the requested device kind (a claimed camera captures the fake engine's test pattern; a camera-less identity gets `NotFoundError`, like a real machine without one). -- `media.navigator.permission.fake = true` (camoufox.cfg) makes the fake - devices count as capturing, which is what exposes labels after a grant. +- The patch treats the identity's devices (`LocalMediaDevice::IsIdentityDevice()`) + as real hardware, so they get the permission prompt, count as capturing + and expose their labels after a grant. `media.navigator.permission.fake` + stays off, as in stock Firefox, because a page can detect it. ## Config keys diff --git a/docs/beta-testing-ff146.md b/docs/beta-testing-ff146.md deleted file mode 100644 index fc8810f76..000000000 --- a/docs/beta-testing-ff146.md +++ /dev/null @@ -1,64 +0,0 @@ -# Testing Firefox 146 Beta - -This guide explains how to test the experimental Firefox 146 build of Camoufox. - -> **Note:** The FF146 build is experimental and may contain bugs. For a stable production version, use branch `releases/135`. - -## Build from Source - -1. Clone the repository: -```bash -git clone --depth 1 https://github.com/daijro/camoufox -cd camoufox -``` - -2. Set up the build environment: -```bash -make dir -make bootstrap # only needed once -``` - -3. Build for your target platform: -```bash -python3 multibuild.py --target --arch -``` - -| Parameter | Options | -|-----------|---------| -| `--target` | `linux`, `windows`, `macos` | -| `--arch` | `x86_64`, `arm64`, `i686` | - -Build artifacts will appear in the `dist/` folder. - -### Default Install Directories - -When using the Python library (`camoufox fetch`), the default install directory is: - -| OS | Install Directory | -|------|-------------------| -| **Linux** | `~/.cache/camoufox/` | -| **macOS** | `~/Library/Caches/camoufox/` | -| **Windows** | `C:\Users\\AppData\Local\camoufox\camoufox\Cache\` | - -## Replacing the Binary - -To test FF146 with an existing Camoufox installation: - -1. Build from source using the instructions above -2. Extract the built zip from `dist/` -3. Replace the binary at the corresponding path for your OS: - -**Linux:** -```bash -cp /path/to/built/camoufox-bin ~/.cache/camoufox/camoufox-bin -``` - -**macOS:** -```bash -cp /path/to/built/Camoufox.app ~/Library/Caches/camoufox/Camoufox.app -``` - -**Windows:** -```powershell -copy C:\path\to\built\camoufox.exe C:\Users\\AppData\Local\camoufox\camoufox\Cache\camoufox.exe -``` diff --git a/docs/input-dispatch.md b/docs/input-dispatch.md index e31ac007d..cabaac84a 100644 --- a/docs/input-dispatch.md +++ b/docs/input-dispatch.md @@ -75,7 +75,7 @@ reachable from the same slot (`apz-repaints-flushed`, `TabSwitchDone`, the drag path's waits), none of which has failed yet. **The static check.** `scripts/check-input-dispatch.py`, wired into -`.github/workflows/lint.yml`. Two exemptions, both content-process: +the `static` job of `.github/workflows/tests.yml`. Two exemptions, both content-process: `PageAgent.js` (drag events, already content-relative, no ack) and `FrameTree.js` (the ack *producer*). diff --git a/docs/patch-upgrading-guide.md b/docs/patch-upgrading-guide.md index 457b14580..c8e86d9cc 100644 --- a/docs/patch-upgrading-guide.md +++ b/docs/patch-upgrading-guide.md @@ -1,16 +1,16 @@ -# Firefox Patch Upgrading Guide for LLMs +# Firefox Patch Upgrading Guide -This guide provides step-by-step instructions for updating Camoufox patches when upgrading Firefox versions. Patches frequently break due to Firefox API changes, file reorganizations, and line number shifts. - -**All patches are located in the `patches/` directory.** There are no separate context patches to merge—per-context functionality is already built into the patches. +How to update Camoufox's patches when `upstream.sh` moves to a new Firefox +version. Patches break because Firefox renames APIs, moves code and shifts line +numbers; this guide covers finding and fixing those rejects. ## Table of Contents 1. [Understanding the Patch System](#understanding-the-patch-system) -2. [Preparation](#preparation) +2. [The Source Tree and Its Make Targets](#the-source-tree-and-its-make-targets) 3. [General Workflow](#general-workflow) 4. [Fixing Common Reject Types](#fixing-common-reject-types) -5. [Context Patch Merging](#context-patch-merging) *(Historical - skip for future updates)* +5. [Per-Context Machinery](#per-context-machinery) 6. [Testing and Validation](#testing-and-validation) 7. [Best Practices](#best-practices) @@ -20,66 +20,89 @@ This guide provides step-by-step instructions for updating Camoufox patches when ### Patch Categories -All Camoufox patches are in the `patches/` directory: +All patches live under `patches/`, and `scripts/patch.py` applies every +`*.patch` in it (subdirectories included), sorted by file name: -- **Core Patches**: `0-playwright.patch`, `1-leak-fixes.patch`, etc. -- **Feature Patches**: `webrtc-ip-spoofing.patch`, `anti-font-fingerprinting.patch`, etc. -- All patches now include per-user-context (per-Playwright-context) support built-in +- **Playwright**: `playwright/0-playwright.patch` (Juggler integration) and + `playwright/1-leak-fixes.patch`. Their names sort first, and every other + patch is written against a tree that already has them. +- **Feature patches**: `webrtc-ip-spoofing.patch`, + `anti-font-fingerprinting.patch`, etc. Per-user-context (per-Playwright-context) + support is built into each one. +- **`librewolf/`, `ghostery/`**: patches taken from those projects. -**Historical Note**: Context patches (e.g., `font-fingerprinting.context.patch`, `webrtc.context.patch`) were previously separate but have been merged into their base patches as of Firefox 146. You will not find `.context.patch` files in the repository. +Compile-time dependencies between patches (MaskConfig, RoverfoxStorageManager) +are listed in [`patches/patch-dependencies.md`](../patches/patch-dependencies.md). ### Key Infrastructure Files - **RoverfoxStorageManager.cpp/h**: Thread-safe key-value storage for per-context data -- **Manager Classes**: FontSpacingSeedManager, WebRTCIPManager, etc. -- **Window.webidl**: Exposes JavaScript APIs to Playwright +- **Manager Classes**: AudioFingerprintManager, WebRTCIPManager, etc. +- **Window.webidl**: Exposes the per-context setters to Playwright --- -## Preparation +## The Source Tree and Its Make Targets -### 1. Reset to Clean State +The Firefox tree is `camoufox--/` (from `upstream.sh`). It is +a git repository whose `unpatched` tag is plain Firefox plus `additions/` and +`settings/`. Run every target from the repository root: -**IMPORTANT**: Always use `make clean` to reset to fresh Firefox source: +| Target | What it does | +|---|---| +| `make dir` | Fetches and extracts Firefox if the tree is missing. Otherwise resets it to `unpatched`, runs `mach clobber` and `git clean -fdx` (the object directory goes too), re-copies additions, then applies every patch and lists the ones that left rejects. | +| `make revert` | `git reset --hard unpatched`. Untracked files stay, including new files that patches created. | +| `make clean` | `mach clobber`, `git clean -fdx`, then `make revert`: unpatched Firefox with nothing left over, without re-fetching. | +| `make patch ./patches/x.patch` | Applies one patch (`patch -p1`). | +| `make unpatch ./patches/x.patch` | Reverses one patch. | +| `make first-checkpoint` | Commits the current tree and tags it `first-checkpoint`. | +| `make workspace ./patches/x.patch` | Unapplies `x` if it is applied, runs `first-checkpoint`, then applies `x` again, so the working tree differs from the checkpoint by exactly that patch. | +| `make diff` | `git diff first-checkpoint`. Redirect it into the patch file. | -```bash -make clean -``` +`git diff` does not show untracked files. Before `make diff`, mark new files +with `git add -N ` inside the source tree, or they will be missing from +the patch. + +--- -**DO NOT** use `git reset` or `git clean` commands directly in the Firefox source directory - these can delete untracked files needed for the build. +## General Workflow -### 2. Identify Patches to Update +### Step 1: Bump the Version and Find the Broken Patches -Check which patches exist: +Update `version` and `release` in `upstream.sh`, then: ```bash -ls patches/*.patch +make dir ``` -### 3. Understand Patch Dependencies - -Some patches depend on others being applied first: -- `1-leak-fixes.patch` requires `0-playwright.patch` -- Check the Makefile or patch comments for dependency chains - ---- +`patch.py` applies every patch and ends with a list of the ones that failed and +their reject files. It deletes the `.rej` files after listing them, so reproduce +each failure one patch at a time (Step 2). -## General Workflow +### Step 2: Set Up One Patch -### Step 1: Apply Base Patch and Identify Rejects +Start from a clean unpatched tree, apply what the patch builds on (at least the +Playwright patches, plus anything from `patches/patch-dependencies.md`), +checkpoint, then apply the broken patch. Use `make clean` rather than +`make revert` here: files that other patches created survive a revert and make +`patch` stop on "previously applied" prompts. ```bash -cd camoufox- -patch -p1 < ../patches/patch-name.patch +make clean +make patch ./patches/playwright/0-playwright.patch +make patch ./patches/playwright/1-leak-fixes.patch +make first-checkpoint +make patch ./patches/patch-name.patch # fails, leaving .rej files ``` -Find reject files: +Find the reject files: ```bash +cd camoufox-- find . -name '*.rej' -type f ``` -### Step 2: Analyze Each Reject File +### Step 3: Analyze Each Reject File Read the reject file to understand what failed: @@ -92,7 +115,7 @@ Reject files show: - `-` lines: What the patch expected to find (old code) - `+` lines: What the patch wanted to add (new code) -### Step 3: Locate the Correct Position in Firefox Code +### Step 4: Locate the Correct Position in Firefox Code The line numbers in rejects are usually wrong for the new Firefox version. You need to: @@ -100,42 +123,31 @@ The line numbers in rejects are usually wrong for the new Firefox version. You n 2. **Understand what the patch is doing** 3. **Find equivalent location** in new Firefox code -### Step 4: Apply Changes Manually +### Step 5: Apply Changes Manually -Use the Edit tool to apply the rejected changes to the correct location. +Edit the file to make the rejected change at the correct location. -### Step 5: Remove Reject Files +### Step 6: Remove Reject Files -After fixing all rejects: +After fixing all rejects, delete the `.rej` files and any `.orig` backups +`patch` left, so they do not end up in the diff: ```bash -rm -f path/to/file.cpp.rej -find . -name '*.rej' -type f # Verify all removed +find . -name '*.rej' -o -name '*.orig' | xargs rm -f ``` -### Step 6: Generate Updated Patch +### Step 7: Write the Updated Patch -```bash -# Add any new files first -git add new/file.cpp new/file.h - -# Generate patch with both staged and unstaged changes -git diff --cached --binary > /tmp/patch-name.patch -git diff --binary >> /tmp/patch-name.patch +From the repository root: -# Copy to patches directory -cp /tmp/patch-name.patch ../patches/patch-name.patch +```bash +(cd camoufox-- && git add -N path/to/new/file.cpp) # new files only +make diff > patches/patch-name.patch ``` -### Step 7: Verify Patch Applies Cleanly +### Step 8: Verify -```bash -cd .. -make clean -cd camoufox- -patch -p1 < ../patches/patch-name.patch -find . -name '*.rej' -type f # Should return nothing -``` +Run `make dir` again. The patch should no longer be listed as failing. --- @@ -285,77 +297,19 @@ Simply apply the patch manually at the correct line number. The code hasn't chan --- -## Context Patch Merging (Historical - Not Applicable for Future Updates) - -**NOTE**: As of Firefox 146, all context patches have been merged into their base patches. This section is kept for historical reference and understanding how the patches evolved. Future Firefox updates will only need to update patches in the `patches/` directory. - ---- - -**Historical Context**: Context patches previously added per-user-context functionality to base patches. The workflow was different from simple patch updates. - -### Historical Goal +## Per-Context Machinery -Merge all changes from `*.context.patch` into the corresponding base patch so there's only one comprehensive patch file. +Most spoofing patches carry per-context support. When porting one, expect these +pieces: -### Historical Example: font-fingerprinting.context.patch → anti-font-fingerprinting.patch - -### Workflow - -1. **Reset to clean Firefox**: - ```bash - make clean - ``` - -2. **Apply base patch first**: - ```bash - cd camoufox- - patch -p1 < ../patches/anti-font-fingerprinting.patch - ``` - -3. **Apply context patch on top**: - ```bash - patch -p1 < ../font-fingerprinting.context.patch - ``` - -4. **Fix any rejects** (usually include conflicts since base patch may have some overlapping changes) - -5. **Generate combined patch**: - ```bash - # Add new files (e.g., FontSpacingSeedManager.cpp/h) - git add dom/base/FontSpacingSeedManager.cpp - git add dom/base/FontSpacingSeedManager.h - git add dom/base/RoverfoxStorageManager.cpp - git add dom/base/RoverfoxStorageManager.h - - # Generate combined patch - git diff --cached --binary > /tmp/anti-font-fingerprinting.patch - git diff --binary >> /tmp/anti-font-fingerprinting.patch - - # Replace base patch - cp /tmp/anti-font-fingerprinting.patch ../patches/anti-font-fingerprinting.patch - ``` - -6. **Verify combined patch**: - ```bash - cd .. - make clean - cd camoufox- - patch -p1 < ../patches/anti-font-fingerprinting.patch - find . -name '*.rej' -type f # Should be empty - ``` - -### What Context Patches Add - -Context patches typically add: - -1. **Manager classes** (e.g., FontSpacingSeedManager, WebRTCIPManager): +1. **Manager classes** (e.g., AudioFingerprintManager, WebRTCIPManager): - Store per-context settings using RoverfoxStorageManager - Provide WebIDL-compatible enable/disable checks - Handle self-destructing functions 2. **Window.webidl functions**: - JavaScript APIs exposed to Playwright - - Examples: `setFontSpacingSeed()`, `setWebRTCIPv4()` + - Examples: `setAudioFingerprintSeed()`, `setWebRTCIPv4()` 3. **nsGlobalWindowInner.cpp implementations**: - Extract userContextId from window/document/docshell @@ -363,9 +317,10 @@ Context patches typically add: - Self-destruct logic (remove function after first use) 4. **Core logic changes**: - - Replace global config (MaskConfig) with per-context manager + - Consult the per-context manager before the global config (MaskConfig) - Pass userContextId through call chains - - Query manager for per-context values + +See [`per-context-patches.md`](per-context-patches.md) for the full list. --- @@ -375,25 +330,18 @@ Context patches typically add: After updating a patch, always verify: -1. **Patch applies cleanly**: - ```bash - make clean - cd camoufox- - patch -p1 < ../patches/patch-name.patch - find . -name '*.rej' -type f - ``` - -2. **No reject files remain** +1. **Every patch applies cleanly**: `make dir` lists no failures. -3. **Build compiles** (if feasible): +2. **Build compiles** (if feasible): ```bash - cd .. make build ``` ### Full Testing -For critical patches, test with actual Playwright scenarios after building. +Run the suites that cover the patch (see [`ci/README.md`](../ci/README.md)): +`python3 -m ci.run_patch_guards --binary ` is the most direct +evidence that a patch which still applies was not neutered by the upgrade. --- @@ -401,18 +349,18 @@ For critical patches, test with actual Playwright scenarios after building. ### DO: -1. ✅ **Always use `make clean`** to reset Firefox source +1. ✅ **Start each patch from `make clean`** plus the patches it builds on 2. ✅ **Read and understand** what the patch is trying to do before fixing rejects 3. ✅ **Search for API changes** in Firefox release notes when functions have changed 4. ✅ **Use grep/search** extensively to find where code moved 5. ✅ **Extract userContextId properly** using the standard pattern -6. ✅ **Test patches apply cleanly** before considering them done +6. ✅ **Check with `make dir`** that the whole stack applies before considering a patch done 7. ✅ **Keep commits atomic** - one patch fix per session 8. ✅ **Document major API changes** you discover ### DON'T: -1. ❌ **Don't use `git reset` or `git clean`** on Firefox source directory +1. ❌ **Don't hand-edit `.patch` files** - edit the tree and regenerate with `make diff` 2. ❌ **Don't leave TODO comments** - fix things properly as you go 3. ❌ **Don't guess parameter values** - extract them properly or investigate 4. ❌ **Don't skip verification** - always test the patch applies cleanly @@ -424,75 +372,9 @@ For critical patches, test with actual Playwright scenarios after building. 1. **Assuming reject line numbers are accurate**: They're usually wrong in new Firefox versions 2. **Not understanding API changes**: Firefox refactors often - read the new code -3. **Forgetting to add new files**: Use `git add` before generating patch -4. **Not testing on clean source**: Always verify with `make clean` -5. **Leaving reject files**: Remove all `.rej` files after fixing - ---- - -## Example: Complete Patch Update Session - -Here's a complete example of updating `0-playwright.patch` from Firefox 144 to Firefox 146: - -### 1. Reset and Apply - -```bash -make clean -cd camoufox-146.0.1-beta.25 -patch -p1 < ../patches/0-playwright.patch -``` - -### 2. Find Rejects - -```bash -find . -name '*.rej' -type f -``` - -Output shows 20 reject files. - -### 3. Analyze First Reject - -```bash -cat dom/base/Navigator.cpp.rej -``` - -Shows parameter order changed in `GetAcceptLanguages`. - -### 4. Fix the Reject - -Search for the function in the actual file, understand the new signature, apply changes manually. - -### 5. Repeat for All Rejects - -Work through each reject systematically. - -### 6. Discover API Change - -Firefox 146 refactored mouse events from individual parameters to `SynthesizeMouseEventData` and `SynthesizeMouseEventOptions`. Port all mouse event logic to new API. - -### 7. Remove Rejects - -```bash -rm -f dom/base/Navigator.cpp.rej dom/base/Element.cpp.rej ... -find . -name '*.rej' -type f # Verify empty -``` - -### 8. Generate New Patch - -```bash -git diff --binary > /tmp/0-playwright.patch -cp /tmp/0-playwright.patch ../patches/0-playwright.patch -``` - -### 9. Verify - -```bash -cd .. -make clean -cd camoufox-146.0.1-beta.25 -patch -p1 < ../patches/0-playwright.patch -find . -name '*.rej' -type f # Should be empty -``` +3. **Forgetting new files**: `git diff` skips untracked files; `git add -N` them before `make diff` +4. **Diffing against the wrong base**: `make first-checkpoint` before applying the patch you are fixing, or `make diff` will include its dependencies +5. **Leaving reject files**: Remove all `.rej` and `.orig` files after fixing --- @@ -570,17 +452,17 @@ if (userContextId == 0) { When updating patches for a new Firefox version: -- [ ] Use `make clean` to reset to fresh Firefox source -- [ ] Apply patch and identify all reject files +- [ ] Bump `upstream.sh` and run `make dir` to list the failing patches +- [ ] For each: `make clean`, apply its dependencies, `make first-checkpoint`, `make patch` it - [ ] Analyze each reject to understand what changed - [ ] Search Firefox source for moved/refactored code - [ ] Fix rejects by porting logic to new Firefox APIs - [ ] Extract userContextId properly using standard patterns - [ ] Don't leave TODO comments - fix everything immediately -- [ ] Remove all `.rej` files after fixing -- [ ] Add any new files with `git add` -- [ ] Generate new patch with `git diff --cached --binary` + `git diff --binary` -- [ ] Verify patch applies cleanly to fresh source +- [ ] Remove all `.rej` and `.orig` files after fixing +- [ ] `git add -N` any new files +- [ ] `make diff > patches/.patch` +- [ ] `make dir` applies the whole stack cleanly - [ ] Document any major API changes discovered --- @@ -590,7 +472,3 @@ When updating patches for a new Firefox version: - Firefox source: https://searchfox.org/ - Firefox API documentation: https://firefox-source-docs.mozilla.org/ - Mercurial repository: https://hg.mozilla.org/mozilla-central/ - ---- - -**Last Updated**: December 2025 (Firefox 146 upgrade) diff --git a/docs/per-context-patches.md b/docs/per-context-patches.md index 27bef41fc..21b8f5ef8 100644 --- a/docs/per-context-patches.md +++ b/docs/per-context-patches.md @@ -2,30 +2,29 @@ Camoufox spoofs fingerprints globally via `CAMOU_CONFIG` — every browser context shares the same identity. These patches add **per-context isolation**, so each Playwright context can have a unique, deterministic fingerprint. This lets you run multiple concurrent sessions from a single Camoufox process without cross-context correlation. -### What's New +### The Patches -**New patches (8):** +**Per-context patches (with a `window.setXxx()` API):** +- `anti-font-fingerprinting.patch` — adds `RoverfoxStorageManager` (the shared per-context store) and gives each font group its context's userContextId, which `font-list-spoofing.patch` uses to pick that context's font list - `audio-fingerprint-manager.patch` — per-context audio fingerprint seeding (all 6 AudioBuffer + AnalyserNode methods) - `timezone-spoofing.patch` — true per-realm timezone isolation via SpiderMonkey DateTimeInfo +- `screen-spoofing.patch` — per-context screen dimensions and color depth via `ScreenDimensionManager` - `navigator-spoofing.patch` — per-context platform, oscpu, hardwareConcurrency, userAgent +- `webrtc-ip-spoofing.patch` — per-context WebRTC IP, including `getStats()` sanitization and IPv6 - `webgl-spoofing.patch` — per-context UNMASKED_VENDOR/RENDERER_WEBGL -- `canvas-spoofing.patch` — per-context canvas 2D fingerprint noise - `font-list-spoofing.patch` — per-context installed font list filtering via thread-local propagation - `speech-voices-spoofing.patch` — per-context `speechSynthesis.getVoices()` filtering -- `cross-process-storage.patch` — IPDL message for content-to-parent pref writes, enabling cross-process fingerprint storage -**Enhanced existing patches (5):** -- `anti-font-fingerprinting.patch` — added `RoverfoxStorageManager` (cross-process Preferences-based storage), `WordCacheKey` fix (userContextId in glyph cache to prevent cross-context cache hits), random font subset generation -- `screen-spoofing.patch` — replaces old `screen-hijacker.patch` with full per-context support via `ScreenDimensionManager` -- `webrtc-ip-spoofing.patch` — added `getStats()` API sanitization, per-context IP storage, comprehensive IPv6 regex -- `geolocation-spoofing.patch` — updated for Firefox 146, fixed malformed hunks and moz.build line offsets -- `locale-spoofing.patch` — updated for Firefox 146 compatibility +**Infrastructure:** +- `cross-process-storage.patch` — IPDL messages for content-to-parent storage writes, so per-context values reach every process + +There is no canvas pixel noise: Camoufox leaves `toDataURL()`/`getImageData()` +output as the GPU and fonts produce it. ## Quick Reference | Function | Patch | What it controls | |----------|-------|-----------------| -| `window.setFontSpacingSeed(seed)` | `anti-font-fingerprinting.patch` | Canvas `measureText()` letter spacing | | `window.setAudioFingerprintSeed(seed)` | `audio-fingerprint-manager.patch` | Audio buffer/analyser fingerprint hash | | `window.setTimezone(tz)` | `timezone-spoofing.patch` | `Date`, `Intl.DateTimeFormat`, all time APIs | | `window.setScreenDimensions(w, h)` | `screen-spoofing.patch` | `screen.width`, `screen.height` | @@ -38,11 +37,10 @@ Camoufox spoofs fingerprints globally via `CAMOU_CONFIG` — every browser conte | `window.setWebRTCIPv6(ip)` | `webrtc-ip-spoofing.patch` | WebRTC IPv6 addresses | | `window.setWebGLVendor(vendor)` | `webgl-spoofing.patch` | `UNMASKED_VENDOR_WEBGL` parameter | | `window.setWebGLRenderer(renderer)` | `webgl-spoofing.patch` | `UNMASKED_RENDERER_WEBGL` parameter | -| `window.setCanvasSeed(seed)` | `canvas-spoofing.patch` | Canvas 2D `toDataURL()`/`getImageData()` hash | | `window.setFontList(fonts)` | `font-list-spoofing.patch` | Which fonts appear "installed" to fingerprinters | | `window.setSpeechVoices(voices)` | `speech-voices-spoofing.patch` | `speechSynthesis.getVoices()` filtering | -All 16 functions **self-destruct after the first call** — page JavaScript cannot detect them via `typeof window.setTimezone`. +All 15 functions **self-destruct after the first call** — page JavaScript cannot detect them via `typeof window.setTimezone`. --- @@ -65,9 +63,6 @@ const context = await browser.newContext({ await context.addInitScript((values) => { const w = window; - if (typeof w.setFontSpacingSeed === 'function') { - w.setFontSpacingSeed(values.fontSpacingSeed); - } if (typeof w.setAudioFingerprintSeed === 'function') { w.setAudioFingerprintSeed(values.audioFingerprintSeed); } @@ -101,9 +96,6 @@ await context.addInitScript((values) => { if (typeof w.setWebGLRenderer === 'function') { w.setWebGLRenderer(values.webglRenderer); } - if (typeof w.setCanvasSeed === 'function') { - w.setCanvasSeed(values.canvasSeed); - } if (values.fontList && values.fontList.length > 0 && typeof w.setFontList === 'function') { w.setFontList(values.fontList.join(',')); } @@ -111,7 +103,6 @@ await context.addInitScript((values) => { w.setSpeechVoices(values.speechVoices); } }, { - fontSpacingSeed: 12345678, audioFingerprintSeed: 87654321, timezone: 'America/New_York', screenWidth: 1920, @@ -121,10 +112,9 @@ await context.addInitScript((values) => { navigatorPlatform: 'MacIntel', navigatorOscpu: 'Intel Mac OS X 10.15', hardwareConcurrency: 8, - userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:146.0) Gecko/20100101 Firefox/146.0', + userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:152.0) Gecko/20100101 Firefox/152.0', webglVendor: 'Intel Inc.', webglRenderer: 'Intel Iris OpenGL Engine', - canvasSeed: 55555555, fontList: ['Arial', 'Helvetica', 'Georgia', 'Courier New', 'Verdana', 'Times New Roman'], speechVoices: 'Microsoft David,Microsoft Zira,Google US English', }); @@ -147,9 +137,8 @@ await ctxA.addInitScript((v) => { if (typeof window.setTimezone === 'function') window.setTimezone(v.tz); if (typeof window.setAudioFingerprintSeed === 'function') window.setAudioFingerprintSeed(v.audio); if (typeof window.setScreenDimensions === 'function') window.setScreenDimensions(v.sw, v.sh); - if (typeof window.setCanvasSeed === 'function') window.setCanvasSeed(v.canvas); if (typeof window.setWebGLRenderer === 'function') window.setWebGLRenderer(v.gpu); -}, { tz: 'America/New_York', audio: 11111, sw: 1920, sh: 1080, canvas: 44444, gpu: 'Intel Iris OpenGL Engine' }); +}, { tz: 'America/New_York', audio: 11111, sw: 1920, sh: 1080, gpu: 'Intel Iris OpenGL Engine' }); // Context B — appears as a Tokyo user with Apple GPU (fully isolated from A) const ctxB = await browser.newContext(); @@ -157,9 +146,8 @@ await ctxB.addInitScript((v) => { if (typeof window.setTimezone === 'function') window.setTimezone(v.tz); if (typeof window.setAudioFingerprintSeed === 'function') window.setAudioFingerprintSeed(v.audio); if (typeof window.setScreenDimensions === 'function') window.setScreenDimensions(v.sw, v.sh); - if (typeof window.setCanvasSeed === 'function') window.setCanvasSeed(v.canvas); if (typeof window.setWebGLRenderer === 'function') window.setWebGLRenderer(v.gpu); -}, { tz: 'Asia/Tokyo', audio: 99999, sw: 2560, sh: 1440, canvas: 88888, gpu: 'Apple M1' }); +}, { tz: 'Asia/Tokyo', audio: 99999, sw: 2560, sh: 1440, gpu: 'Apple M1' }); ``` --- @@ -176,7 +164,7 @@ All patches share `RoverfoxStorageManager`, a thread-safe C++ key-value store ke 3. The value is stored in RoverfoxStorageManager's local HashMap cache (thread-safe `nsTHashMap` protected by `Mutex`) 4. The value is also written to Firefox Preferences (`Preferences::SetCString`) with a `roverfox.s.` prefix — all value types (uint32, bool, string) are serialized as CString internally 5. In content processes, values are sent to the parent (browser) process via sync IPC (`SendRoverfoxStoragePut`) -6. Some patches also store the value under ucid=0 as a global fallback — this ensures workers that cannot resolve a specific `userContextId` can still read the value. Patches with ucid=0 fallback: **audio**, **canvas**, **navigator** (all 4 functions), **timezone**, **webgl**. Patches without: font-spacing, screen, font-list, speech-voices, webrtc-ip +6. Some patches also store the value under ucid=0 as a global fallback — this ensures workers that cannot resolve a specific `userContextId` can still read the value. Patches with ucid=0 fallback: **audio**, **navigator** (all 4 functions), **screen**, **timezone**, **webgl**. Patches without: font-spacing, font-list, speech-voices, webrtc-ip **Read path (3-tier fallback):** 1. **Local cache** — in-process `nsTHashMap` protected by `Mutex` (fastest, same-process reads) @@ -229,7 +217,7 @@ Workers resolve `userContextId` via `WorkerPrivate::GetOriginAttributes()`, whic The `camoufox.cfg` file sets Firefox preferences at startup (before `prefs.js` is loaded). Key settings: - `fission.autostart = true` — keeps Fission (site isolation) enabled. Some WAFs can detect disabled Fission. With the cross-process storage patch, Fission works correctly because values are synced across all content processes. -- `fission.webContentIsolationStrategy = 1` — standard isolation strategy. +- `fission.webContentIsolationStrategy = 0` — no site isolation: cross-site iframes stay in their parent's process (no out-of-process iframes); COOP handling and BFCache-in-parent stay active. - `dom.ipc.processPrelaunch.enabled = false` — prevents Firefox from reusing pre-launched content processes that may have stale overridden values (locale, timezone). Ensures each new content process starts clean. - No `dom.ipc.processCount` override — Firefox uses its default multi-process behavior. The cross-process storage patch eliminates the need for `processCount=1`. @@ -239,23 +227,16 @@ The `camoufox.cfg` file sets Firefox preferences at startup (before `prefs.js` i ### 1. anti-font-fingerprinting.patch -**Controls:** Canvas `measureText()` letter spacing — makes text width measurements unique per context. - -**How it works:** Stores a seed per context, then applies a deterministic spacing transformation in HarfBuzz (the text shaping engine). The seed is propagated through the entire text rendering pipeline: `nsTextFrame` → `gfxFont` → `gfxTextRun` → `gfxHarfBuzzShaper`. - -The transformation adds ~0.0-0.1 em of extra spacing using a Linear Congruential Generator seeded with the profile's value. Same seed always produces the same spacing. - -**Also provides:** `RoverfoxStorageManager` — the shared storage layer used by all other per-context patches. See the [Cross-Process Storage](#cross-process-storage-cross-process-storagepatch) section for how it works across processes. +**Controls:** nothing a page can see by itself. It is the groundwork the other per-context patches build on. -**WordCacheKey fix:** Added `mUserContextId` to the `WordCacheKey` struct in `gfxFont.h`. Without this, Firefox's shaped word cache shared results across contexts — context 1's font spacing result would be returned for context 2 (a cache hit based on text content alone). The fix adds `mUserContextId` to both constructors, the hash computation (via `* 0x1000000`), and the `match()` comparison, ensuring each context has its own cache entries. Also adds `GetUserContextId()` virtual method to `gfxShapedText` and `gfxShapedWord` so the context ID propagates through the text run pipeline. +**Provides:** +- `RoverfoxStorageManager`, the shared storage layer used by all other per-context patches. See the [Cross-Process Storage](#cross-process-storage-cross-process-storagepatch) section for how it works across processes. +- The userContextId on each `gfxFontGroup`, read from the document's `BrowsingContext` through a `GetDocument()` hook on `FontVisibilityProvider`. `font-list-spoofing.patch` uses it to apply that context's font list. -**API:** -```javascript -window.setFontSpacingSeed(12345678); // uint32 seed -``` +Text is shaped exactly as stock Firefox shapes it. An earlier glyph-spacing seed was removed because the widths it produced match no real installation (`ci/tribal-rules.yml`: `no-glyph-spacing-noise`). -**New C++ files:** `FontSpacingSeedManager.h/cpp`, `RoverfoxStorageManager.h/cpp` -**Modified Firefox files (22):** `nsGlobalWindowInner.cpp/h`, `CanvasRenderingContext2D.cpp`, `OffscreenCanvas.cpp`, `WorkerPrivate.h`, `Window.webidl`, `moz.build` (dom/base), `gfxHarfBuzzShaper.cpp`, `gfxTextRun.cpp/h`, `gfxFont.cpp/h`, `nsFontMetrics.cpp/h`, `nsLayoutUtils.cpp/h`, `nsPresContext.cpp`, `nsTextFrame.cpp`, `MathMLTextRunFactory.cpp`, `nsTextRunTransformations.cpp`, `nsMathMLChar.cpp`, `FontVisibilityProvider.h` +**New C++ files:** `RoverfoxStorageManager.h/cpp` +**Modified Firefox files:** `moz.build` (dom/base), `nsGlobalWindowInner.cpp`, `OffscreenCanvas.cpp`, `WorkerPrivate.h`, `gfxPlatformFontList.cpp`, `gfxTextRun.cpp/h`, `nsPresContext.cpp`, `FontVisibilityProvider.h` --- @@ -322,7 +303,7 @@ window.setTimezone('America/New_York'); // IANA timezone ID Also hooks `nsMediaFeatures.cpp` so CSS media queries like `matchMedia('(device-width: 1920px)')` return results consistent with `screen.width`. Without this, fingerprinters can detect a mismatch between the JavaScript API and CSS media queries. -This replaces the old `screen-hijacker.patch` (which only supported global config). It includes the same global `CAMOU_CONFIG` fallback, so it works for both single-context and multi-context use cases. +The global `CAMOU_CONFIG` fallback means it works for both single-context and multi-context use cases. **API:** ```javascript @@ -389,7 +370,7 @@ window.setWebRTCIPv6('2001:db8::1'); // proxy exit IPv6 (optional) window.setNavigatorPlatform('Win32'); // navigator.platform window.setNavigatorOscpu('Windows NT 10.0; Win64; x64'); // navigator.oscpu window.setNavigatorHardwareConcurrency(8); // navigator.hardwareConcurrency -window.setNavigatorUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:146.0) Gecko/20100101 Firefox/146.0'); +window.setNavigatorUserAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:152.0) Gecko/20100101 Firefox/152.0'); ``` **Global config fallback (no JavaScript needed):** @@ -427,35 +408,7 @@ window.setWebGLRenderer('Intel Iris OpenGL Engine'); // UNMASKED_RENDERER_WEBGL --- -### 8. canvas-spoofing.patch - -**Controls:** Canvas 2D fingerprint hash — websites draw text, shapes, and gradients on a canvas, then call `toDataURL()` or `getImageData()` to hash the pixel output. GPU, driver, and font rendering differences make this hash highly unique. - -**How it works:** Stores a seed per context via `CanvasFingerprintManager`, then hooks both canvas data extraction paths in `CanvasRenderingContext2D.cpp`: -- `GetImageBuffer()` — used by `toDataURL()` and `toBlob()`, returns pixels in **BGRA** format -- `GetImageData()` — used by `ctx.getImageData()`, returns pixels in **RGBA** format - -The noise algorithm is **format-agnostic**: for each selected pixel, it iterates RGB channels (skipping alpha) and modifies the first non-zero channel by +/-1. This works correctly regardless of whether byte 0 is Red (RGBA) or Blue (BGRA). - -**Zero-pixel preservation:** Channels with value 0 are skipped. This means `clearRect()` followed by `getImageData()` returns all zeros — no false noise on transparent pixels. This is important because CreepJS specifically tests for noise in cleared canvas regions as a detection vector. - -**Noise is deterministic, not random:** Same seed always produces the same pixel modifications, so fingerprinters calling `toDataURL()` multiple times get identical results. This is critical — random noise is trivially detected by calling the API twice and comparing outputs. - -**Worker support:** Includes a `WorkerPrivate` fallback for resolving `userContextId` when canvas operations happen in a Web Worker via OffscreenCanvas. - -**MaskConfig fallback:** If no per-context seed is set, checks `MaskConfig::GetUint32("canvas:seed")` from `CAMOU_CONFIG`. - -**API:** -```javascript -window.setCanvasSeed(55555555); // uint32 seed -``` - -**New C++ files:** `CanvasFingerprintManager.h/cpp` -**Modified Firefox files:** `nsGlobalWindowInner.cpp/h`, `CanvasRenderingContext2D.cpp`, `Window.webidl`, `moz.build` - ---- - -### 9. font-list-spoofing.patch +### 8. font-list-spoofing.patch **Controls:** Which fonts appear "installed" to fingerprinting scripts. Websites detect fonts by measuring text widths (canvas `measureText()`) — if the width changes compared to a fallback font, the font is present. Each context can have a different subset of fonts. @@ -493,7 +446,7 @@ window.setFontList('Arial,Helvetica,Georgia,Courier New,Verdana'); --- -### 10. speech-voices-spoofing.patch +### 9. speech-voices-spoofing.patch **Controls:** `speechSynthesis.getVoices()` — the list of installed text-to-speech voices. This varies by OS and installed language packs, making it a fingerprinting vector. Each context can expose a different subset of voices. @@ -512,7 +465,7 @@ window.setSpeechVoices('Microsoft David,Samantha,Alex'); --- -### 11. cross-process-storage.patch +### 10. cross-process-storage.patch **Controls:** Cross-process synchronization of all per-context fingerprint values. This is an infrastructure patch — it has no JavaScript API of its own. It enables all other per-context patches to work correctly when Firefox runs content in multiple processes (Fission). @@ -570,7 +523,7 @@ For per-context geolocation, use Playwright's built-in `context.setGeolocation() ## Build Notes -**SOURCES vs UNIFIED_SOURCES:** Most new `.cpp` manager files use `SOURCES` (separate compilation) in `moz.build` to avoid namespace pollution (`mozilla::dom::mozilla::dom::`) that occurs when files including `RoverfoxStorageManager.h` are concatenated in unified builds. Currently in `SOURCES`: `AudioFingerprintManager.cpp`, `WebRTCIPManager.cpp`, `NavigatorManager.cpp`, `WebGLParamsManager.cpp`, `CanvasFingerprintManager.cpp`, `FontListManager.cpp`, `SpeechVoicesManager.cpp`. Four files use `UNIFIED_SOURCES` instead: `FontSpacingSeedManager.cpp`, `RoverfoxStorageManager.cpp` (both from `anti-font-fingerprinting.patch`), `TimezoneManager.cpp` (from `timezone-spoofing.patch`), and `ScreenDimensionManager.cpp` (from `screen-spoofing.patch`) — these were written before the SOURCES pattern was established and happen to compile without namespace issues in their alphabetical position. +**SOURCES vs UNIFIED_SOURCES:** Most new `.cpp` manager files use `SOURCES` (separate compilation) in `moz.build` to avoid namespace pollution (`mozilla::dom::mozilla::dom::`) that occurs when files including `RoverfoxStorageManager.h` are concatenated in unified builds. Currently in `SOURCES`: `AudioFingerprintManager.cpp`, `WebRTCIPManager.cpp`, `NavigatorManager.cpp`, `WebGLParamsManager.cpp`, `FontListManager.cpp`, `SpeechVoicesManager.cpp`, `ScreenDimensionManager.cpp`. Two files use `UNIFIED_SOURCES` and compile without namespace issues in their alphabetical position: `RoverfoxStorageManager.cpp` (from `anti-font-fingerprinting.patch`) and `TimezoneManager.cpp` (from `timezone-spoofing.patch`). **EXPORTS sort conflicts:** Each patch uses a separate `EXPORTS.mozilla.dom += ["Header.h"]` statement near its `SOURCES` block, rather than inserting into the main sorted EXPORTS list. This avoids sort conflicts when multiple patches add headers at similar alphabetical positions. @@ -578,7 +531,7 @@ For per-context geolocation, use Playwright's built-in `context.setGeolocation() **Patch independence:** All patches apply independently to vanilla Firefox. Context lines in hunks reference unpatched source files. Patches apply alphabetically and use fuzzy matching for line shifts caused by other patches. -**camoufox.cfg:** The `settings/camoufox.cfg` file sets `fission.autostart=true`, `fission.webContentIsolationStrategy=1`, and `dom.ipc.processPrelaunch.enabled=false`. No `dom.ipc.processCount` override is needed — the cross-process storage patch enables all per-context values to sync across Firefox's default multi-process architecture. +**camoufox.cfg:** The `settings/camoufox.cfg` file sets `fission.autostart=true`, `fission.webContentIsolationStrategy=0`, and `dom.ipc.processPrelaunch.enabled=false`. No `dom.ipc.processCount` override is needed — the cross-process storage patch enables all per-context values to sync across Firefox's default multi-process architecture. --- @@ -608,16 +561,16 @@ what differs per identity is which of them are on the search path. **What each `fonts.conf` defines:** - **Generic family defaults** — `sans-serif`, `serif`, `monospace`, `cursive`, `fantasy`, `system-ui` mapped to OS-appropriate fonts - **TTC weight-variant aliases** — macOS TrueType Collections register with weight suffixes ("PingFang HK Light") but Linux fontconfig only sees the base name. Aliases rewrite queries so CreepJS marker font detection works cross-platform. -- **MONO redirect** — "MONO" is a Linux-only font. Redirected to the OS-appropriate monospace (Menlo on macOS, Cousine on Linux) to prevent host OS leakage. +- **MONO redirect** — "MONO" is a Linux marker font. The Linux and Windows configs redirect it to `monospace` so it measures like the monospace baseline; the macOS config leaves it unmatched, so it falls through to Menlo as on a real Mac. - **Rendering settings** — Standardized antialias, hinting, and lcdfilter across all configs. **Runtime path rewriting:** At launch time, `utils._generate_fontconfig()` reads the bundled `fonts.conf` and replaces its single `fonts` with one absolute `` per group the claimed OS reads (from `fonts/groups.json`). This is what prevents cross-OS font leakage — a face the claimed OS must not see is simply not on the search path — and it also avoids CWD-dependent path issues. The parent `fonts/` directory is never named: fontconfig scans `` **recursively**, so naming it would make every other OS's faces reachable for glyph fallback even though the allowlist hides them from direct lookup. `scripts/verify-fonts.py` asserts that no file outside an OS's own groups is reachable under its conf. -**`FONTCONFIG_PATH` environment variable:** Must be set when launching Camoufox on Linux. Points to the correct OS-specific fontconfig directory (e.g. `camoufox/fontconfig/macos/`). The Go launcher sets this dynamically based on the target OS. +**`FONTCONFIG_FILE` environment variable:** On a Linux host, `utils.get_env_vars()` points `FONTCONFIG_FILE` at the generated `fonts.conf` for the claimed OS. A browser launched without the Python (or TypeScript) wrapper does not get it and uses the system fontconfig. --- -## Python Library Changes +## Python Library The Camoufox Python package (`pythonlib/`) generates fingerprints for both `NewBrowser` (global CAMOU_CONFIG) and `NewContext` (per-context init script). **fpgen is the default for both paths.** Real fingerprint presets are available as an opt-in alternative. @@ -630,9 +583,9 @@ The Camoufox Python package (`pythonlib/`) generates fingerprints for both `NewB **Recommended for v149+ binaries:** opt into bundled real fingerprints via `fingerprint_preset=True`. The library auto-selects the v150 preset bundle -(`fingerprint-presets-v150.json`, 312 real fingerprints scraped from v149–v152 +(`fingerprint-presets-v150.json`, 288 real fingerprints scraped from v149–v152 browsers) for any binary at Firefox ≥ 149, and falls back to the original -bundle (`fingerprint-presets.json`, 123 presets) for older binaries. UA strings +bundle (`fingerprint-presets.json`, 109 presets) for older binaries. UA strings are rewritten to match the active binary's Firefox version, so opting in costs nothing for compatibility. @@ -657,16 +610,14 @@ bundles are shipped in the wheel. | Property | Source | Notes | |----------|--------|-------| -| UA, platform, HWC, oscpu | fpgen or preset | UA version patched to match Camoufox Firefox version | +| UA, platform, HWC, oscpu | fpgen or preset | UA version patched to the browser's Firefox version (NewContext reads it from Playwright's `browser.version` unless `ff_version` is given) | | Screen dims, colorDepth | fpgen or preset | Viewport adjusted by -28px for browser chrome | -| WebGL vendor/renderer | `sample_webgl()` from `webgl_data.db` | OS-weighted probability sampling. The generator's own GPU fields are not mapped in `fpgen.yml` yet, so both paths call `sample_webgl()` for WebGL. fpgen does carry a full WebGL set (999 renderers against webgl_data.db's 33) -- wiring it through is the follow-up. | -| Font list | `_generate_random_font_subset()` | Random 30-78% of OS fonts. Essential + marker fonts always included. NOT from presets — generated fresh per call. | -| Font spacing seed | `randint(1, 2^32-1)` | Excludes 0 (0 = no-op in C++) | -| Audio seed | `randint(1, 2^32-1)` | Excludes 0 | -| Canvas seed | `randint(1, 2^32-1)` | Excludes 0 | -| Timezone | From preset, or Intl.DateTimeFormat fallback in init script | NewBrowser: from preset or geolocation detection. NewContext: preset or browser default. | -| Speech voices | `_generate_random_voice_subset()` | Random 40-80% of OS voices. Essential voices always included. macOS: 6 essentials + random subset of ~184. Windows: all voices (too few to subset). Linux: empty (no native voices). NOT from presets — generated fresh per call. | -| WebRTC IP | Not set by default | User sets via `window.setWebRTCIPv4()`. NewContext init script defaults to empty string `""` | +| WebGL vendor/renderer | Preset, or `sample_webgl_for_screen()` in `webgl.py` | A generated identity draws a GPU weighted by fpgen's share of Firefox on the OS, never a software rasteriser or a discrete GPU behind a netbook screen. `launch_options()` adds that GPU's recorded parameters, extensions and shader precisions from fpgen (`webgl_for_gpu()`), WebGL2 from the same device as WebGL1. | +| Font list | `_generate_random_font_subset()` | One weighted OS-version base in full, plus each addition unit at its measured probability; marker fonts always included. See [FONTS.md](FONTS.md). NOT from presets. | +| Audio seed | Derived from the identity (NewBrowser) or `randint(1, 2^32-1)` (NewContext) | Never 0 | +| Timezone | From preset, or `timezone` in `CAMOU_CONFIG` | The init script calls `setTimezone()` only for an explicit value; otherwise the C++ side falls back to `CAMOU_CONFIG` (set from geoip at launch) or the browser default. | +| Speech voices | `_generate_random_voice_subset()` | Follows the measured model in `voice-manifests.json`: Windows gets the display language's OneCore pack plus its legacy Desktop voices at their measured rate; macOS the compact + Eloquence base plus rare downloads; Linux speech-dispatcher's espeak-ng list. Seeded by the identity. NOT from presets. | +| WebRTC IP | Not set by default | NewContext's `webrtc_ip` (or the proxy's exit IP) goes to `window.setWebRTCIPv4()` or `window.setWebRTCIPv6()` by address family; an invalid address raises `InvalidIP`. Without one, the init script calls `setWebRTCIPv4("")` | | Geolocation | User parameter or geoip detection | Via Playwright `context.setGeolocation()` | ### Key Files @@ -675,26 +626,26 @@ bundles are shipped in the wheel. - `generate_context_fingerprint()` — main API. Returns `{init_script, context_options, config, preset}` - `from_preset()` — converts real preset to CAMOU_CONFIG format - `from_fpgen()` — converts an fpgen fingerprint dict to CAMOU_CONFIG using `fpgen.yml` mappings -- `_build_init_script()` — generates JavaScript IIFE calling 15 `window.setXxx()` functions with `typeof` guards (`setWebRTCIPv6` is not included — IPv6 is optional and rarely set) -- `_generate_random_font_subset()` — unique random font subset per call (Fisher-Yates, essential + marker fonts always included) -- `_generate_random_voice_subset()` — unique random voice subset per call (essential voices always included, OS-aware) +- `_build_init_script()` — generates a JavaScript IIFE calling the `window.setXxx()` functions with `typeof` guards (every setter except `setWebRTCIPv6` — IPv6 is optional and rarely set) +- `_generate_random_font_subset()` — the font list of one plausible machine of the OS (weighted base + per-unit draws, marker fonts always included) +- `_generate_random_voice_subset()` — the voice list of one plausible machine of the OS, as MaskConfig voice objects **`utils.py`** — Global browser launch configuration: - `launch_options()` — builds CAMOU_CONFIG env var, Playwright args, and Firefox prefs - Font subset generated via same `_generate_random_font_subset()` function - Voice subset generated via same `_generate_random_voice_subset()` function -- WebGL sampled via same `sample_webgl()` function +- WebGL drawn via the same `webgl.py` functions - Config validated against `properties.json` before serialization -**`fingerprint-presets.json`** — Original bundled real fingerprints organized by OS (macOS 30, Windows 75, Linux 18). Each preset includes navigator properties, screen dimensions, WebGL params, and speech voices. Used for Firefox < 149 binaries. Font and voice data not used from presets — generated fresh per launch. +**`fingerprint-presets.json`** — Original bundled real fingerprints organized by OS (macOS 18, Windows 73, Linux 18). Each preset includes navigator properties, screen dimensions, the WebGL vendor/renderer, and speech voices. Used for Firefox < 149 binaries. Font and voice data not used from presets — generated fresh per launch. -**`fingerprint-presets-v150.json`** — Newer bundle covering Firefox v149–v152 (macOS 67, Windows 180, Linux 65; 312 total). Same schema as the original. Auto-selected by `load_presets()` when the active binary reports Firefox ≥ 149. +**`fingerprint-presets-v150.json`** — Newer bundle covering Firefox v149–v152 (macOS 45, Windows 178, Linux 65; 288 total). Same schema as the original. Auto-selected by `load_presets()` when the active binary reports Firefox ≥ 149. -**`fonts.json`** — Complete OS-specific font lists for random font subset generation. +**`fonts.json`, `font-bases.json`, `font-groups.json`** — OS font lists, the OS-version bases and the addition units with their probabilities (see [FONTS.md](FONTS.md)). -**`voices.json`** — Complete OS-specific speech voice lists for random voice subset generation. macOS: 190 voices, Windows: 53 voices, Linux: empty. Format: `"Name:locale:type"` — names extracted at load time. +**`voice-manifests.json`, `voice-uris.json`** — The per-OS model the voice draw follows (a base, language packs and additions, each entry `"Name:locale:type"`) and the real `voiceURI` a stock browser reports for each voice. -**`properties.json`** — Includes `audio:seed` and `canvas:seed` as `CAMOU_CONFIG` properties (uint type). These enable the MaskConfig fallback in the audio and canvas patches when using global config without per-context JavaScript. +**`properties.json`** — Includes `audio:seed` as a `CAMOU_CONFIG` property (uint type), the MaskConfig fallback for the audio patch when using global config without per-context JavaScript. **`camoufox.cfg`** — Sets `fission.autostart=true` and `dom.ipc.processPrelaunch.enabled=false`. No `dom.ipc.processCount` override needed with cross-process storage. diff --git a/docs/playwright-maintenance.md b/docs/playwright-maintenance.md index a4aeb9202..b7e09deca 100644 --- a/docs/playwright-maintenance.md +++ b/docs/playwright-maintenance.md @@ -4,7 +4,10 @@ This document describes how to maintain Playwright integration in Camoufox. ## Overview -Camoufox integrates Playwright's browser automation capabilities through patches and additional files. These need to be kept in sync with upstream Playwright development. +Camoufox integrates Playwright's browser automation through a patch and a copy of +Playwright's Juggler protocol. Both started from upstream Playwright and both +carry Camoufox changes, so upstream updates are ported into them, never copied +over them. ## Patch Files @@ -12,106 +15,78 @@ Location: `patches/playwright/` | File | Purpose | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `0-playwright.patch` | Playwright's upstream patches. Must be kept up to date with [bootstrap.diff](https://github.com/microsoft/playwright/blob/main/browser_patches/firefox/patches/bootstrap.diff) | -| `1-leak-fixes.patch` | Undos certain patches from `0-playwright.patch` to fix memory leaks | +| `0-playwright.patch` | Playwright's [bootstrap.diff](https://github.com/microsoft/playwright/blob/main/browser_patches/firefox/patches/bootstrap.diff), ported to the Firefox version in `upstream.sh`, plus Camoufox fixes to the Juggler input and navigation paths | +| `1-leak-fixes.patch` | Undoes two changes from `0-playwright.patch` that expose automation: `navigator.webdriver` always reports `false`, and enterprise policies load from Firefox's normal provider instead of Playwright's | + +Both sort ahead of every other patch, so `scripts/patch.py` applies them first. ## Addition Files Location: `additions/juggler/` -The `juggler` directory contains Playwright's Juggler protocol implementation. These files must be kept in sync with: - -**Upstream Source:** https://github.com/microsoft/playwright/tree/main/browser_patches/firefox/juggler +Camoufox's Juggler, started from +[upstream](https://github.com/microsoft/playwright/tree/main/browser_patches/firefox/juggler). +Camoufox changes include the isolated-world page agent and the human cursor +(`input/`), so a straight copy from upstream would remove them. ### Key Files -- **`components/Juggler.js`** - Main Juggler component (legacy JSM format) -- **`components/Juggler.sys.mjs`** - ESM wrapper for Firefox 146+ compatibility +- **`components/Juggler.js`** - Main Juggler component, an ES module that exports `JugglerFactory` - **`components/components.conf`** - XPCOM component registration +- **`jar.mn`** - What gets packaged into `chrome://juggler/content/` -### Firefox 146 ESM Migration - -Firefox 146 removed JSM (JavaScript Module) support in favor of ESM (ES Modules). To maintain compatibility: - -1. **`components.conf`** uses `esModule` field instead of deprecated `jsm` field: - ```python - { - "esModule": "chrome://juggler/content/components/Juggler.sys.mjs", - "constructor": "JugglerFactory", - } - ``` - -2. **`Juggler.sys.mjs`** acts as an ESM wrapper that imports the legacy JSM file: - ```javascript - const { JugglerFactory } = ChromeUtils.import( - "chrome://juggler/content/components/Juggler.js" - ); - export { JugglerFactory }; - ``` +`components.conf` registers the component with the `esModule` field (Firefox +no longer supports `jsm`): -This maintains backward compatibility while satisfying Firefox 146's static component generator requirements. +```python +{ + "esModule": "chrome://juggler/content/components/Juggler.js", + "constructor": "JugglerFactory", +} +``` ## Updating Playwright Integration -### 1. Update Upstream Patches +### 1. Port Upstream Patch Changes -Compare the current `patches/playwright/0-playwright.patch` with Playwright's [bootstrap.diff](https://github.com/microsoft/playwright/blob/main/browser_patches/firefox/patches/bootstrap.diff). +Compare what changed in Playwright's +[bootstrap.diff](https://github.com/microsoft/playwright/blob/main/browser_patches/firefox/patches/bootstrap.diff) +since the last sync and apply those changes to the tree, then regenerate the +patch with the make targets described in +[patch-upgrading-guide.md](patch-upgrading-guide.md): -If changes are needed: ```bash -# Download latest bootstrap.diff -curl -o patches/playwright/0-playwright.patch \ - https://raw.githubusercontent.com/microsoft/playwright/main/browser_patches/firefox/patches/bootstrap.diff - -# Test the build -make clean && make dir && make build +make dir +make workspace ./patches/playwright/0-playwright.patch +# port the upstream changes into camoufox--/ +make diff > patches/playwright/0-playwright.patch ``` -### 2. Update Juggler Files - -Sync `additions/juggler/` with upstream: +### 2. Port Juggler Changes ```bash -# Clone Playwright repository git clone https://github.com/microsoft/playwright.git /tmp/playwright - -# Compare directories diff -r additions/juggler/ /tmp/playwright/browser_patches/firefox/juggler/ - -# Copy updated files (example) -cp -r /tmp/playwright/browser_patches/firefox/juggler/* additions/juggler/ - -# IMPORTANT: Preserve Firefox 146 ESM compatibility -# - Keep additions/juggler/components/Juggler.sys.mjs -# - Keep additions/juggler/components/components.conf with esModule field ``` -### 3. Verify ESM Wrapper Compatibility - -After updating from upstream, ensure the ESM wrapper remains functional: +Port the upstream hunks one by one, keeping the Camoufox changes. After an +update, check that: -1. Check that `Juggler.js` still exports `JugglerFactory`: - ```javascript - var EXPORTED_SYMBOLS = ["Juggler", "JugglerFactory"]; - var JugglerFactory = function() { /* ... */ }; - ``` +1. `Juggler.js` still exports `JugglerFactory`, and `components.conf` still + names it as the `constructor`. +2. `components.conf` uses `esModule`, not `jsm`. +3. Every file upstream added or renamed is listed in `jar.mn`. -2. If upstream changed the export name, update `Juggler.sys.mjs` accordingly. - -3. Verify `components.conf` matches the format above (not upstream's format). - -### 4. Test Build +### 3. Test ```bash -# Clean build to verify component registration -cd camoufox-146.0.1-beta.25 -make clean -cd .. -make dir -cd camoufox-146.0.1-beta.25 -./mach build +make dir && make build +make tests ``` +`make dir` resets the tree, clobbers the object directory and reapplies every +patch, so this is a clean build. + **Expected:** No linker errors about `mozCreateComponent`. **Common Error:** If you see: @@ -129,7 +104,7 @@ This means `components.conf` is not using ESM format. Fix by ensuring it has: **Error:** `Externally-constructed components may not specify 'constructor' or 'legacy_constructor' properties` -**Cause:** Using `"jsm"` field which is unsupported in Firefox 146. +**Cause:** Using the `"jsm"` field, which Firefox no longer supports. **Fix:** Use `"esModule"` field instead. @@ -153,13 +128,11 @@ This means `components.conf` is not using ESM format. Fix by ensuring it has: If the build fails after updating Juggler files: -1. Check that all JSM imports in `Juggler.js` are still valid -2. Verify the ESM wrapper exports match what's imported -3. Ensure no file paths changed in upstream -4. Check for Firefox API changes that might require patches +1. Check that every `ChromeUtils.importESModule()` path in the Juggler files still exists +2. Check that new or renamed files are listed in `jar.mn` +3. Check for Firefox API changes that might require patches ## References - [Playwright Firefox Patches](https://github.com/microsoft/playwright/tree/main/browser_patches/firefox) -- [Firefox 146 Component Registration](https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/basics.html) - [Firefox ESM Migration Guide](https://firefox-source-docs.mozilla.org/dom/script_loader/index.html) diff --git a/jsonvv/README.md b/jsonvv/README.md deleted file mode 100644 index b11a14c7c..000000000 --- a/jsonvv/README.md +++ /dev/null @@ -1,728 +0,0 @@ -# JSONvv - -JSON value validator - -## Overview - -This is a simple JSON schema validator library. It was created for Camoufox to validate passed user configurations. Because I found it useful for other projects, I decided to extract it into a separate library. - -JSONvv's syntax parser is written in pure Python. It does not rely on any dependencies. - -### Example - - - - - - - - - - -
ConfigurationValidator
- -```python -config = { - "username": "johndoe", - "email": "johndoe@example.com", - "age": 30, - "chat": "Hello world!", - "preferences": { - "notifications": True, - "theme": "dark" - }, - "allowed_commands": [ - "/help", "/time", "/weather" - ], - "location": [40.7128, -74.0060], - "hobbies": [ - { - "name": "Traveling", - "cities": ["Paris", "London"] - }, - { - "name": "reading", - "hours": { - "Sunday": 2, - "Monday": 3, - } - } - ] -} -``` - - - -```python -validator = { - "username": "str", # Basic username - "email": "str[/\S+@\S+\.\S+/]", # Validate emails - "age": "int[>=18]", # Age must be 18 or older - "chat": "str | nil", # Optional chat message - "preferences": { - "notifications": "bool", - "theme": "str[light, dark] | nil", # Optional theme - }, - # Commands must start with "/", but not contain "sudo" - "allowed_commands": "array[str[/^//] - str[/sudo/]]", - # Validate coordinate ranges - "location": "tuple[double[-90 - 90], double[-180 - 180]]", - # Handle an array of hobby types - "hobbies": "array[@traveling | @other, >=1]", - "@traveling": { - # Require 1 or more cities/countries iff name is "Traveling" - "*name,type": "str[Traveling]", - "*cities,countries": "array[str[A-Za-z*], >=1]", - }, - "@other": { - "name,type": "str - str[Traveling]", # Non-traveling types - # If hour(s) is specified, require days have >0 hours - "/hours?/": { - "*/day$/": "int[>0]" - } - } -} -``` - -
- -
- -Then, validate the configuration like this: - -```python -from jsonvv import JsonValidator, JvvRuntimeException - -val = JsonValidator(validator) -try: - val.validate(config) -except JvvRuntimeException as exc: - print("Failed:", exc) -else: - print('Config is valid!') -``` - ---- - -## Table of Contents - -- [Key Syntax](#key-syntax) - - [Regex patterns](#regex-patterns) - - [Lists of possible values](#lists-of-possible-values) - - [Required fields (`*`)](#required-fields-) - - [Grouping keys (`$`)](#grouping-keys-) -- [Supported Types](#supported-types) - - [String (`str`)](#string-str) - - [Integer (`int`)](#integer-int) - - [Double (`double`)](#double-double) - - [Boolean (`bool`)](#boolean-bool) - - [Array (`array`)](#array-array) - - [Tuple (`tuple`)](#tuple-tuple) - - [Nested Dictionaries](#nested-dictionaries) - - [Nil (`nil`)](#nil-nil) - - [Any (`any`)](#any-any) - - [Required fields (`*`)](#required-fields-) - - [Type References (`@`)](#type-references-) -- [Advanced Features](#advanced-features) - - [Subtracting Domains (`-`)](#subtracting-domains--) - - [Union Types (`|`)](#union-types-) - - [Conditional Ranges and Values](#conditional-ranges-and-values) -- [Error Handling](#error-handling) - ---- - -## Keys Syntax - -Dictionary keys can be specified in several possible ways: - -- `"key": "type"` -- `"key1,key2,key3": "type"` -- `"/key\d+/": "type"` -- `"*required_key": "type"` - -### Regex patterns - -To use regex in a key, wrap it in `/ ... /`. - -**Syntax:** - -```python -"/key\d+/": "type" -``` - -### Lists of possible values - -To specify a list of keys, use a comma-separated string. - -**Syntax:** - -```python -"key1,key2,key3": "type" -"/k[ey]{2}1/,key2": "type" -``` - -To escape a comma, use `!`. - -### Required fields (`*`) - -Fields marked with `*` are required. The validation will fail without them. - -**Syntax:** - -```python -"*key1": "type" -"*/key\d+/": "type" -``` - -### Grouping keys (`$`) - -Fields that end with `$group_name` are grouped together. If one of the keys is set, all of the keys in the group must also be set as well. - -**Syntax:** - -```python -"isEnabled$group1": "bool" -"value$group1": "int[>0]" -``` - -This will require both `value` is set if and only if `isEnabled` is set. - -Multiple `$` can be used to create more complex group dependencies. - ---- - -## Supported Types - -### String (`str`) - -Represents a string value. Optionally, you can specify a regex pattern that the string must match. - -**Syntax:** - -- Basic string: `"str"` -- With regex pattern: `"str[regex_pattern]"` -- The escape character for regex is `\`, and for commas is `_`. - -**Arguments:** - -- `regex_pattern`: A regular expression that the string must match. If not specified, any string is accepted. - -**Examples:** - -1. Basic string: - - ```python - "username": "str" - ``` - - Accepts any string value for the key `username`. - -2. String with regex pattern: - - ```python - "fullname": "str[/[A-Z][a-z]+ [A-Z][a-z]+/]" - ``` - - Accepts a string that matches the pattern of a first and last name starting with uppercase letters. - -### Integer (`int`) - -Represents an integer value. You can specify conditions like exact values, ranges, and inequalities. - -**Syntax:** - -- Basic integer: `"int"` -- With conditions: `"int[conditions]"` - -**Arguments:** - -- `conditions`: A comma-separated list of conditions. - -**Condition Operators:** - -- `==`: Equal to a specific value. -- `>=`: Greater than or equal to a value. -- `<=`: Less than or equal to a value. -- `>`: Greater than a value. -- `<`: Less than a value. -- `range`: A range between two values (inclusive). - -**Examples:** - -1. Basic integer: - - ```python - "age": "int" - ``` - - Accepts any integer value for the key `age`. - -2. Integer with conditions: - - ```python - "userage": "int[>=0, <=120]" - ``` - - Accepts integer values between 0 and 120 inclusive. - -3. Specific values and ranges - - ```python - "rating": "int[1-5]" - "rating": "int[1,2,3,4-5]" - ``` - - Accepts integer values 1, 2, 3, 4, or 5. - -4. Ranges with negative numbers: - - ```python - "rating": "int[-100 - -90]" - ``` - - Accepts integer values from -100 to -90. - -### Double (`double`) - -Represents a floating-point number. Supports the same conditions as integers. - -**Syntax:** - -- Basic double: `"double"` -- With conditions: `"double[conditions]"` - -**Arguments:** - -- `conditions`: A comma-separated list of conditions. - -**Examples:** - -1. Basic double: - - ```python - "price": "double" - ``` - - Accepts any floating-point number for the key `price`. - -2. Double with conditions: - - ```python - "percentage": "double[>=0.0,<=100.0]" - ``` - - Accepts double values between 0.0 and 100.0 inclusive. - -### Boolean (`bool`) - -Represents a boolean value (`True` or `False`). - -**Syntax:** - -```python -"isActive": "bool" -``` - -Accepts a boolean value for the key `isActive`. - -### Array (`array`) - -Represents a list of elements of a specified type. You can specify conditions on the length of the array. - -**Syntax:** - -- Basic array: `"array[element_type]"` -- With length conditions: `"array[element_type,length_conditions]"` - -**Arguments:** - -- `element_type`: The type of the elements in the array. -- `length_conditions`: Conditions on the array length (same as integer conditions). - -**Examples:** - -1. Basic array: - - ```python - "tags": "array[str]" - ``` - - Accepts a list of strings for the key `tags`. - -2. Array with length conditions: - - ```python - "scores": "array[int[>=0,<=100],>=1,<=5]" - ``` - - Accepts a list of 1 to 5 integers between 0 and 100 inclusive. - -3. Fixed-length array: - - ```python - "coordinates": "array[double, 2]" - ``` - - Accepts a list of exactly 2 double values. - -4. More complex restraints: - ```python - "coordinates": "array[array[int[>0]] - tuple[1, 1]], 2]" - ``` - -### Tuple (`tuple`) - -Represents a fixed-size sequence of elements of specified types. - -**Syntax:** - -```python -"tuple[element_type1, element_type2]" -``` - -**Arguments:** - -- `element_typeN`: The type of the Nth element in the tuple. - -**Examples:** - -1. Basic tuple: - - ```python - "point": "tuple[int, int]" - ``` - - Accepts a tuple or list of two integers. - -2. Tuple with mixed types: - - ```python - "userInfo": "tuple[str, int, bool]" - ``` - - Accepts a tuple of a string, an integer, and a boolean. - -### Nested Dictionaries - -Represents a nested dictionary structure. Dictionaries are defined using Python's dictionary syntax `{}` in the type definitions. - -**Syntax:** - -```python -"settings": { - "volume": "int[>=0,<=100]", - "brightness": "int[>=0,<=100]", - "mode": "str" -} -``` - -**Usage:** - -- Define the expected keys and their types within the dictionary. -- You can use all the supported types for the values. - -**Examples:** - -1. Nested dictionary: - - ```python - "user": { - "name": "str", - "age": "int[>=0]", - "preferences": { - "theme": "str", - "notifications": "bool" - } - } - ``` - - Defines a nested dictionary structure for the key `user`. - -### Nil (`nil`) - -Represents a `None` value. - -**Syntax:** - -```python -"optionalValue": "int | nil" -``` - -**Usage:** - -- Use `nil` to allow a value to be `None`. -- Often used with union types to specify optional values. - -### Any (`any`) - -Represents any value. - -**Syntax:** - -```python -"metadata": "any" -``` - -**Usage:** - -- Use `any` when any value is acceptable. -- Useful for keys where the value is not constrained. - -### Type References (`@`) - -Allows you to define reusable types and reference them. - -**Syntax:** - -- Define a named type: - - ```python - "@typeName": "type_definition" - ``` - -- Reference a named type: - - ```python - "key": "@typeName" - ``` - -**Examples:** - -1. Defining and using a named type: - - ```python - "@positiveInt": "int[>0]" - "userId": "@positiveInt" - ``` - - Defines a reusable type `@positiveInt` and uses it for the key `userId`. - ---- - -## Advanced Features - -### Subtracting Domains (`-`) - -Allows you to specify that a value should not match a certain type or condition. - -**Syntax:** - -```python -"typeA - typeB" -``` - -**Usage:** - -- The value must match `typeA` but not `typeB`. - -**Examples:** - -1. Excluding certain strings: - - ```python - "message": "str - str[.*error.*]" - ``` - - Accepts any string that does not match the regex pattern `.*error.*`. - -2. Excluding a range of numbers: - - ```python - "score": "int[0-100] - int[>=90]" - ``` - - Accepts integers between 0 and 100, excluding values greater than or equal to 90. - -3. Excluding multiple types: - - ```python - "score": "int[>0,<100] - int[>90] - int[<10]" - # Union, then subtraction: - "score": "int[>0,<100] - int[>90] | int[<10]" - "score": "int[>0,<100] - (int[>90] | int[<10])" # same thing - # Use parenthesis to run subtraction first - "score": "int[>0,<50] | (int[<100] - int[<10])" - "score": "(int[<100] - int[<10]) | int[>0,<50]" - ``` - - **Note**: Union is handled before subtraction. - -4. Allowing all but a specific value: - - ```python - "specialNumber": "any - int[0]" - ``` - -### Union Types (`|`) - -Allows you to specify that a value can be one of multiple types. - -**Syntax:** - -```python -"typeA | typeB | typeC" -``` - -**Usage:** - -- The value must match at least one of the specified types. - -**Examples:** - -1. Multiple possible types: - - ```python - "data": "int | str | bool" - ``` - - Accepts an integer, string, or boolean value for the key `data`. - -2. Combining with arrays: - - ```python - "mixedList": "array[int | str]" - ``` - - Accepts a list of integers or strings. - -### Conditional Ranges and Values - -Specifies conditions that values must satisfy, including ranges and specific values. - -**Syntax:** - -- Greater than: `">value"` -- Less than: `"=value"` -- Less than or equal to: `"<="value"` -- Range: `"start-end"` -- Specific values: `"value1,value2,value3"` - -**Examples:** - -1. Integer conditions: - - ```python - "level": "int[>=1,<=10]" - ``` - - Accepts integers from 1 to 10 inclusive. - -2. Double with range: - - ```python - "latitude": "double[-90.0 - 90.0]" - ``` - - Accepts doubles between -90.0 and 90.0 inclusive. - -3. Specific values: - - ```python - "status": "int[1,2,3]" - ``` - - Accepts integers that are either 1, 2, or 3. - ---- - -## Error Handling - -```mermaid -graph TD - Exception --> JvvException - JvvException --> JvvRuntimeException - JvvException --> JvvSyntaxError - - JvvRuntimeException --> UnknownProperty["UnknownProperty
Raised when a key in config
isn't defined in property types"] - JvvRuntimeException --> InvalidPropertyType["InvalidPropertyType
Raised when a value doesn't
match its type definition"] - InvalidPropertyType --> MissingRequiredKey["MissingRequiredKey
Raised when a required key
is missing from config"] - MissingRequiredKey --> MissingGroupKey["MissingGroupKey
Raised when some keys in a
property group are missing"] - - JvvSyntaxError --> PropertySyntaxError["PropertySyntaxError
Raised when property type
definitions have syntax errors"] - - classDef base fill:#eee,stroke:#333,stroke-width:2px; - classDef jvv fill:#d4e6f1,stroke:#2874a6,stroke-width:2px; - classDef runtime fill:#d5f5e3,stroke:#196f3d,stroke-width:2px; - classDef syntax fill:#fdebd0,stroke:#b9770e,stroke-width:2px; - classDef error fill:#fadbd8,stroke:#943126,stroke-width:2px; - - class Exception base; - class JvvException jvv; - class JvvRuntimeException,JvvSyntaxError runtime; - class PropertySyntaxError syntax; - class UnknownProperty,InvalidPropertyType,MissingRequiredKey,MissingGroupKey error; -``` - ---- - -### Types - -- **str**: Basic string type. - - - Arguments: - - `regex_pattern` (optional): A regex pattern the string must match. - - Example: `"str[^[A-Za-z]+$]"` - -- **int**: Integer type with conditions. - - - Arguments: - - `conditions`: Inequalities (`>=`, `<=`, `>`, `<`), specific values (`value1,value2`), ranges (`start-end`). - - Example: `"int[>=0,<=100]"` - -- **double**: Double (floating-point) type with conditions. - - - Arguments: - - Same as `int`. - - Example: `"double[>0.0]"` - -- **bool**: Boolean type. - - - Arguments: None. - - Example: `"bool"` - -- **array**: Array (list) of elements of a specified type. - - - Arguments: - - `element_type`: Type of elements in the array. - - `length_conditions` (optional): Conditions on the array length. - - Example: `"array[int[>=0],>=1,<=10]"` - -- **tuple**: Fixed-size sequence of elements of specified types. - - - Arguments: - - List of element types. - - Example: `"tuple[str, int, bool]"` - -- **nil**: Represents a `None` value. - - - Arguments: None. - - Example: `"nil"` - -- **any**: Accepts any value. - - - Arguments: None. - - Example: `"any"` - -- **Type References**: Reusable type definitions. - - Arguments: - - `@typeName`: Reference to a named type. - - Example: - - Define: `"@positiveInt": "int[>0]"` - - Use: `"userId": "@positiveInt"` - -### Type Combinations - -- **Union Types** (`|`): Value must match one of multiple types. - - - Syntax: `"typeA | typeB"` - - Example: `"str | int"` - -- **Subtracting Domains** (`-`): Value must match `typeA` but not `typeB`. - - Syntax: `"typeA - typeB"` - - Example: `"int - int[13]"` (any integer except 13) - -### Escaping Characters - -- `!`: Escapes commas, slashes, and other jsonvv characters within strings. -- `\`: Escapes within a regex pattern. diff --git a/jsonvv/jsonvv/__init__.py b/jsonvv/jsonvv/__init__.py deleted file mode 100644 index c63e43551..000000000 --- a/jsonvv/jsonvv/__init__.py +++ /dev/null @@ -1,21 +0,0 @@ -from .exceptions import ( - InvalidPropertyType, - JvvException, - JvvRuntimeException, - JvvSyntaxError, - MissingRequiredKey, - PropertySyntaxError, - UnknownProperty, -) -from .validator import JsonValidator - -__all__ = [ - 'JvvRuntimeException', - 'JvvSyntaxError', - 'PropertySyntaxError', - 'JsonValidator', - 'JvvException', - 'InvalidPropertyType', - 'UnknownProperty', - 'MissingRequiredKey', -] diff --git a/jsonvv/jsonvv/__main__.py b/jsonvv/jsonvv/__main__.py deleted file mode 100644 index 2d9fa8bc4..000000000 --- a/jsonvv/jsonvv/__main__.py +++ /dev/null @@ -1,70 +0,0 @@ -import argparse -import json -import sys -from pathlib import Path -from typing import Any, Dict - -from jsonvv.exceptions import InvalidPropertyType, JvvSyntaxError, UnknownProperty -from jsonvv.validator import JsonValidator - - -def load_json(file_path: Path) -> Dict[str, Any]: - """ - Load and parse a JSON file. - """ - try: - with open(file_path) as f: - return json.load(f) - except json.JSONDecodeError as e: - raise ValueError(f"Invalid JSON in {file_path}: {e}") - except FileNotFoundError: - raise ValueError(f"File not found: {file_path}") - - -def main(): - """JSON Value Validator - Validate JSON data against a schema.""" - parser = argparse.ArgumentParser( - description="JSON Value Validator - Validate JSON data against a schema." - ) - parser.add_argument( - 'properties_file', type=Path, help='JSON file containing the property type definitions' - ) - parser.add_argument( - '-i', '--input', type=Path, help='JSON file containing the data to validate' - ) - parser.add_argument( - '--check', action='store_true', help='Check if the properties file is valid' - ) - - args = parser.parse_args() - - try: - # Load property types - property_types = load_json(args.properties_file) - validator = JsonValidator(property_types) - - if args.check: - print("✓ Property types are valid") - return - - if not args.input: - parser.error("Either --input or --check must be specified") - - # Load and validate data - data = load_json(args.input) - validator.validate(data) - print("✓ Data is valid") - - except (InvalidPropertyType, UnknownProperty) as e: - print(f"Validation Error: {e}", file=sys.stderr) - sys.exit(1) - except JvvSyntaxError as e: - print(f"Syntax Error: {e}", file=sys.stderr) - sys.exit(1) - except ValueError as e: - print(f"File Error: {e}", file=sys.stderr) - sys.exit(1) - - -if __name__ == "__main__": - main() diff --git a/jsonvv/jsonvv/exceptions.py b/jsonvv/jsonvv/exceptions.py deleted file mode 100644 index 9001923bb..000000000 --- a/jsonvv/jsonvv/exceptions.py +++ /dev/null @@ -1,33 +0,0 @@ -"""Exception classes for jsonvv""" - - -class JvvException(Exception): - pass - - -class JvvRuntimeException(JvvException): - pass - - -class JvvSyntaxError(JvvException): - pass - - -class UnknownProperty(JvvRuntimeException, ValueError): - pass - - -class InvalidPropertyType(JvvRuntimeException, TypeError): - pass - - -class MissingRequiredKey(InvalidPropertyType): - pass - - -class MissingGroupKey(MissingRequiredKey): - pass - - -class PropertySyntaxError(JvvSyntaxError): - pass diff --git a/jsonvv/jsonvv/parser.py b/jsonvv/jsonvv/parser.py deleted file mode 100644 index 5b95ddced..000000000 --- a/jsonvv/jsonvv/parser.py +++ /dev/null @@ -1,309 +0,0 @@ -from dataclasses import dataclass -from typing import Any, Dict, List - -from .exceptions import InvalidPropertyType -from .strings import string_validator -from .types import ( - AnyType, - ArrayType, - BaseType, - BoolType, - DoubleType, - IntType, - NilType, - StringType, - SubtractionType, - TupleType, - Type, - UnionType, -) - - -class Parser: - def __init__(self, type_str: str): - self.type_str = type_str - self.pos = 0 - self.length = len(type_str) - - def parse(self) -> Type: - """Main entry point""" - result = self.parse_subtraction() # Start with subtraction instead of union - self.skip_whitespace() - if self.pos < self.length: - raise RuntimeError(f"Unexpected character at position {self.pos}") - return result - - def parse_union(self) -> Type: - """Handles type1 | type2 | type3""" - types = [self.parse_term()] # Parse first term - - while self.pos < self.length: - self.skip_whitespace() - if not self.match('|'): - break - types.append(self.parse_term()) # Parse additional terms - - return types[0] if len(types) == 1 else UnionType(types) - - def parse_subtraction(self) -> Type: - """Handles type1 - type2""" - left = self.parse_union() # Start with union - - while self.pos < self.length: - self.skip_whitespace() - if not self.match('-'): - break - right = self.parse_union() # Parse right side as union - left = SubtractionType(left, right) - - return left - - def parse_term(self) -> Type: - """Handles basic terms and parenthesized expressions""" - self.skip_whitespace() - - if self.match('('): - type_obj = self.parse_subtraction() # Parse subtraction inside parens - if not self.match(')'): - raise RuntimeError("Unclosed parenthesis") - return type_obj - - return self.parse_basic_type() - - def parse_basic_type(self) -> Type: - """Handles basic types with conditions""" - name = self.parse_identifier() - - # Special handling for array type - if name == 'array': - return self.parse_array_type() - - # Special handling for tuple type - if name == 'tuple': - # Don't advance position, let parse_tuple_type handle it - return self.parse_tuple_type() - - conditions = None - self.skip_whitespace() - - if self.match('['): - start = self.pos - # For all types, just capture everything until the closing bracket - bracket_count = 1 # Track nested brackets - while self.pos < self.length: - if self.type_str[self.pos] == '[': - bracket_count += 1 - elif self.type_str[self.pos] == ']': - bracket_count -= 1 - if bracket_count == 0: - break - self.pos += 1 - - if bracket_count > 0: - raise RuntimeError("Unclosed '['") - conditions = self.type_str[start : self.pos] - - if not self.match(']'): - raise RuntimeError("Expected ']'") - - # Return appropriate type based on name - if name == 'str': - return StringType(conditions) - elif name == 'int': - return IntType(conditions) - elif name == 'double': - return DoubleType(conditions) - elif name == 'bool': - return BoolType() - elif name == 'any': - return AnyType() - elif name == 'nil': - return NilType() # Add this type - elif name == 'tuple': - return self.parse_tuple_type() - elif name.startswith('@'): - return ReferenceType(name[1:]) - return BaseType(name, conditions) - - def peek(self, char: str) -> bool: - """Looks ahead for a character without advancing position""" - self.skip_whitespace() - return self.pos < self.length and self.type_str[self.pos] == char - - def parse_array_type(self) -> Type: - """Handles array[type, length?]""" - if not self.match('['): - return ArrayType(AnyType(), None) # Default array type - - # Parse the element type (which could be a complex type) - element_type = self.parse_subtraction() # Start with subtraction to handle all cases - - length_conditions = None - self.skip_whitespace() - - # Check for length conditions after comma - if self.match(','): - self.skip_whitespace() - start = self.pos - while self.pos < self.length and self.type_str[self.pos] != ']': - self.pos += 1 - if self.pos >= self.length: - raise RuntimeError("Unclosed array type") - length_conditions = self.type_str[start : self.pos].strip() - - if not self.match(']'): - raise RuntimeError("Expected ']' in array type") - - return ArrayType(element_type, length_conditions) - - def parse_tuple_type(self) -> Type: - """Handles tuple[type1, type2, ...]""" - - if not self.match('['): - raise RuntimeError("Expected '[' after 'tuple'") - - types = [] - while True: - self.skip_whitespace() - if self.match(']'): - break - - # Parse complex type expressions within tuple arguments - type_obj = self.parse_subtraction() # Start with subtraction to handle all operations - types.append(type_obj) - - self.skip_whitespace() - if not self.match(','): - if self.match(']'): - break - raise RuntimeError("Expected ',' or ']' in tuple type") - - return TupleType(types) - - def parse_identifier(self) -> str: - """Parses an identifier""" - self.skip_whitespace() - start = self.pos - - # Only consume alphanumeric and underscore characters - while self.pos < self.length and ( - self.type_str[self.pos].isalnum() or self.type_str[self.pos] in '_.@!' - ): - self.pos += 1 - - if start == self.pos: - raise RuntimeError(f'Expected identifier at position {self.pos}') - - result = self.type_str[start : self.pos] - return result - - def skip_whitespace(self) -> None: - """Skips whitespace characters""" - while self.pos < self.length and self.type_str[self.pos].isspace(): - self.pos += 1 - - def match(self, char: str) -> bool: - """Tries to match a character, advances position if matched""" - self.skip_whitespace() - if self.pos < self.length and self.type_str[self.pos] == char: - self.pos += 1 - return True - return False - - def peek_word(self, word: str) -> bool: - """Looks ahead for a word without advancing position""" - self.skip_whitespace() - return ( - self.pos + len(word) <= self.length - and self.type_str[self.pos : self.pos + len(word)] == word - and ( - self.pos + len(word) == self.length - or not self.type_str[self.pos + len(word)].isalnum() - ) - ) - - -''' -Python's import system is a pain, -so I'm moving DictType and ReferenceType here. -''' - - -@dataclass -class DictType(Type): - type_dict: Dict[str, Any] - type_registry: Dict[str, Any] - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if not isinstance(value, dict): - raise InvalidPropertyType(f"Expected dict at {'.'.join(path)}, got {type(value)}") - - # Track matched patterns and required keys - any_pattern_matched = False - required_patterns = { - pattern[1:]: False for pattern in self.type_dict if pattern.startswith('*') - } - - for key, val in value.items(): - pattern_matched = False - for pattern, type_def in self.type_dict.items(): - # Strip * for required patterns when matching - match_pattern = pattern[1:] if pattern.startswith('*') else pattern - - if string_validator(key, match_pattern): - pattern_matched = True - any_pattern_matched = True - - # Mark required pattern as found - if pattern.startswith('*'): - required_patterns[match_pattern] = True - - # Parse the type definition string into a Type object - expected_type = parse_type_def(type_def, type_registry) - expected_type.validate(val, path + [key], type_registry) - - if not pattern_matched: - raise InvalidPropertyType( - f"Key {key} at {'.'.join(path)} does not match any allowed patterns" - ) - - # Check if all required patterns were matched - missing_required = [pattern for pattern, found in required_patterns.items() if not found] - if missing_required: - raise InvalidPropertyType( - f"Missing required properties matching patterns: {', '.join(missing_required)} at {'.'.join(path)}" - ) - - if not any_pattern_matched: - raise InvalidPropertyType(f"No properties at {'.'.join(path)} matched any patterns") - - -@dataclass -class ReferenceType(Type): - name: str - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if self.name not in type_registry: - raise RuntimeError(f"Unknown type reference: @{self.name}") - - ref_type = type_registry[self.name] - - if isinstance(ref_type, dict): - # Create a DictType for dictionary references - dict_type = DictType(ref_type, type_registry) - dict_type.validate(value, path, type_registry) - else: - # For non-dictionary types - ref_type.validate(value, path, type_registry) - - def __str__(self) -> str: - return f"@{self.name}" - - -def parse_type_def(type_def: Any, type_registry: Dict[str, Type]) -> Type: - if isinstance(type_def, str): - parser = Parser(type_def) - return parser.parse() - elif isinstance(type_def, dict): - return DictType(type_def, type_registry) - raise InvalidPropertyType(f"Invalid type definition: {type_def}") diff --git a/jsonvv/jsonvv/strings.py b/jsonvv/jsonvv/strings.py deleted file mode 100644 index 56d202ba7..000000000 --- a/jsonvv/jsonvv/strings.py +++ /dev/null @@ -1,64 +0,0 @@ -import re -from typing import List - - -class StringValidator: - def __init__(self, pattern: str): - self.pattern = pattern - self.patterns = self._split_patterns(pattern) - - def _split_patterns(self, p: str) -> List[str]: - patterns = [] - current = [] - in_regex = False - i = 0 - - while i < len(p): - if p[i] == '/' and (i == 0 or p[i - 1] != '!'): - in_regex = not in_regex - current.append(p[i]) - elif p[i] == ',' and not in_regex: - # Check if comma is escaped - if i > 0 and p[i - 1] == '!': - current.append(',') - else: - # End of pattern - patterns.append(''.join(current)) - current = [] - else: - current.append(p[i]) - i += 1 - - if current: - patterns.append(''.join(current)) - - result = [p.strip() for p in patterns if p.strip()] - return result - - def _is_regex_pattern(self, p: str) -> bool: - is_regex = p.startswith('/') and p.endswith('/') and not p.endswith('!/') - return is_regex - - def _clean_literal_pattern(self, p: str) -> str: - return re.sub(r'!(.)', r'\1', p) - - def validate(self, value: str) -> bool: - for p in self.patterns: - p = self._clean_literal_pattern(p) - if self._is_regex_pattern(p): - regex = p[1:-1] - match = bool(re.match(regex, value)) - if match: - return True - else: - match = value == p - if match: - return True - - return False - - -def string_validator(value: str, pattern: str) -> bool: - validator = StringValidator(pattern) - result = validator.validate(value) - return result diff --git a/jsonvv/jsonvv/types.py b/jsonvv/jsonvv/types.py deleted file mode 100644 index 21357bd06..000000000 --- a/jsonvv/jsonvv/types.py +++ /dev/null @@ -1,262 +0,0 @@ -from abc import ABC, abstractmethod -from dataclasses import dataclass -from typing import Any, Dict, List, Optional, Union - -from .exceptions import InvalidPropertyType -from .strings import string_validator - -TYPE_NAMES = {'array', 'tuple', 'str', 'int', 'double', 'bool', 'any', 'nil', 'tuple'} - - -class Type(ABC): - @abstractmethod - def validate(self, value: Any, path: List[str], type_registry: Dict[str, 'Type']) -> None: - pass - - -@dataclass -class BaseType(Type): - """Base class for all types""" - - name: str - conditions: Optional[str] = None - - def __post_init__(self): - # Raise error early - if not self.name.startswith('@') and self.name not in TYPE_NAMES: - raise InvalidPropertyType(f'Unknown base type {self.name}') - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if self.name in type_registry: - type_registry[self.name].validate(value, path, type_registry) - else: - raise RuntimeError(f'Unknown base type {self.name}') - - -@dataclass -class NilType(Type): - """Represents a nil/null type""" - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if value is not None: - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: expected nil, got {value}" - ) - - def __str__(self) -> str: - return "nil" - - -@dataclass -class StringType(Type): - pattern: Optional[str] = None - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if not isinstance(value, str): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: expected string, got {type(value).__name__}" - ) - - if self.pattern: - if not string_validator(value, self.pattern): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: {value} does not match pattern '{self.pattern}'" - ) - - def __str__(self) -> str: - return f"str[{self.pattern}]" if self.pattern else "str" - - -@dataclass -class NumericalType(Type): - conditions: Optional[str] = None - numeric_type: Type = float # Default to float - type_name: str = "number" # For error messages - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - allowed_types = (int, float) if self.numeric_type is float else (int,) - if not isinstance(value, allowed_types): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: expected {self.type_name}, got {type(value).__name__}" - ) - if self.conditions and not self._check_conditions(self.numeric_type(value)): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: {value} does not match conditions '{self.conditions}'" - ) - - def _check_conditions(self, value: Union[int, float]) -> bool: - if not self.conditions: - return True - - # Split by comma and handle each condition - conditions = [c.strip() for c in self.conditions.split(',')] - - for condition in conditions: - try: - # Handle comparisons - if '>=' in condition: - if value >= self.numeric_type(condition.replace('>=', '')): - return True - elif '<=' in condition: - if value <= self.numeric_type(condition.replace('<=', '')): - return True - elif '>' in condition: - if value > self.numeric_type(condition.replace('>', '')): - return True - elif '<' in condition: - if value < self.numeric_type(condition.replace('<', '')): - return True - # Handle ranges (e.g., "1.5-5.5") - elif '-' in condition[1:]: - # split by the -, ignoring the first character - range_s, range_e = condition[1:].split('-', 1) - range_s = self.numeric_type(condition[0] + range_s) - range_e = self.numeric_type(range_e) - if range_s <= value <= range_e: - return True - # Handle single values - else: - if value == self.numeric_type(condition): - return True - except ValueError: - continue - - return False - - def __str__(self) -> str: - return f"{self.type_name}[{self.conditions}]" if self.conditions else self.type_name - - -@dataclass -class IntType(NumericalType): - def __init__(self, conditions: Optional[str] = None): - super().__init__(conditions=conditions, numeric_type=int, type_name="int") - - -@dataclass -class DoubleType(NumericalType): - def __init__(self, conditions: Optional[str] = None): - super().__init__(conditions=conditions, numeric_type=float, type_name="double") - - -@dataclass -class AnyType(Type): - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - # Any type accepts all values - pass - - def __str__(self) -> str: - return "any" - - -@dataclass -class BoolType(Type): - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if not isinstance(value, bool): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: expected bool, got {type(value).__name__}" - ) - - -@dataclass -class ArrayType(Type): - element_type: Type - length_conditions: Optional[str] = None - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if not isinstance(value, list): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: expected array, got {type(value).__name__}" - ) - - if self.length_conditions: - array_len = len(value) - length_validator = IntType(self.length_conditions) - try: - length_validator._check_conditions(array_len) - except Exception: - raise InvalidPropertyType( - f"Invalid array length at {'.'.join(path)}: got length {array_len}" - ) - - for i, item in enumerate(value): - self.element_type.validate(item, path + [str(i)], type_registry) - - -@dataclass -class TupleType(Type): - element_types: List[Type] - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - if not isinstance(value, (list, tuple)): - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: expected tuple, got {type(value).__name__}" - ) - - if len(value) != len(self.element_types): - raise InvalidPropertyType( - f"Invalid tuple length at {'.'.join(path)}: expected {len(self.element_types)}, got {len(value)}" - ) - - for i, (item, expected_type) in enumerate(zip(value, self.element_types)): - expected_type.validate(item, path + [str(i)], type_registry) - - -@dataclass -class UnionType(Type): - types: List[Type] - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - errors = [] - for t in self.types: - try: - t.validate(value, path, type_registry) - return # If any type validates successfully, we're done - except InvalidPropertyType as e: - errors.append(str(e)) - - # If we get here, none of the types validated - raise InvalidPropertyType( - f"Invalid value at {'.'.join(path)}: {value} does not match any of the allowed types" - ) - - def __str__(self) -> str: - return f"({' | '.join(str(t) for t in self.types)})" - - -@dataclass -class SubtractionType(Type): - base_type: Type - subtracted_type: Type - - def validate(self, value: Any, path: List[str], type_registry: Dict[str, Type]) -> None: - path_str = '.'.join(path) - - # First check if value matches base type - matches_base = True - try: - self.base_type.validate(value, path, type_registry) - except InvalidPropertyType: - matches_base = False - raise - - # Then check if value matches subtracted type - matches_subtracted = True - try: - self.subtracted_type.validate(value, path, type_registry) - matches_subtracted = True - except InvalidPropertyType: - matches_subtracted = False - - # Final validation decision - if matches_base and matches_subtracted: - raise InvalidPropertyType(f"Invalid value at {path_str}: {value} matches excluded type") - elif matches_base and not matches_subtracted: - return - else: - raise InvalidPropertyType( - f"Invalid value at {path_str}: {value} does not match base type" - ) - - def __str__(self) -> str: - return f"({self.base_type} - {self.subtracted_type})" diff --git a/jsonvv/jsonvv/validator.py b/jsonvv/jsonvv/validator.py deleted file mode 100644 index ec09ae990..000000000 --- a/jsonvv/jsonvv/validator.py +++ /dev/null @@ -1,170 +0,0 @@ -from typing import Any, Dict, List, Optional - -from .exceptions import ( - MissingGroupKey, - MissingRequiredKey, - PropertySyntaxError, - UnknownProperty, -) -from .parser import parse_type_def -from .strings import string_validator -from .types import Type - - -class JsonValidator: - def __init__(self, property_types): - self.property_types = property_types - # Create a registry for reference types and parsed type definitions - self.type_registry = {} - self.parsed_types = {} - # Track property groups - self.groups: Dict[str, List[str]] = {} - # Validate and pre-parse all type definitions - self.parse_types(property_types) - - def validate(self, config_map): - # First validate groups - self.validate_groups(config_map) - # Then validate the rest - validate_config(config_map, self.property_types, self.type_registry, self.parsed_types) - - def parse_types(self, property_types: Dict[str, Any], path: str = ""): - """Validates and pre-parses all type definitions.""" - for key, value in property_types.items(): - current_path = f"{path}.{key}" if path else key - - # Register reference types - if key.startswith('@'): - if len(key) == 1: - raise PropertySyntaxError( - f"Invalid key '{current_path}': '@' must be followed by a reference name" - ) - self.type_registry[key[1:]] = value - - # Validate key syntax for required properties - if key.startswith('*') and len(key) == 1: - raise PropertySyntaxError( - f"Invalid key '{current_path}': '*' must be followed by a property name" - ) - - # Register group dependencies - orig_key: Optional[str] = None - while (idx := key.rfind('$')) != -1: - # Get the original key before all $ - if orig_key is None: - orig_key = key.split('$', 1)[0] - # Add to group registry - key, group = key[:idx], key[idx + 1 :] - if group not in self.groups: - self.groups[group] = [] - self.groups[group].append(orig_key) - - if isinstance(value, dict): - # Recursively validate and parse nested dictionaries - self.parse_types(value, current_path) - elif isinstance(value, str): - try: - # Pre-parse the type definition and store it - self.parsed_types[current_path] = parse_type_def(value, self.type_registry) - except Exception as e: - raise PropertySyntaxError( - f"Invalid type definition for '{current_path}': {str(e)}" - ) - else: - raise PropertySyntaxError( - f"Invalid type definition for '{current_path}': must be a string or dictionary" - ) - - def validate_groups(self, config_map: Dict[str, Any]) -> None: - """Validates that grouped properties are all present or all absent.""" - group_presence: Dict[str, bool] = {} - - # Check which groups have any properties present - for group, props in self.groups.items(): - group_presence[group] = any(prop in config_map for prop in props) - - # Validate group completeness - for group, is_present in group_presence.items(): - props = self.groups[group] - if is_present: - # If any property in group exists, all must exist - missing = [prop for prop in props if prop not in config_map] - if missing: - raise MissingGroupKey( - f"Incomplete property group ${group}: missing {', '.join(missing)}" - ) - else: - # If no property in group exists, none should exist - present = [prop for prop in props if prop in config_map] - if present: - raise MissingGroupKey( - f"Incomplete property group ${group}: found {', '.join(present)} but missing {', '.join(set(props) - set(present))}" - ) - - -def validate_config( - config_map: Dict[str, Any], - property_types: Dict[str, Any], - type_registry: Dict[str, Type], - parsed_types: Dict[str, Type], - parent_registry: Dict[str, Type] = None, - path: str = "", -) -> None: - """Validates a configuration map against property types.""" - - # Create a new registry for this scope, inheriting from parent if it exists - local_registry = dict(parent_registry or type_registry) - - # Track required properties - required_props = {key[1:]: False for key in property_types if key.startswith('*')} - - # Validate each property in config - for key, value in config_map.items(): - type_def = None - current_path = f"{path}.{key}" if path else key - - # Strip group suffix for type lookup - lookup_key = key.split('$')[0] if '$' in key else key - - if lookup_key in property_types: - type_def = property_types[lookup_key] - - # If the value is a dict and type_def is also a dict, recurse with new scope - if isinstance(value, dict) and isinstance(type_def, dict): - validate_config( - value, type_def, type_registry, parsed_types, local_registry, current_path - ) - continue - - elif '*' + lookup_key in property_types: - type_def = property_types['*' + lookup_key] - required_props[lookup_key] = True - else: - # Check pattern matches - for pattern, pattern_type in property_types.items(): - if pattern.startswith('@') or pattern.startswith('*'): - continue - pattern_base = pattern.split('$')[0] if '$' in pattern else pattern - if string_validator(lookup_key, pattern_base): - type_def = pattern_type - current_path = f"{path}.{pattern}" if path else pattern - break - - if type_def is None: - raise UnknownProperty(f"Unknown property: {key}") - - # Use pre-parsed type if available, otherwise parse it - expected_type = parsed_types.get(current_path) - if expected_type is None: - expected_type = parse_type_def(type_def, local_registry) - expected_type.validate(value, [key], local_registry) - - # Check for missing required properties - missing_required = [key for key, found in required_props.items() if not found] - if missing_required: - raise MissingRequiredKey(f"Missing required properties: {', '.join(missing_required)}") - - # Check for missing required properties - missing_required = [key for key, found in required_props.items() if not found] - if missing_required: - raise MissingRequiredKey(f"Missing required properties: {', '.join(missing_required)}") diff --git a/jsonvv/publish.sh b/jsonvv/publish.sh deleted file mode 100644 index d09aeec87..000000000 --- a/jsonvv/publish.sh +++ /dev/null @@ -1,12 +0,0 @@ -rm -rf ./dist - -vermin . --eval-annotations --target=3.8 --violations jsonvv/ || exit 1 - -python -m build -twine check dist/* - -read -p "Confirm publish? (y/n) " -n 1 -r -echo -if [[ $REPLY =~ ^[Yy]$ ]]; then - twine upload dist/* -fi diff --git a/jsonvv/pyproject.toml b/jsonvv/pyproject.toml deleted file mode 100644 index 7806339c7..000000000 --- a/jsonvv/pyproject.toml +++ /dev/null @@ -1,25 +0,0 @@ -[build-system] -requires = ["poetry-core>=1.0.0"] -build-backend = "poetry.core.masonry.api" - -[tool.poetry] -name = "jsonvv" -version = "0.2.2" -description = "JSON value validator" -authors = ["daijro "] -license = "MIT" -repository = "https://github.com/daijro/camoufox" -homepage = "https://github.com/daijro/camoufox/tree/main/pythonlib/jsonvv" -readme = "README.md" -keywords = [ - "json", - "validator", - "validation", - "typing", -] - -[tool.poetry.dependencies] -python = "^3.8" - -[tool.poetry.scripts] -jsonvv = "jsonvv.__main__:main" diff --git a/legacy/README.md b/legacy/README.md deleted file mode 100644 index 96e1e2e34..000000000 --- a/legacy/README.md +++ /dev/null @@ -1,6 +0,0 @@ -## Deprecated Assets - -##### 2024-11-21 - -- Old launcher has been deprecated due to it not supporting non-Linux platforms, and for using FF's debugging protocol to load addons [#90](https://github.com/daijro/camoufox/issues/90). -- `generate-locales.sh` (based on [LibreWolf's locale build system](https://gitlab.com/librewolf-community/browser/source/-/blob/3dc56de7b0665724bf3842198cebe961c42a81e0/scripts/generate-locales.sh)) was deprecated due to "Camoufox" leaking to the page [#90](https://github.com/daijro/camoufox/issues/90). \ No newline at end of file diff --git a/legacy/launcher/build.sh b/legacy/launcher/build.sh deleted file mode 100644 index 5cbacc615..000000000 --- a/legacy/launcher/build.sh +++ /dev/null @@ -1,34 +0,0 @@ -#!/bin/bash - -if [ $# -ne 2 ]; then - echo "Usage: $0 " - echo "arch: x86_64, i686, arm64" - echo "os: linux, windows, macos" - exit 1 -fi - -ARCH=$1 -OS=$2 - -case $ARCH in - x86_64) GOARCH=amd64 ;; - i686) GOARCH=386 ;; - arm64) GOARCH=arm64 ;; - *) echo "Invalid architecture"; exit 1 ;; -esac - -case $OS in - linux) GOOS=linux ;; - windows) GOOS=windows ;; - macos) GOOS=darwin ;; - *) echo "Invalid OS"; exit 1 ;; -esac - -[ "$OS" = "windows" ] && OUTPUT="launch.exe" || OUTPUT="launch" - -rm -rf ./dist launch launch.exe - -echo Building launcher... -GOOS=$GOOS GOARCH=$GOARCH go build -o dist/$OUTPUT || exit 1 - -echo "Complete: launcher/dist/$OUTPUT" \ No newline at end of file diff --git a/legacy/launcher/constants.go b/legacy/launcher/constants.go deleted file mode 100644 index 58ac57e1c..000000000 --- a/legacy/launcher/constants.go +++ /dev/null @@ -1,47 +0,0 @@ -package main - -import ( - "regexp" - "strings" -) - -// Default addons to extract to /addons -var DefaultAddons = map[string]string{ - "uBO": "https://addons.mozilla.org/firefox/downloads/latest/ublock-origin/latest.xpi", - "BPC": "https://gitflic.ru/project/magnolia1234/bpc_uploads/blob/raw?file=bypass_paywalls_clean-latest.xpi", -} - -// Exclude lines from output -var ExclusionRules = []string{ - // Ignore search related warnings - "^console\\.error:\\ Search", - "SearchService", - "SearchEngineSelector", - // Ignore glxtest errors - "\\[GFX1\\-\\]:", - // Ignore meaningless lines - "^console\\.error:\\ \\(\\{\\}\\)$", - "^console\\.error:\\ \"Could\\ not\\ record\\ event:\\ \"\\ \\(\\{\\}\\)$", - "^\\s*?$", - "Rejected by Camoufox\\.$", - // Ignore missing urlbar provider errors - "^JavaScript\\ error:\\ resource:///modules/UrlbarProvider", -} - -// Convert ExclusionRules into a regex command -var ExclusionRegex = regexp.MustCompile(".*(" + strings.Join(ExclusionRules, "|") + ").*") - -// List of fonts for each OS -var FontsByOS = map[string][]string{ - "windows": { - "Arial", "Arial Black", "Bahnschrift", "Calibri", "Calibri Light", "Cambria", "Cambria Math", "Candara", "Candara Light", "Comic Sans MS", "Consolas", "Constantia", "Corbel", "Corbel Light", "Courier New", "Ebrima", "Franklin Gothic Medium", "Gabriola", "Gadugi", "Georgia", "HoloLens MDL2 Assets", "Impact", "Ink Free", "Javanese Text", "Leelawadee UI", "Leelawadee UI Semilight", "Lucida Console", "Lucida Sans Unicode", "MS Gothic", "MS PGothic", "MS UI Gothic", "MV Boli", "Malgun Gothic", "Malgun Gothic Semilight", "Marlett", "Microsoft Himalaya", "Microsoft JhengHei", "Microsoft JhengHei Light", "Microsoft JhengHei UI", "Microsoft JhengHei UI Light", "Microsoft New Tai Lue", "Microsoft PhagsPa", "Microsoft Sans Serif", "Microsoft Tai Le", "Microsoft YaHei", "Microsoft YaHei Light", "Microsoft YaHei UI", "Microsoft YaHei UI Light", "Microsoft Yi Baiti", "MingLiU-ExtB", "MingLiU_HKSCS-ExtB", "Mongolian Baiti", "Myanmar Text", "NSimSun", "Nirmala UI", "Nirmala UI Semilight", "PMingLiU-ExtB", "Palatino Linotype", "Segoe Fluent Icons", "Segoe MDL2 Assets", "Segoe Print", "Segoe Script", "Segoe UI", "Segoe UI Black", "Segoe UI Emoji", "Segoe UI Historic", "Segoe UI Light", "Segoe UI Semibold", "Segoe UI Semilight", "Segoe UI Symbol", "Segoe UI Variable", "SimSun", "SimSun-ExtB", "Sitka", "Sitka Text", "Sylfaen", "Symbol", "Tahoma", "Times New Roman", "Trebuchet MS", "Twemoji Mozilla", "Verdana", "Webdings", "Wingdings", "Yu Gothic", "Yu Gothic Light", "Yu Gothic Medium", "Yu Gothic UI", "Yu Gothic UI Light", "Yu Gothic UI Semibold", "Yu Gothic UI Semilight", "\u5b8b\u4f53", "\u5fae\u8edf\u6b63\u9ed1\u9ad4", "\u5fae\u8edf\u6b63\u9ed1\u9ad4 Light", "\u5fae\u8f6f\u96c5\u9ed1", "\u5fae\u8f6f\u96c5\u9ed1 Light", "\u65b0\u5b8b\u4f53", "\u65b0\u7d30\u660e\u9ad4-ExtB", "\u6e38\u30b4\u30b7\u30c3\u30af", "\u6e38\u30b4\u30b7\u30c3\u30af Light", "\u6e38\u30b4\u30b7\u30c3\u30af Medium", "\u7d30\u660e\u9ad4-ExtB", "\u7d30\u660e\u9ad4_HKSCS-ExtB", "\ub9d1\uc740 \uace0\ub515", "\ub9d1\uc740 \uace0\ub515 Semilight", "\uff2d\uff33 \u30b4\u30b7\u30c3\u30af", "\uff2d\uff33 \uff30\u30b4\u30b7\u30c3\u30af", - }, - - "macos": { - ".Al Bayan PUA", ".Al Nile PUA", ".Al Tarikh PUA", ".Apple Color Emoji UI", ".Apple SD Gothic NeoI", ".Aqua Kana", ".Aqua Kana Bold", ".Aqua \u304b\u306a", ".Aqua \u304b\u306a \u30dc\u30fc\u30eb\u30c9", ".Arial Hebrew Desk Interface", ".Baghdad PUA", ".Beirut PUA", ".Damascus PUA", ".DecoType Naskh PUA", ".Diwan Kufi PUA", ".Farah PUA", ".Geeza Pro Interface", ".Geeza Pro PUA", ".Helvetica LT MM", ".Hiragino Kaku Gothic Interface", ".Hiragino Sans GB Interface", ".Keyboard", ".KufiStandardGK PUA", ".LastResort", ".Lucida Grande UI", ".Muna PUA", ".Nadeem PUA", ".New York", ".Noto Nastaliq Urdu UI", ".PingFang HK", ".PingFang SC", ".PingFang TC", ".SF Arabic", ".SF Arabic Rounded", ".SF Compact", ".SF Compact Rounded", ".SF NS", ".SF NS Mono", ".SF NS Rounded", ".Sana PUA", ".Savoye LET CC.", ".ThonburiUI", ".ThonburiUIWatch", ".\u82f9\u65b9-\u6e2f", ".\u82f9\u65b9-\u7b80", ".\u82f9\u65b9-\u7e41", ".\u860b\u65b9-\u6e2f", ".\u860b\u65b9-\u7c21", ".\u860b\u65b9-\u7e41", "Academy Engraved LET", "Al Bayan", "Al Nile", "Al Tarikh", "American Typewriter", "Andale Mono", "Apple Braille", "Apple Chancery", "Apple Color Emoji", "Apple SD Gothic Neo", "Apple SD \uc0b0\ub3cc\uace0\ub515 Neo", "Apple Symbols", "AppleGothic", "AppleMyungjo", "Arial", "Arial Black", "Arial Hebrew", "Arial Hebrew Scholar", "Arial Narrow", "Arial Rounded MT Bold", "Arial Unicode MS", "Athelas", "Avenir", "Avenir Black", "Avenir Black Oblique", "Avenir Book", "Avenir Heavy", "Avenir Light", "Avenir Medium", "Avenir Next", "Avenir Next Condensed", "Avenir Next Condensed Demi Bold", "Avenir Next Condensed Heavy", "Avenir Next Condensed Medium", "Avenir Next Condensed Ultra Light", "Avenir Next Demi Bold", "Avenir Next Heavy", "Avenir Next Medium", "Avenir Next Ultra Light", "Ayuthaya", "Baghdad", "Bangla MN", "Bangla Sangam MN", "Baskerville", "Beirut", "Big Caslon", "Bodoni 72", "Bodoni 72 Oldstyle", "Bodoni 72 Smallcaps", "Bodoni Ornaments", "Bradley Hand", "Brush Script MT", "Chalkboard", "Chalkboard SE", "Chalkduster", "Charter", "Charter Black", "Cochin", "Comic Sans MS", "Copperplate", "Corsiva Hebrew", "Courier", "Courier New", "Czcionka systemowa", "DIN Alternate", "DIN Condensed", "Damascus", "DecoType Naskh", "Devanagari MT", "Devanagari Sangam MN", "Didot", "Diwan Kufi", "Diwan Thuluth", "Euphemia UCAS", "Farah", "Farisi", "Font Sistem", "Font de sistem", "Font di sistema", "Font sustava", "Fonte do Sistema", "Futura", "GB18030 Bitmap", "Galvji", "Geeza Pro", "Geneva", "Georgia", "Gill Sans", "Grantha Sangam MN", "Gujarati MT", "Gujarati Sangam MN", "Gurmukhi MN", "Gurmukhi MT", "Gurmukhi Sangam MN", "Heiti SC", "Heiti TC", "Heiti-\uac04\uccb4", "Heiti-\ubc88\uccb4", "Helvetica", "Helvetica Neue", "Herculanum", "Hiragino Kaku Gothic Pro", "Hiragino Kaku Gothic Pro W3", "Hiragino Kaku Gothic Pro W6", "Hiragino Kaku Gothic ProN", "Hiragino Kaku Gothic ProN W3", "Hiragino Kaku Gothic ProN W6", "Hiragino Kaku Gothic Std", "Hiragino Kaku Gothic Std W8", "Hiragino Kaku Gothic StdN", "Hiragino Kaku Gothic StdN W8", "Hiragino Maru Gothic Pro", "Hiragino Maru Gothic Pro W4", "Hiragino Maru Gothic ProN", "Hiragino Maru Gothic ProN W4", "Hiragino Mincho Pro", "Hiragino Mincho Pro W3", "Hiragino Mincho Pro W6", "Hiragino Mincho ProN", "Hiragino Mincho ProN W3", "Hiragino Mincho ProN W6", "Hiragino Sans", "Hiragino Sans GB", "Hiragino Sans GB W3", "Hiragino Sans GB W6", "Hiragino Sans W0", "Hiragino Sans W1", "Hiragino Sans W2", "Hiragino Sans W3", "Hiragino Sans W4", "Hiragino Sans W5", "Hiragino Sans W6", "Hiragino Sans W7", "Hiragino Sans W8", "Hiragino Sans W9", "Hoefler Text", "Hoefler Text Ornaments", "ITF Devanagari", "ITF Devanagari Marathi", "Impact", "InaiMathi", "Iowan Old Style", "Iowan Old Style Black", "J\u00e4rjestelm\u00e4fontti", "Kailasa", "Kannada MN", "Kannada Sangam MN", "Kefa", "Khmer MN", "Khmer Sangam MN", "Kohinoor Bangla", "Kohinoor Devanagari", "Kohinoor Gujarati", "Kohinoor Telugu", "Kokonor", "Krungthep", "KufiStandardGK", "Lao MN", "Lao Sangam MN", "Lucida Grande", "Luminari", "Malayalam MN", "Malayalam Sangam MN", "Marion", "Marker Felt", "Menlo", "Microsoft Sans Serif", "Mishafi", "Mishafi Gold", "Monaco", "Mshtakan", "Mukta Mahee", "MuktaMahee Bold", "MuktaMahee ExtraBold", "MuktaMahee ExtraLight", "MuktaMahee Light", "MuktaMahee Medium", "MuktaMahee Regular", "MuktaMahee SemiBold", "Muna", "Myanmar MN", "Myanmar Sangam MN", "Nadeem", "New Peninim MT", "Noteworthy", "Noto Nastaliq Urdu", "Noto Sans Adlam", "Noto Sans Armenian", "Noto Sans Armenian Blk", "Noto Sans Armenian ExtBd", "Noto Sans Armenian ExtLt", "Noto Sans Armenian Light", "Noto Sans Armenian Med", "Noto Sans Armenian SemBd", "Noto Sans Armenian Thin", "Noto Sans Avestan", "Noto Sans Bamum", "Noto Sans Bassa Vah", "Noto Sans Batak", "Noto Sans Bhaiksuki", "Noto Sans Brahmi", "Noto Sans Buginese", "Noto Sans Buhid", "Noto Sans CanAborig", "Noto Sans Canadian Aboriginal", "Noto Sans Carian", "Noto Sans CaucAlban", "Noto Sans Caucasian Albanian", "Noto Sans Chakma", "Noto Sans Cham", "Noto Sans Coptic", "Noto Sans Cuneiform", "Noto Sans Cypriot", "Noto Sans Duployan", "Noto Sans EgyptHiero", "Noto Sans Egyptian Hieroglyphs", "Noto Sans Elbasan", "Noto Sans Glagolitic", "Noto Sans Gothic", "Noto Sans Gunjala Gondi", "Noto Sans Hanifi Rohingya", "Noto Sans HanifiRohg", "Noto Sans Hanunoo", "Noto Sans Hatran", "Noto Sans ImpAramaic", "Noto Sans Imperial Aramaic", "Noto Sans InsPahlavi", "Noto Sans InsParthi", "Noto Sans Inscriptional Pahlavi", "Noto Sans Inscriptional Parthian", "Noto Sans Javanese", "Noto Sans Kaithi", "Noto Sans Kannada", "Noto Sans Kannada Black", "Noto Sans Kannada ExtraBold", "Noto Sans Kannada ExtraLight", "Noto Sans Kannada Light", "Noto Sans Kannada Medium", "Noto Sans Kannada SemiBold", "Noto Sans Kannada Thin", "Noto Sans Kayah Li", "Noto Sans Kharoshthi", "Noto Sans Khojki", "Noto Sans Khudawadi", "Noto Sans Lepcha", "Noto Sans Limbu", "Noto Sans Linear A", "Noto Sans Linear B", "Noto Sans Lisu", "Noto Sans Lycian", "Noto Sans Lydian", "Noto Sans Mahajani", "Noto Sans Mandaic", "Noto Sans Manichaean", "Noto Sans Marchen", "Noto Sans Masaram Gondi", "Noto Sans Meetei Mayek", "Noto Sans Mende Kikakui", "Noto Sans Meroitic", "Noto Sans Miao", "Noto Sans Modi", "Noto Sans Mongolian", "Noto Sans Mro", "Noto Sans Multani", "Noto Sans Myanmar", "Noto Sans Myanmar Blk", "Noto Sans Myanmar ExtBd", "Noto Sans Myanmar ExtLt", "Noto Sans Myanmar Light", "Noto Sans Myanmar Med", "Noto Sans Myanmar SemBd", "Noto Sans Myanmar Thin", "Noto Sans NKo", "Noto Sans Nabataean", "Noto Sans New Tai Lue", "Noto Sans Newa", "Noto Sans Ol Chiki", "Noto Sans Old Hungarian", "Noto Sans Old Italic", "Noto Sans Old North Arabian", "Noto Sans Old Permic", "Noto Sans Old Persian", "Noto Sans Old South Arabian", "Noto Sans Old Turkic", "Noto Sans OldHung", "Noto Sans OldNorArab", "Noto Sans OldSouArab", "Noto Sans Oriya", "Noto Sans Osage", "Noto Sans Osmanya", "Noto Sans Pahawh Hmong", "Noto Sans Palmyrene", "Noto Sans Pau Cin Hau", "Noto Sans PhagsPa", "Noto Sans Phoenician", "Noto Sans PsaPahlavi", "Noto Sans Psalter Pahlavi", "Noto Sans Rejang", "Noto Sans Samaritan", "Noto Sans Saurashtra", "Noto Sans Sharada", "Noto Sans Siddham", "Noto Sans Sora Sompeng", "Noto Sans SoraSomp", "Noto Sans Sundanese", "Noto Sans Syloti Nagri", "Noto Sans Syriac", "Noto Sans Tagalog", "Noto Sans Tagbanwa", "Noto Sans Tai Le", "Noto Sans Tai Tham", "Noto Sans Tai Viet", "Noto Sans Takri", "Noto Sans Thaana", "Noto Sans Tifinagh", "Noto Sans Tirhuta", "Noto Sans Ugaritic", "Noto Sans Vai", "Noto Sans Wancho", "Noto Sans Warang Citi", "Noto Sans Yi", "Noto Sans Zawgyi", "Noto Sans Zawgyi Blk", "Noto Sans Zawgyi ExtBd", "Noto Sans Zawgyi ExtLt", "Noto Sans Zawgyi Light", "Noto Sans Zawgyi Med", "Noto Sans Zawgyi SemBd", "Noto Sans Zawgyi Thin", "Noto Serif Ahom", "Noto Serif Balinese", "Noto Serif Hmong Nyiakeng", "Noto Serif Myanmar", "Noto Serif Myanmar Blk", "Noto Serif Myanmar ExtBd", "Noto Serif Myanmar ExtLt", "Noto Serif Myanmar Light", "Noto Serif Myanmar Med", "Noto Serif Myanmar SemBd", "Noto Serif Myanmar Thin", "Noto Serif Yezidi", "Optima", "Oriya MN", "Oriya Sangam MN", "PT Mono", "PT Sans", "PT Sans Caption", "PT Sans Narrow", "PT Serif", "PT Serif Caption", "Palatino", "Papyrus", "Party LET", "Phosphate", "Ph\u00f4ng ch\u1eef H\u1ec7 th\u1ed1ng", "PingFang HK", "PingFang SC", "PingFang TC", "Plantagenet Cherokee", "Police syst\u00e8me", "Raanana", "Rendszerbet\u0171t\u00edpus", "Rockwell", "STIX Two Math", "STIX Two Text", "STIXGeneral", "STIXIntegralsD", "STIXIntegralsSm", "STIXIntegralsUp", "STIXIntegralsUpD", "STIXIntegralsUpSm", "STIXNonUnicode", "STIXSizeFiveSym", "STIXSizeFourSym", "STIXSizeOneSym", "STIXSizeThreeSym", "STIXSizeTwoSym", "STIXVariants", "STSong", "Sana", "Sathu", "Savoye LET", "Seravek", "Seravek ExtraLight", "Seravek Light", "Seravek Medium", "Shree Devanagari 714", "SignPainter", "SignPainter-HouseScript", "Silom", "Sinhala MN", "Sinhala Sangam MN", "Sistem Fontu", "Skia", "Snell Roundhand", "Songti SC", "Songti TC", "Sukhumvit Set", "Superclarendon", "Symbol", "Systeemlettertype", "System Font", "Systemschrift", "Systemskrift", "Systemtypsnitt", "Syst\u00e9mov\u00e9 p\u00edsmo", "Tahoma", "Tamil MN", "Tamil Sangam MN", "Telugu MN", "Telugu Sangam MN", "Thonburi", "Times", "Times New Roman", "Tipo de letra del sistema", "Tipo de letra do sistema", "Tipus de lletra del sistema", "Trattatello", "Trebuchet MS", "Verdana", "Waseem", "Webdings", "Wingdings", "Wingdings 2", "Wingdings 3", "Zapf Dingbats", "Zapfino", "\u0393\u03c1\u03b1\u03bc\u03bc\u03b1\u03c4\u03bf\u03c3\u03b5\u03b9\u03c1\u03ac \u03c3\u03c5\u03c3\u03c4\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2", "\u0421\u0438\u0441\u0442\u0435\u043c\u043d\u0438\u0439 \u0448\u0440\u0438\u0444\u0442", "\u0421\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0439 \u0448\u0440\u0438\u0444\u0442", "\u05d2\u05d5\u05e4\u05df \u05de\u05e2\u05e8\u05db\u05ea", "\u0627\u0644\u0628\u064a\u0627\u0646", "\u0627\u0644\u062a\u0627\u0631\u064a\u062e", "\u0627\u0644\u0646\u064a\u0644", "\u0628\u063a\u062f\u0627\u062f", "\u0628\u064a\u0631\u0648\u062a", "\u062c\u064a\u0632\u0629", "\u062e\u0637 \u0627\u0644\u0646\u0638\u0627\u0645", "\u062f\u0645\u0634\u0642", "\u062f\u064a\u0648\u0627\u0646 \u062b\u0644\u062b", "\u062f\u064a\u0648\u0627\u0646 \u0643\u0648\u0641\u064a", "\u0635\u0646\u0639\u0627\u0621", "\u0641\u0627\u0631\u0633\u064a", "\u0641\u0631\u062d", "\u0643\u0648\u0641\u064a", "\u0645\u0646\u0649", "\u0645\u0650\u0635\u062d\u0641\u064a", "\u0645\u0650\u0635\u062d\u0641\u064a \u0630\u0647\u0628\u064a", "\u0646\u062f\u064a\u0645", "\u0646\u0633\u062e", "\u0648\u0633\u064a\u0645", "\u0906\u0908\u0970\u091f\u0940\u0970\u090f\u092b\u093c\u0970 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940", "\u0906\u0908\u0970\u091f\u0940\u0970\u090f\u092b\u093c\u0970 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u092e\u0930\u093e\u0920\u0940", "\u0915\u094b\u0939\u093f\u0928\u0942\u0930 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940", "\u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u090f\u092e\u0970\u091f\u0940\u0970", "\u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u0938\u0902\u0917\u092e \u090f\u092e\u0970\u090f\u0928\u0970", "\u0936\u094d\u0930\u0940 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u096d\u0967\u096a", "\u0e41\u0e1a\u0e1a\u0e2d\u0e31\u0e01\u0e29\u0e23\u0e23\u0e30\u0e1a\u0e1a", "\u2e41\u7175\u6120\u82a9\u82c8", "\u30b7\u30b9\u30c6\u30e0\u30d5\u30a9\u30f3\u30c8", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 Pro", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 Pro W4", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 ProN", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 ProN W4", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d Pro", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d Pro W3", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d Pro W6", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d ProN", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d ProN W3", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d ProN W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Pro", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Pro W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Pro W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 ProN", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 ProN W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 ProN W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Std", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Std W8", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 StdN", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 StdN W8", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 \u7c21\u4f53\u4e2d\u6587", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 \u7c21\u4f53\u4e2d\u6587 W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 \u7c21\u4f53\u4e2d\u6587 W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W0", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W1", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W2", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W4", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W5", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W7", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W8", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W9", "\u51ac\u9752\u9ed1\u4f53\u7b80\u4f53\u4e2d\u6587", "\u51ac\u9752\u9ed1\u4f53\u7b80\u4f53\u4e2d\u6587 W3", "\u51ac\u9752\u9ed1\u4f53\u7b80\u4f53\u4e2d\u6587 W6", "\u51ac\u9752\u9ed1\u9ad4\u7c21\u9ad4\u4e2d\u6587", "\u51ac\u9752\u9ed1\u9ad4\u7c21\u9ad4\u4e2d\u6587 W3", "\u51ac\u9752\u9ed1\u9ad4\u7c21\u9ad4\u4e2d\u6587 W6", "\u5b8b\u4f53-\u7b80", "\u5b8b\u4f53-\u7e41", "\u5b8b\u9ad4-\u7c21", "\u5b8b\u9ad4-\u7e41", "\u7cfb\u7d71\u5b57\u9ad4", "\u7cfb\u7edf\u5b57\u4f53", "\u82f9\u65b9-\u6e2f", "\u82f9\u65b9-\u7b80", "\u82f9\u65b9-\u7e41", "\u8371\u8389\u834d\u836d\u8a70\u8353\u2050\u726f", "\u8371\u8389\u834d\u836d\u8a70\u8353\u2053\u7464", "\u8371\u8389\u834d\u836d\u8a70\u8353\u8356\u8362\u834e", "\u8371\u8389\u834d\u836d\u8adb\u8353\u2050\u726f", "\u8371\u8389\u834d\u836d\u96be\u92a9\u2050\u726f", "\u860b\u65b9-\u6e2f", "\u860b\u65b9-\u7c21", "\u860b\u65b9-\u7e41", "\u9ed1\u4f53-\u7b80", "\u9ed1\u4f53-\u7e41", "\u9ed1\u9ad4-\u7c21", "\u9ed1\u9ad4-\u7e41", "\u9ed2\u4f53-\u7c21", "\u9ed2\u4f53-\u7e41", "\uc2dc\uc2a4\ud15c \uc11c\uccb4", - }, - - "linux": { - "Arimo", "Cousine", "Noto Naskh Arabic", "Noto Sans Adlam", "Noto Sans Armenian", "Noto Sans Balinese", "Noto Sans Bamum", "Noto Sans Bassa Vah", "Noto Sans Batak", "Noto Sans Bengali", "Noto Sans Buginese", "Noto Sans Buhid", "Noto Sans Canadian Aboriginal", "Noto Sans Chakma", "Noto Sans Cham", "Noto Sans Cherokee", "Noto Sans Coptic", "Noto Sans Deseret", "Noto Sans Devanagari", "Noto Sans Elbasan", "Noto Sans Ethiopic", "Noto Sans Georgian", "Noto Sans Grantha", "Noto Sans Gujarati", "Noto Sans Gunjala Gondi", "Noto Sans Gurmukhi", "Noto Sans Hanifi Rohingya", "Noto Sans Hanunoo", "Noto Sans Hebrew", "Noto Sans JP", "Noto Sans Javanese", "Noto Sans KR", "Noto Sans Kannada", "Noto Sans Kayah Li", "Noto Sans Khmer", "Noto Sans Khojki", "Noto Sans Khudawadi", "Noto Sans Lao", "Noto Sans Lepcha", "Noto Sans Limbu", "Noto Sans Lisu", "Noto Sans Mahajani", "Noto Sans Malayalam", "Noto Sans Mandaic", "Noto Sans Masaram Gondi", "Noto Sans Medefaidrin", "Noto Sans Meetei Mayek", "Noto Sans Mende Kikakui", "Noto Sans Miao", "Noto Sans Modi", "Noto Sans Mongolian", "Noto Sans Mro", "Noto Sans Multani", "Noto Sans Myanmar", "Noto Sans NKo", "Noto Sans New Tai Lue", "Noto Sans Newa", "Noto Sans Ol Chiki", "Noto Sans Oriya", "Noto Sans Osage", "Noto Sans Osmanya", "Noto Sans Pahawh Hmong", "Noto Sans Pau Cin Hau", "Noto Sans Rejang", "Noto Sans Runic", "Noto Sans SC", "Noto Sans Samaritan", "Noto Sans Saurashtra", "Noto Sans Sharada", "Noto Sans Shavian", "Noto Sans Sinhala", "Noto Sans Sora Sompeng", "Noto Sans Soyombo", "Noto Sans Sundanese", "Noto Sans Syloti Nagri", "Noto Sans Symbols", "Noto Sans Symbols 2", "Noto Sans Syriac", "Noto Sans TC", "Noto Sans Tagalog", "Noto Sans Tagbanwa", "Noto Sans Tai Le", "Noto Sans Tai Tham", "Noto Sans Tai Viet", "Noto Sans Takri", "Noto Sans Tamil", "Noto Sans Telugu", "Noto Sans Thaana", "Noto Sans Thai", "Noto Sans Tifinagh", "Noto Sans Tifinagh APT", "Noto Sans Tifinagh Adrar", "Noto Sans Tifinagh Agraw Imazighen", "Noto Sans Tifinagh Ahaggar", "Noto Sans Tifinagh Air", "Noto Sans Tifinagh Azawagh", "Noto Sans Tifinagh Ghat", "Noto Sans Tifinagh Hawad", "Noto Sans Tifinagh Rhissa Ixa", "Noto Sans Tifinagh SIL", "Noto Sans Tifinagh Tawellemmet", "Noto Sans Tirhuta", "Noto Sans Vai", "Noto Sans Wancho", "Noto Sans Warang Citi", "Noto Sans Yi", "Noto Sans Zanabazar Square", "Noto Serif Armenian", "Noto Serif Balinese", "Noto Serif Bengali", "Noto Serif Devanagari", "Noto Serif Dogra", "Noto Serif Ethiopic", "Noto Serif Georgian", "Noto Serif Grantha", "Noto Serif Gujarati", "Noto Serif Gurmukhi", "Noto Serif Hebrew", "Noto Serif Kannada", "Noto Serif Khmer", "Noto Serif Khojki", "Noto Serif Lao", "Noto Serif Malayalam", "Noto Serif Myanmar", "Noto Serif NP Hmong", "Noto Serif Sinhala", "Noto Serif Tamil", "Noto Serif Telugu", "Noto Serif Thai", "Noto Serif Tibetan", "Noto Serif Yezidi", "STIX Two Math", "Tinos", "Twemoji Mozilla", - }, -} diff --git a/legacy/launcher/exec.go b/legacy/launcher/exec.go deleted file mode 100644 index 9fde13cac..000000000 --- a/legacy/launcher/exec.go +++ /dev/null @@ -1,166 +0,0 @@ -package main - -import ( - "bufio" - "fmt" - "io" - "os" - "os/exec" - "os/signal" - "path/filepath" - "runtime" - "syscall" -) - -func getExecutableName() string { - // Get the executable name based on the OS - switch normalizeOS(runtime.GOOS) { - case "linux": - return getPath("camoufox-bin") - case "macos": - return getPath("Camoufox.app") - case "windows": - return getPath("camoufox.exe") - default: - // This should never be reached due to the check in normalizeOS - return "" - } -} - -func setExecutablePermissions(execPath string) error { - // Set executable permissions if needed - switch normalizeOS(runtime.GOOS) { - case "macos": - return filepath.Walk(execPath, func(path string, info os.FileInfo, err error) error { - if err != nil { - return err - } - return maybeSetPermission(path, 0755) - }) - case "linux": - return maybeSetPermission(execPath, 0755) - } - return nil -} - -func maybeSetPermission(path string, mode os.FileMode) error { - info, err := os.Stat(path) - if err != nil { - return err - } - - currentMode := info.Mode().Perm() - if currentMode != mode { - return os.Chmod(path, mode) - } - return nil -} - -func filterOutput(r io.Reader, w io.Writer) { - scanner := bufio.NewScanner(r) - for scanner.Scan() { - line := scanner.Text() - if !ExclusionRegex.MatchString(line) { - fmt.Fprintln(w, line) - } - } -} - -// Run Camoufox -func runCamoufox(execName string, args []string, addonsList []string, stderrPath string) { - // If addons are specified, get the debug port - var debugPortInt int - if len(addonsList) > 0 { - debugPortInt = getDebugPort(&args) - } - - // For macOS, use "open" command to launch the app - if normalizeOS(runtime.GOOS) == "macos" { - args = append([]string{"-a", execName}, args...) - execName = "open" - } - - // Print args - cmd := exec.Command(execName, args...) - - setProcessGroupID(cmd) - - stdout, err := cmd.StdoutPipe() - if err != nil { - fmt.Printf("Error creating stdout pipe: %v\n", err) - os.Exit(1) - } - stderr, err := cmd.StderrPipe() - if err != nil { - fmt.Printf("Error creating stderr pipe: %v\n", err) - os.Exit(1) - } - - // Set up signal handling - sigChan := make(chan os.Signal, 1) - signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM) - - if err := cmd.Start(); err != nil { - fmt.Printf("Error starting %s: %v\n", execName, err) - os.Exit(1) - } - - if len(addonsList) > 0 { - go tryLoadAddons(debugPortInt, addonsList) - } - - // Channel to signal when the subprocess has finished - subprocessDone := make(chan struct{}) - - // Start a goroutine to handle signals - go func() { - select { - case <-sigChan: - killProcessGroup(cmd) - case <-subprocessDone: - // Subprocess has finished, exit the Go process - os.Exit(0) - } - }() - - done := make(chan bool) - - go func() { - filterOutput(stdout, os.Stdout) - done <- true - }() - go func() { - // If stderrPath is not empty, write to the file - fmt.Printf("Setting stderr to file: %s\n", stderrPath) - if stderrPath != "" { - file, err := os.Create(stderrPath) - if err != nil { - fmt.Printf("Error creating stderr file: %v\n", err) - os.Exit(1) - } - defer file.Close() - filterOutput(stderr, file) - } - filterOutput(stderr, os.Stderr) - }() - - <-done - <-done - - // Wait for the command to finish - if err := cmd.Wait(); err != nil { - if exitErr, ok := err.(*exec.ExitError); ok { - // If the subprocess exited with an error, use its exit code - os.Exit(exitErr.ExitCode()) - } else { - fmt.Printf("Error running %s: %v\n", execName, err) - os.Exit(1) - } - } - - // Signal that the subprocess has finished - close(subprocessDone) - - // Wait here to allow the signal handling goroutine to exit the process - select {} -} diff --git a/legacy/launcher/go.mod b/legacy/launcher/go.mod deleted file mode 100644 index b8109c887..000000000 --- a/legacy/launcher/go.mod +++ /dev/null @@ -1,7 +0,0 @@ -module launch - -go 1.23.0 - -require github.com/mileusna/useragent v1.3.4 - -require github.com/goccy/go-json v0.10.3 diff --git a/legacy/launcher/go.sum b/legacy/launcher/go.sum deleted file mode 100644 index c82eb6ffc..000000000 --- a/legacy/launcher/go.sum +++ /dev/null @@ -1,4 +0,0 @@ -github.com/goccy/go-json v0.10.3 h1:KZ5WoDbxAIgm2HNbYckL0se1fHD6rz5j4ywS6ebzDqA= -github.com/goccy/go-json v0.10.3/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= -github.com/mileusna/useragent v1.3.4 h1:MiuRRuvGjEie1+yZHO88UBYg8YBC/ddF6T7F56i3PCk= -github.com/mileusna/useragent v1.3.4/go.mod h1:3d8TOmwL/5I8pJjyVDteHtgDGcefrFUX4ccGOMKNYYc= diff --git a/legacy/launcher/load-addons.go b/legacy/launcher/load-addons.go deleted file mode 100644 index c8450e1b2..000000000 --- a/legacy/launcher/load-addons.go +++ /dev/null @@ -1,157 +0,0 @@ -package main - -import ( - "bufio" - "encoding/json" - "fmt" - "net" - "os" - "strconv" - "strings" - "time" -) - -// Gets the debug port from the args, or creates a new one if not provided -func getDebugPort(args *[]string) int { - debugPort := parseArgs("-start-debugger-server", "", args, false) - - var debugPortInt int - var err error - if debugPort == "" { - // Create new debugger server port - debugPortInt = getOpenPort() - // Add -start-debugger-server {debugPort} to args - *args = append(*args, "-start-debugger-server", strconv.Itoa(debugPortInt)) - } else { - debugPortInt, err = strconv.Atoi(debugPort) - if err != nil { - fmt.Printf("Error parsing debug port. Must be an integer: %v\n", err) - os.Exit(1) - } - } - return debugPortInt -} - -// Confirm paths are valid -func confirmPaths(paths []string) { - for _, path := range paths { - if _, err := os.Stat(path); err != nil { - fmt.Printf("Error: %s is not a valid addon path.\n", path) - os.Exit(1) - } - } -} - -// Generate an open port -func getOpenPort() int { - ln, err := net.Listen("tcp", ":0") // listen on a random port - if err != nil { - return 0 - } - defer ln.Close() - - addr := ln.Addr().(*net.TCPAddr) // type assert to *net.TCPAddr to get the Port - return addr.Port -} - -// Waits for the server to start, then loads the addons -func tryLoadAddons(debugPortInt int, addonsList []string) { - // Wait for the server to be open - for { - conn, err := net.Dial("tcp", fmt.Sprintf("localhost:%d", debugPortInt)) - if err == nil { - conn.Close() - break - } - time.Sleep(10 * time.Millisecond) - } - - // Load addons - for _, addon := range addonsList { - go loadFirefoxAddon(debugPortInt, addon) - } -} - -// Firefox addon loader -// Ported from this Nodejs implementation: -// https://github.com/microsoft/playwright/issues/7297#issuecomment-1211763085 -func loadFirefoxAddon(port int, addonPath string) bool { - conn, err := net.Dial("tcp", fmt.Sprintf("%s:%d", "localhost", port)) - if err != nil { - return false - } - defer conn.Close() - - success := false - - send := func(data map[string]string) error { - jsonData, err := json.Marshal(data) - if err != nil { - return err - } - _, err = fmt.Fprintf(conn, "%d:%s", len(jsonData), jsonData) - return err - } - - err = send(map[string]string{ - "to": "root", - "type": "getRoot", - }) - if err != nil { - return false - } - - onMessage := func(message map[string]interface{}) bool { - if addonsActor, ok := message["addonsActor"].(string); ok { - err := send(map[string]string{ - "to": addonsActor, - "type": "installTemporaryAddon", - "addonPath": addonPath, - }) - if err != nil { - return true - } - } - - if _, ok := message["addon"]; ok { - success = true - return true - } - - if _, ok := message["error"]; ok { - return true - } - - return false - } - - reader := bufio.NewReader(conn) - for { - lengthStr, err := reader.ReadString(':') - if err != nil { - break - } - length, err := strconv.Atoi(strings.TrimSuffix(lengthStr, ":")) - if err != nil { - break - } - - jsonData := make([]byte, length) - _, err = reader.Read(jsonData) - if err != nil { - break - } - - var message map[string]interface{} - err = json.Unmarshal(jsonData, &message) - if err != nil { - break - } - - if onMessage(message) { - break - } - } - - return success -} diff --git a/legacy/launcher/main.go b/legacy/launcher/main.go deleted file mode 100644 index 4a4c76fe5..000000000 --- a/legacy/launcher/main.go +++ /dev/null @@ -1,219 +0,0 @@ -package main - -import ( - "fmt" - "os" - "path/filepath" - "runtime" - "strings" - "unicode/utf8" - - json "github.com/goccy/go-json" - "github.com/mileusna/useragent" -) - -func main() { - args := os.Args[1:] - - configPath := parseArgs("--config", "{}", &args, true) - addons := parseArgs("--addons", "[]", &args, true) - excludeAddons := parseArgs("--exclude-addons", "[]", &args, true) - stderrPath := parseArgs("--stderr", "", &args, true) - - //*** PARSE CONFIG ***// - - // Read and parse the config file - var configMap map[string]interface{} - parseJson(configPath, &configMap) - validateConfig(configMap) - - // Add "debug: True" to the config - if stderrPath != "" { - configMap["debug"] = true - } - - //*** PARSE ADDONS ***// - - // If addons are passed, handle them - var addonsList []string - parseJson(addons, &addonsList) - - // Confirm addon paths are valid - confirmPaths(addonsList) - - // Add the default addons, excluding the ones specified in --exclude-addons - var excludeAddonsList []string - parseJson(excludeAddons, &excludeAddonsList) - - addDefaultAddons(excludeAddonsList, &addonsList) - - //*** FONTS ***// - - // Determine the target OS - userAgentOS := determineUserAgentOS(configMap) - // Add OS specific fonts - updateFonts(configMap, userAgentOS) - - //*** LAUNCH ***// - - setEnvironmentVariables(configMap, userAgentOS) - - // Run the Camoufox executable - execName := getExecutableName() - if err := setExecutablePermissions(execName); err != nil { - fmt.Printf("Error setting executable permissions: %v\n", err) - os.Exit(1) - } - runCamoufox(execName, args, addonsList, stderrPath) -} - -// Returns the absolute path relative to the launcher -func getPath(path string) string { - execPath, err := os.Executable() - if err != nil { - fmt.Printf("Error getting executable path: %v\n", err) - os.Exit(1) - } - execDir := filepath.Dir(execPath) - - addonPath := filepath.Join(execDir, path) - return addonPath -} - -// Parses & removes an argument from the args list -func parseArgs(param string, defaultValue string, args *[]string, removeFromArgs bool) string { - for i := 0; i < len(*args); i++ { - if (*args)[i] != param { - continue - } - if i+1 < len(*args) { - value := (*args)[i+1] - if removeFromArgs { - *args = append((*args)[:i], (*args)[i+2:]...) - } - return value - } - fmt.Printf("Error: %s flag requires a value\n", param) - os.Exit(1) - } - return defaultValue -} - -// fileExists checks if a file exists -func fileExists(path string) bool { - _, err := os.Stat(path) - return err == nil -} - -// Parses a JSON string or file into a map -func parseJson(argv string, target interface{}) { - // Unmarshal the config input into a map - var data []byte - var err error - - // Check if the input is a file path or inline JSON - if fileExists(argv) { - data, err = os.ReadFile(argv) - if err != nil { - fmt.Printf("Error reading JSON file: %v\n", err) - os.Exit(1) - } - } else { - // Assume it's inline JSON - data = []byte(argv) - } - - if err := json.Unmarshal(data, target); err != nil { - fmt.Printf("Invalid JSON: %v\n", err) - os.Exit(1) - } -} - -// Determines the target OS from the user agent string if provided -func determineUserAgentOS(configMap map[string]interface{}) string { - // Determine the OS from the user agent string if provided - defaultOS := normalizeOS(runtime.GOOS) - if ua, ok := configMap["navigator.userAgent"].(string); ok { - parsedUA := useragent.Parse(ua) - if parsedUA.OS != "" { - return normalizeOS(parsedUA.OS) - } - } - return defaultOS -} - -// Get the OS name as {macos, windows, linux} -func normalizeOS(osName string) string { - osName = strings.ToLower(osName) - switch { - case osName == "darwin" || strings.Contains(osName, "mac"): - return "macos" - case strings.Contains(osName, "win"): - return "windows" - default: - return "linux" - } -} - -// Add fonts associated with the OS to the config map -func updateFonts(configMap map[string]interface{}, userAgentOS string) { - fonts, ok := configMap["fonts"].([]interface{}) - if !ok { - fonts = []interface{}{} - } - existingFonts := make(map[string]bool) - for _, font := range fonts { - if f, ok := font.(string); ok { - existingFonts[f] = true - } - } - for _, font := range FontsByOS[userAgentOS] { - if !existingFonts[font] { - fonts = append(fonts, font) - } - } - configMap["fonts"] = fonts -} - -// Update the config map with the fonts and environment variables -func setEnvironmentVariables(configMap map[string]interface{}, userAgentOS string) { - updatedConfigData, err := json.Marshal(configMap) - if err != nil { - fmt.Printf("Error updating config: %v\n", err) - os.Exit(1) - } - - // Validate utf8 - if !utf8.Valid(updatedConfigData) { - fmt.Println("Config is not valid UTF-8") - os.Exit(1) - } - - // Split the config into chunks of 2047 characters if the OS is Windows, - // otherwise split into 32767 characters - var chunkSize int - if normalizeOS(runtime.GOOS) == "windows" { - chunkSize = 2047 - } else { - chunkSize = 32767 - } - - configStr := string(updatedConfigData) - for i := 0; i < len(configStr); i += chunkSize { - end := i + chunkSize - if end > len(configStr) { - end = len(configStr) - } - chunk := configStr[i:end] - envName := fmt.Sprintf("CAMOU_CONFIG_%d", (i/chunkSize)+1) - if err := os.Setenv(envName, chunk); err != nil { - fmt.Printf("Error setting %s: %v\n", envName, err) - os.Exit(1) - } - } - - if normalizeOS(runtime.GOOS) == "linux" { - fontconfigPath := getPath(filepath.Join("fontconfig", userAgentOS)) - os.Setenv("FONTCONFIG_PATH", fontconfigPath) - } -} diff --git a/legacy/launcher/procgroup-unix.go b/legacy/launcher/procgroup-unix.go deleted file mode 100644 index d85a0fcbc..000000000 --- a/legacy/launcher/procgroup-unix.go +++ /dev/null @@ -1,17 +0,0 @@ -//go:build !windows -// +build !windows - -package main - -import ( - "os/exec" - "syscall" -) - -func setProcessGroupID(cmd *exec.Cmd) { - cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} -} - -func killProcessGroup(cmd *exec.Cmd) { - syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) -} diff --git a/legacy/launcher/procgroup-win.go b/legacy/launcher/procgroup-win.go deleted file mode 100644 index 2bf68d32e..000000000 --- a/legacy/launcher/procgroup-win.go +++ /dev/null @@ -1,16 +0,0 @@ -//go:build windows -// +build windows - -package main - -import ( - "os/exec" -) - -func setProcessGroupID(cmd *exec.Cmd) { - // Windows doesn't support process groups in the same way -} - -func killProcessGroup(cmd *exec.Cmd) { - cmd.Process.Kill() -} diff --git a/legacy/launcher/validate.go b/legacy/launcher/validate.go deleted file mode 100644 index 44dc7dbe1..000000000 --- a/legacy/launcher/validate.go +++ /dev/null @@ -1,75 +0,0 @@ -package main - -import ( - "fmt" - "math" - "os" - "reflect" - "runtime" -) - -type Property struct { - Property string `json:"property"` - Type string `json:"type"` -} - -func validateConfig(configMap map[string]interface{}) { - properties := loadProperties() - - // Create a map for quick lookup of property types - propertyTypes := make(map[string]string) - for _, prop := range properties { - propertyTypes[prop.Property] = prop.Type - } - - for key, value := range configMap { - expectedType, exists := propertyTypes[key] - if !exists { - fmt.Printf("Warning: Unknown property %s in config\n", key) - continue - } - - if !validateType(value, expectedType) { - fmt.Printf("Invalid type for property %s. Expected %s, got %T\n", key, expectedType, value) - os.Exit(1) - } - } -} - -func loadProperties() []Property { - // Get the path to the properties.json file - var propertiesPath string - if normalizeOS(runtime.GOOS) == "macos" { - propertiesPath = getPath("Camoufox.app/Contents/Resources/properties.json") - } else { - propertiesPath = getPath("properties.json") - } - var properties []Property - // Parse the JSON file - parseJson(propertiesPath, &properties) - return properties -} - -func validateType(value interface{}, expectedType string) bool { - switch expectedType { - case "str": - _, ok := value.(string) - return ok - case "int": - v, ok := value.(float64) - return ok && v == math.Trunc(v) - case "uint": - v, ok := value.(float64) - return ok && v == math.Trunc(v) && v >= 0 - case "double": - _, ok := value.(float64) - return ok - case "bool": - _, ok := value.(bool) - return ok - case "array": - return reflect.TypeOf(value).Kind() == reflect.Slice - default: - return false - } -} diff --git a/legacy/launcher/xpi-dl.go b/legacy/launcher/xpi-dl.go deleted file mode 100644 index 66d74b957..000000000 --- a/legacy/launcher/xpi-dl.go +++ /dev/null @@ -1,146 +0,0 @@ -package main - -import ( - "archive/zip" - "fmt" - "io" - "net/http" - "os" - "path/filepath" -) - -// Downloads and extracts the default addons -func addDefaultAddons(excludeList []string, addonsList *[]string) { - // Build a map from DefaultAddons, excluding keys found in excludeAddonsList - addonsMap := make(map[string]string) - for name, url := range DefaultAddons { - if len(excludeList) == 0 || !contains(excludeList, name) { - addonsMap[name] = url - } - } - - // Download if not already downloaded - maybeDownloadAddons(addonsMap, addonsList) -} - -// Downloads and extracts the addon -func downloadAndExtract(url, extractPath string) error { - // Create a temporary file to store the downloaded zip - tempFile, err := os.CreateTemp("", "camoufox-addon-*.zip") - if err != nil { - return fmt.Errorf("failed to create temp file: %w", err) - } - defer os.Remove(tempFile.Name()) // Clean up the temp file when done - - // Download the zip file - resp, err := http.Get(url) - if err != nil { - return fmt.Errorf("failed to download addon: %w", err) - } - defer resp.Body.Close() - - // Write the body to the temp file - _, err = io.Copy(tempFile, resp.Body) - if err != nil { - return fmt.Errorf("failed to write addon to temp file: %w", err) - } - - // Close the file before unzipping - tempFile.Close() - - // Extract the zip file - err = unzip(tempFile.Name(), extractPath) - if err != nil { - return fmt.Errorf("failed to extract addon: %w", err) - } - - return nil -} - -// Extracts the zip file -func unzip(src, dest string) error { - r, err := zip.OpenReader(src) - if err != nil { - return err - } - defer r.Close() - - for _, f := range r.File { - fpath := filepath.Join(dest, f.Name) - - if f.FileInfo().IsDir() { - os.MkdirAll(fpath, os.ModePerm) - continue - } - - if err = os.MkdirAll(filepath.Dir(fpath), os.ModePerm); err != nil { - return err - } - - outFile, err := os.OpenFile(fpath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, f.Mode()) - if err != nil { - return err - } - - rc, err := f.Open() - if err != nil { - outFile.Close() - return err - } - - _, err = io.Copy(outFile, rc) - outFile.Close() - rc.Close() - - if err != nil { - return err - } - } - return nil -} - -// Checks if a slice contains a string -func contains(slice []string, item string) bool { - for _, s := range slice { - if s == item { - return true - } - } - return false -} - -// Returns the absolute path to the target addon location -func getAddonPath(addonName string) string { - return getPath(filepath.Join("addons", addonName)) -} - -// Downloads and extracts the addons -func maybeDownloadAddons(addons map[string]string, addonsList *[]string) { - for addonName, url := range addons { - // Get the addon path - addonPath := getAddonPath(addonName) - - // Check if the addon is already extracted - if _, err := os.Stat(addonPath); !os.IsNotExist(err) { - // Add the existing addon path to addonsList - *addonsList = append(*addonsList, addonPath) - continue - } - - // Addon doesn't exist, create directory and download - err := os.MkdirAll(addonPath, 0755) - if err != nil { - fmt.Printf("Failed to create directory for %s: %v\n", addonName, err) - continue - } - - err = downloadAndExtract(url, addonPath) - if err != nil { - fmt.Printf("Failed to download and extract %s: %v\n", addonName, err) - } else { - fmt.Printf("Successfully downloaded and extracted %s\n", addonName) - // Add the new addon directory path to addonsList - *addonsList = append(*addonsList, addonPath) - } - } -} diff --git a/legacy/scripts/generate-locales.sh b/legacy/scripts/generate-locales.sh deleted file mode 100644 index e5fa3bb66..000000000 --- a/legacy/scripts/generate-locales.sh +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/bash - -if [ ! -f browser/locales/shipped-locales ]; then - echo "ERROR: Run this script from the root of the Camoufox source code" - exit 1 -fi - -rm -rf browser/locales/l10n -mkdir browser/locales/l10n - -N=8 -for i in $(seq $N); do echo; done -total=$(wc -l < browser/locales/shipped-locales) - -echo_status() { - printf "\033[$((($N - $n) + 1))A$@ %40s\r\033[$((($N - $n) + 1))B" -} - -generate_locale() { - if echo " en-US ca ja " | grep -q " $1 "; then - echo_status "Skipping locale \"$1\"" - sleep 1 - echo_status - return - fi - echo_status "Downloading locale \"$1\"" - wget -q -O browser/locales/l10n/$1.zip https://hg.mozilla.org/l10n-central/$1/archive/tip.zip - echo_status "Extracting locale \"$1\"" - 7z x -y -obrowser/locales/l10n browser/locales/l10n/$1.zip > /dev/null - mv browser/locales/l10n/$1-*/ browser/locales/l10n/$1/ - rm -f browser/locales/l10n/$1.zip - echo_status "Generating locale \"$1\"" - mv browser/locales/l10n/$1/browser/branding/official browser/locales/l10n/$1/browser/branding/camoufox - find browser/locales/l10n/$1 -type f -exec sed -i -e 's/Mozilla Firefox/Camoufox/g' {} \; - find browser/locales/l10n/$1 -type f -exec sed -i -e 's/Mozilla/Camoufox/g' {} \; - find browser/locales/l10n/$1 -type f -exec sed -i -e 's/Firefox/Camoufox/g' {} \; - echo_status "Done" - sleep 0.3 - echo_status -} - -while read in; do - ((n=n%N)); ((n++==0)) && wait - generate_locale $in & -done < browser/locales/shipped-locales - -wait - -printf "\033[$(($N))A\rGenerated $total locales %-40s\n" \ No newline at end of file diff --git a/native-tests/_churn.py b/native-tests/_churn.py index 3599f43a0..feeb0e00d 100644 --- a/native-tests/_churn.py +++ b/native-tests/_churn.py @@ -6,7 +6,7 @@ memory scales with the count. The theory this is built to test is that Camoufox adds per-something state that -stock Firefox does not have -- an isolated world, a canvas noise seed, a font +stock Firefox does not have -- an isolated world, an audio noise seed, a font list -- and that something churning fast enough (an ad stack creating and destroying iframes, compiling scripts, drawing to canvases) accumulates it. diff --git a/native-tests/test_contexts_vs_browsers.py b/native-tests/test_contexts_vs_browsers.py index 63ec74c00..a374993d7 100644 --- a/native-tests/test_contexts_vs_browsers.py +++ b/native-tests/test_contexts_vs_browsers.py @@ -32,6 +32,7 @@ from __future__ import annotations import asyncio +import json from pathlib import Path import pytest @@ -219,20 +220,144 @@ async def test_closing_one_context_does_not_disturb_another(binary): ) +# The whole fingerprint a site computes, run by the page itself: Playwright's +# evaluate runs in Camoufox's isolated world, which may not read audio sample +# data, so the page writes its result into the DOM for the test to read. +FONT_CANDIDATES = [ + "Arial", "Arial Black", "Calibri", "Cambria", "Candara", "Consolas", "Constantia", + "Corbel", "Courier New", "Ebrima", "Franklin Gothic Medium", "Gabriola", "Gadugi", + "Georgia", "Impact", "Ink Free", "Javanese Text", "Leelawadee UI", "Lucida Console", + "Lucida Sans Unicode", "Malgun Gothic", "Marlett", "Microsoft Himalaya", + "Microsoft JhengHei", "Microsoft New Tai Lue", "Microsoft PhagsPa", "Microsoft Sans Serif", + "Microsoft Tai Le", "Microsoft YaHei", "Microsoft Yi Baiti", "MingLiU-ExtB", + "Mongolian Baiti", "MS Gothic", "MV Boli", "Myanmar Text", "Nirmala UI", + "Palatino Linotype", "Segoe Print", "Segoe Script", "Segoe UI", "Segoe UI Emoji", + "SimSun", "Sitka Small", "Sylfaen", "Symbol", "Tahoma", "Times New Roman", + "Trebuchet MS", "Verdana", "Webdings", "Wingdings", "Yu Gothic", "Aptos", "Bahnschrift", + "HoloLens MDL2 Assets", "Cascadia Code", "Cascadia Mono", "Segoe Fluent Icons", + "American Typewriter", "Andale Mono", "Apple Chancery", "Apple Color Emoji", + "Apple SD Gothic Neo", "Avenir", "Avenir Next", "Baskerville", "Big Caslon", + "Chalkboard", "Chalkboard SE", "Charter", "Cochin", "Copperplate", "Didot", + "Futura", "Geneva", "Gill Sans", "Helvetica", "Helvetica Neue", "Herculanum", + "Hoefler Text", "Lucida Grande", "Marker Felt", "Menlo", "Monaco", "Noteworthy", + "Optima", "Papyrus", "PingFang SC", "Rockwell", "SF Pro", "Skia", "Snell Roundhand", + "Zapfino", "Arial Hebrew", "Hiragino Sans", "Kohinoor Devanagari", "Thonburi", + "DejaVu Sans", "DejaVu Serif", "DejaVu Sans Mono", "Liberation Sans", + "Liberation Serif", "Liberation Mono", "Noto Sans", "Noto Serif", "Noto Color Emoji", + "Noto Sans CJK SC", "Ubuntu", "Ubuntu Mono", "Cantarell", "FreeSans", "FreeSerif", + "Nimbus Sans", "Nimbus Roman", "C059", "P052", "URW Bookman", "Droid Sans Fallback", +] + +FINGERPRINT_PAGE = """""" + + +async def full_fingerprint(page) -> dict: + """Everything a fingerprinting script reads, computed in the page. + + Served from an https URL Playwright fulfils locally: mediaDevices and other + APIs exist only in a secure context, which about:blank content is not. + """ + html = FINGERPRINT_PAGE.replace("FONTS", json.dumps(FONT_CANDIDATES)) + await page.route("https://fingerprint.camoufox.test/", + lambda route: route.fulfill(content_type="text/html", body=html)) + await page.goto("https://fingerprint.camoufox.test/") + for _ in range(60): + result = await page.evaluate("document.documentElement.dataset.fingerprint || null") + if result: + fingerprint = json.loads(result) + assert "error" not in fingerprint, fingerprint["error"] + return fingerprint + await asyncio.sleep(0.5) + raise AssertionError("the fingerprint page produced no result in 30s") + + async def test_two_browsers_get_different_fingerprints(binary): + """Two launches must not present the same device. + + Measured on the whole fingerprint a site computes. The coarse values alone + can legitimately match -- two real Macs share a UA, a 2560x1440 screen and + 8 cores, and CI pins timezone and language -- so a check on those alone + failed whenever two draws landed on a common machine. Fonts, voices, the + GPU and the per-identity audio seed together cannot. + """ from camoufox.async_api import AsyncCamoufox async def one() -> dict: async with AsyncCamoufox(executable_path=str(binary), headless=True, i_know_what_im_doing=True) as browser: context, page = await open_page(browser) - values = await probe(page) + fingerprint = await full_fingerprint(page) await context.close() - return values + return fingerprint a, b = await asyncio.gather(one(), one()) - differing = [k for k in PROBES if a.get(k) != b.get(k)] + assert "audio" in a and "audio" in b, (a, b) + differing = sorted(k for k in a if a.get(k) != b.get(k)) assert differing, f"two separate browser launches produced an identical fingerprint: {a}" + # The audio noise is seeded per identity, so it must differ on its own: + # equal hashes would mean the seed stopped reaching the browser. + assert a["audio"] != b["audio"], f"both launches rendered audio hash {a['audio']}" async def test_a_context_survives_its_sibling_browser(binary): diff --git a/native-tests/test_tribal_rules.py b/native-tests/test_tribal_rules.py index 2066616a8..2d216a784 100644 --- a/native-tests/test_tribal_rules.py +++ b/native-tests/test_tribal_rules.py @@ -18,6 +18,7 @@ import importlib import inspect +import json import sys from pathlib import Path from typing import Any, Dict, List @@ -541,6 +542,47 @@ def test_a_sandbox_held_over_a_page_window_is_nuked_not_just_dropped(): ) +def test_instant_animations_are_an_opt_in(): + patch = (REPO_ROOT / "patches" / "no-css-animations.patch").read_text(encoding="utf-8") + assert 'MaskConfig::GetBool("instantAnimations")' in patch, explain("animations-run-on-stock-timing") + assert "disableInstantAnimations" not in patch, explain("animations-run-on-stock-timing") + + +def test_spoofed_voices_complete_without_a_config_switch(): + patch = (REPO_ROOT / "patches" / "voice-spoofing.patch").read_text(encoding="utf-8") + assert "fakeCompletion" not in patch and "DispatchError(0, 0)" not in patch, ( + explain("spoofed-voices-speak") + ) + + +def test_no_glyph_spacing_seed_anywhere(): + """No config key, no setter, no shaper hook.""" + declared = { + entry["property"] + for entry in json.loads((REPO_ROOT / "settings" / "properties.json").read_text()) + } + assert "fonts:spacing_seed" not in declared, explain("no-glyph-spacing-noise") + for source in [*sorted((REPO_ROOT / "patches").rglob("*.patch")), + REPO_ROOT / "pythonlib" / "camoufox" / "fingerprints.py"]: + assert "FontSpacingSeed" not in source.read_text(encoding="utf-8", errors="ignore"), ( + f"{source.relative_to(REPO_ROOT)} still carries the spacing seed" + + explain("no-glyph-spacing-noise") + ) + + +def test_no_canvas_seed_is_declared_or_sent(): + """Nothing in the browser reads a canvas seed, so neither launcher sends one.""" + declared = { + entry["property"] + for entry in json.loads((REPO_ROOT / "settings" / "properties.json").read_text()) + } + assert not {k for k in declared if k.startswith("canvas:")}, explain("canvas-is-not-noised") + fingerprints = REPO_ROOT / "pythonlib" / "camoufox" / "fingerprints.py" + assert "setCanvasSeed" not in fingerprints.read_text(encoding="utf-8"), ( + "fingerprints.py still calls setCanvasSeed" + explain("canvas-is-not-noised") + ) + + def test_the_canvas_check_hashes_pixels_rather_than_a_data_url_prefix(): """A truncated data URL is mostly PNG header, not image. diff --git a/patches/anti-font-fingerprinting.patch b/patches/anti-font-fingerprinting.patch index a276c9671..d7de58fa5 100644 --- a/patches/anti-font-fingerprinting.patch +++ b/patches/anti-font-fingerprinting.patch @@ -1,179 +1,3 @@ -diff --git a/dom/base/FontSpacingSeedManager.cpp b/dom/base/FontSpacingSeedManager.cpp -new file mode 100644 -index 0000000000..e07de3e753 ---- /dev/null -+++ b/dom/base/FontSpacingSeedManager.cpp -@@ -0,0 +1,91 @@ -+#include "FontSpacingSeedManager.h" -+#include "nsPrintfCString.h" -+#include "MaskConfig.hpp" -+#include "nsGlobalWindowInner.h" -+#include "xpcpublic.h" -+#include "mozilla/dom/BrowsingContext.h" -+#include "mozilla/dom/Document.h" -+#include "nsDocShell.h" -+#include "nsPIDOMWindow.h" -+ -+namespace mozilla { -+namespace dom { -+ -+/* static */ nsString -+FontSpacingSeedManager::KeyForUserContext(uint32_t userContextId) { -+ nsString key; -+ key.AppendLiteral(u"seed_"); -+ key.AppendInt(userContextId); -+ return key; -+} -+ -+/* static */ nsString -+FontSpacingSeedManager::DisabledKeyForUserContext(uint32_t userContextId) { -+ nsString key; -+ key.AppendLiteral(u"disabled_"); -+ key.AppendInt(userContextId); -+ return key; -+} -+ -+/* static */ void -+FontSpacingSeedManager::SetSeed(uint32_t userContextId, uint32_t seed) { -+ nsString key = KeyForUserContext(userContextId); -+ RoverfoxStorageManager::PutUint(key, seed); -+ -+ // Mark the function as disabled for this context after first use -+ DisableFunction(userContextId); -+} -+ -+/* static */ uint32_t -+FontSpacingSeedManager::GetSeed(uint32_t userContextId) { -+ nsString key = KeyForUserContext(userContextId); -+ uint32_t seed = 0; -+ if (RoverfoxStorageManager::GetUint(key, seed) && seed != 0) { -+ return seed; -+ } -+ // Fallback to CAMOU_CONFIG for Workers in separate processes -+ if (auto val = MaskConfig::GetUint32("fonts:spacing_seed")) { -+ return val.value(); -+ } -+ return 0; -+} -+ -+/* static */ bool -+FontSpacingSeedManager::HasSeed(uint32_t userContextId) { -+ nsString key = KeyForUserContext(userContextId); -+ uint32_t seed = 0; -+ return RoverfoxStorageManager::GetUint(key, seed); -+} -+ -+/* static */ bool -+FontSpacingSeedManager::IsFunctionDisabled(uint32_t userContextId) { -+ nsString key = DisabledKeyForUserContext(userContextId); -+ bool disabled = false; -+ return RoverfoxStorageManager::GetBool(key, disabled) && disabled; -+} -+ -+/* static */ void -+FontSpacingSeedManager::DisableFunction(uint32_t userContextId) { -+ nsString key = DisabledKeyForUserContext(userContextId); -+ RoverfoxStorageManager::PutBool(key, true); -+} -+ -+/* static */ bool -+FontSpacingSeedManager::IsFunctionEnabledForWebIDL(JSContext* aCx, JSObject* aObj) { -+ nsGlobalWindowInner* win = xpc::WindowOrNull(aObj); -+ if (!win) { -+ return false; -+ } -+ -+ uint32_t userContextId = 0; -+ if (BrowsingContext* bc = win->GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ -+ bool disabled = false; -+ RoverfoxStorageManager::GetBool(DisabledKeyForUserContext(userContextId), disabled); -+ return !disabled; -+} -+ -+} // namespace dom -+} // namespace mozilla -\ No newline at end of file -diff --git a/dom/base/FontSpacingSeedManager.h b/dom/base/FontSpacingSeedManager.h -new file mode 100644 -index 0000000000..63d43f3b86 ---- /dev/null -+++ b/dom/base/FontSpacingSeedManager.h -@@ -0,0 +1,71 @@ -+#ifndef mozilla_dom_FontSpacingSeedManager_h -+#define mozilla_dom_FontSpacingSeedManager_h -+ -+#include "nsString.h" -+#include "RoverfoxStorageManager.h" -+ -+namespace mozilla { -+namespace dom { -+ -+/** -+ * FontSpacingSeedManager manages font spacing seeds per user context. -+ * This enables privacy-preserving font fingerprinting by allowing deterministic -+ * font spacing modifications that are isolated by user context. -+ */ -+class FontSpacingSeedManager { -+public: -+ /** -+ * Set the font spacing seed for a given user context. -+ * @param userContextId The user context ID (0 for default context) -+ * @param seed The seed value to use for font spacing modifications -+ */ -+ static void SetSeed(uint32_t userContextId, uint32_t seed); -+ -+ /** -+ * Get the font spacing seed for a given user context. -+ * @param userContextId The user context ID -+ * @return The seed value, or 0 if no seed has been set -+ */ -+ static uint32_t GetSeed(uint32_t userContextId); -+ -+ /** -+ * Check if a seed has been set for a given user context. -+ * @param userContextId The user context ID -+ * @return true if a seed has been set, false otherwise -+ */ -+ static bool HasSeed(uint32_t userContextId); -+ -+ /** -+ * Check if the setFontSpacingSeed function has been used and should be disabled for a context. -+ * @param userContextId The user context ID -+ * @return true if the function has been used for this context and should not appear on new windows -+ */ -+ static bool IsFunctionDisabled(uint32_t userContextId); -+ -+ /** -+ * Mark the setFontSpacingSeed function as used/disabled for a context. -+ * @param userContextId The user context ID -+ */ -+ static void DisableFunction(uint32_t userContextId); -+ -+ /** -+ * WebIDL-compatible function to check if setFontSpacingSeed should be enabled. -+ * This extracts the user context from the window and checks if disabled. -+ * @param aCx JavaScript context -+ * @param aObj JavaScript object (window) -+ * @return true if function should be available, false otherwise -+ */ -+ static bool IsFunctionEnabledForWebIDL(JSContext* aCx, JSObject* aObj); -+ -+private: -+ // Helper to convert userContextId to string key -+ static nsString KeyForUserContext(uint32_t userContextId); -+ -+ // Helper to create key for tracking function disabled state -+ static nsString DisabledKeyForUserContext(uint32_t userContextId); -+}; -+ -+} // namespace dom -+} // namespace mozilla -+ -+#endif // mozilla_dom_FontSpacingSeedManager_h -\ No newline at end of file diff --git a/dom/base/RoverfoxStorageManager.cpp b/dom/base/RoverfoxStorageManager.cpp new file mode 100644 index 0000000000..dcb359d25a @@ -422,18 +246,10 @@ index 0000000000..e6d64378e1 + +#endif // mozilla_dom_RoverfoxStorageManager_h diff --git a/dom/base/moz.build b/dom/base/moz.build -index 7e8d942191..d922362ddd 100644 +index 7e8d942191..e44e7259bf 100644 --- a/dom/base/moz.build +++ b/dom/base/moz.build -@@ -191,6 +191,7 @@ EXPORTS.mozilla.dom += [ - "External.h", - "FastFrontRemovableArray.h", - "FilteredNodeIterator.h", -+ "FontSpacingSeedManager.h", - "FormData.h", - "FragmentDirective.h", - "FragmentOrElement.h", -@@ -257,6 +258,7 @@ EXPORTS.mozilla.dom += [ +@@ -257,6 +257,7 @@ EXPORTS.mozilla.dom += [ "RequestCallbackManager.h", "ResizeObserver.h", "ResponsiveImageSelector.h", @@ -441,15 +257,7 @@ index 7e8d942191..d922362ddd 100644 "SameProcessMessageQueue.h", "ScreenLuminance.h", "ScreenOrientation.h", -@@ -372,6 +374,7 @@ UNIFIED_SOURCES += [ - "EventSource.cpp", - "EventSourceEventService.cpp", - "External.cpp", -+ "FontSpacingSeedManager.cpp", - "FormData.cpp", - "FragmentDirective.cpp", - "FragmentOrElement.cpp", -@@ -461,6 +464,7 @@ UNIFIED_SOURCES += [ +@@ -461,6 +462,7 @@ UNIFIED_SOURCES += [ "RemoteOuterWindowProxy.cpp", "ResizeObserver.cpp", "ResponsiveImageSelector.cpp", @@ -458,7 +266,7 @@ index 7e8d942191..d922362ddd 100644 "ScreenLuminance.cpp", "ScreenOrientation.cpp", diff --git a/dom/base/nsGlobalWindowInner.cpp b/dom/base/nsGlobalWindowInner.cpp -index 7cdddd10c1..8c4eb1b6c7 100644 +index 7cdddd10c1..922ddbc377 100644 --- a/dom/base/nsGlobalWindowInner.cpp +++ b/dom/base/nsGlobalWindowInner.cpp @@ -56,6 +56,7 @@ @@ -469,91 +277,15 @@ index 7cdddd10c1..8c4eb1b6c7 100644 #include "mozilla/ExtensionPolicyService.h" #include "mozilla/FloatingPoint.h" #include "mozilla/FlushType.h" -@@ -241,6 +242,9 @@ +@@ -241,6 +242,8 @@ #include "nsICookieService.h" #include "nsID.h" #include "nsIDOMStorageManager.h" -+#include "FontSpacingSeedManager.h" +#include "nsDocShell.h" +#include "mozilla/OriginAttributes.h" #include "nsIDOMXULControlElement.h" #include "nsIDeviceSensors.h" #include "nsIDocShell.h" -@@ -7728,6 +7732,25 @@ IntlUtils* nsGlobalWindowInner::GetIntlUtils(ErrorResult& aError) { - return mIntlUtils; - } - -+void nsGlobalWindowInner::SetFontSpacingSeed(uint32_t seed, ErrorResult& aRv) { -+ uint32_t userContextId = 0; -+ if (BrowsingContext* bc = GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ -+ FontSpacingSeedManager::SetSeed(userContextId, seed); -+ -+ // Self-destruct: set to undefined first to bust IC/shape caches on ARM64. -+ if (JSContext* cx = nsContentUtils::GetCurrentJSContext()) { -+ JS::Rooted global(cx, JS::CurrentGlobalOrNull(cx)); -+ if (global) { -+ JS::Rooted undef(cx, JS::UndefinedValue()); -+ JS_SetProperty(cx, global, "setFontSpacingSeed", undef); -+ JS_DeleteProperty(cx, global, "setFontSpacingSeed"); -+ } -+ } -+} -+ - void nsGlobalWindowInner::StoreSharedWorker(SharedWorker* aSharedWorker) { - MOZ_ASSERT(aSharedWorker); - MOZ_ASSERT(!mSharedWorkers.Contains(aSharedWorker)); -diff --git a/dom/base/nsGlobalWindowInner.h b/dom/base/nsGlobalWindowInner.h -index 5292a4d543..5b7cd52b9b 100644 ---- a/dom/base/nsGlobalWindowInner.h -+++ b/dom/base/nsGlobalWindowInner.h -@@ -680,6 +680,9 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, - - mozilla::dom::IntlUtils* GetIntlUtils(mozilla::ErrorResult& aRv); - -+ // Font spacing seed for privacy-preserving font fingerprinting -+ void SetFontSpacingSeed(uint32_t seed, mozilla::ErrorResult& aRv); -+ - void StoreSharedWorker(mozilla::dom::SharedWorker* aSharedWorker); - - void ForgetSharedWorker(mozilla::dom::SharedWorker* aSharedWorker); -diff --git a/dom/canvas/CanvasRenderingContext2D.cpp b/dom/canvas/CanvasRenderingContext2D.cpp -index 45b0de48b5..dd79a43beb 100644 ---- a/dom/canvas/CanvasRenderingContext2D.cpp -+++ b/dom/canvas/CanvasRenderingContext2D.cpp -@@ -58,6 +58,7 @@ - #include "mozilla/dom/CanvasRenderingContext2DBinding.h" - #include "mozilla/dom/DOMMatrix.h" - #include "mozilla/dom/Document.h" -+#include "mozilla/dom/BrowsingContext.h" - #include "mozilla/dom/FontFaceSet.h" - #include "mozilla/dom/FontFaceSetImpl.h" - #include "mozilla/dom/GeneratePlaceholderCanvasData.h" -@@ -4775,10 +4776,21 @@ struct MOZ_STACK_CLASS CanvasBidiProcessor final - } else { - flags &= ~gfx::ShapedTextFlags::TEXT_IS_RTL; - } -+ // Extract user context ID via BrowsingContext (canonical source) -+ uint32_t userContextId = 0; -+ if (mCtx && mCtx->mCanvasElement) { -+ if (Document* doc = mCtx->mCanvasElement->GetOwnerDocument()) { -+ if (nsPIDOMWindowInner* win = doc->GetInnerWindow()) { -+ if (BrowsingContext* bc = win->GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ } -+ } -+ } - mTextRun = mFontgrp->MakeTextRun( - aText, aLength, mDrawTarget, mAppUnitsPerDevPixel, flags, - nsTextFrameUtils::Flags::DontSkipDrawingForPendingUserFonts, -- mMissingFonts.get()); -+ mMissingFonts.get(), userContextId); - pfl->Unlock(); - } - diff --git a/dom/canvas/OffscreenCanvas.cpp b/dom/canvas/OffscreenCanvas.cpp index 7234240711..df84b793dc 100644 --- a/dom/canvas/OffscreenCanvas.cpp @@ -574,23 +306,6 @@ index 7234240711..df84b793dc 100644 bool OffscreenCanvas::IsChrome() const { if (NS_IsMainThread()) { nsCOMPtr win = do_QueryInterface(GetRelevantGlobal()); -diff --git a/dom/webidl/Window.webidl b/dom/webidl/Window.webidl -index 77f35801a0..454ef6f4e7 100644 ---- a/dom/webidl/Window.webidl -+++ b/dom/webidl/Window.webidl -@@ -934,6 +934,12 @@ partial interface Window { - readonly attribute VisualViewport visualViewport; - }; - -+// Font spacing seed interface for privacy-preserving font fingerprinting -+partial interface Window { -+ [Throws, Func="mozilla::dom::FontSpacingSeedManager::IsFunctionEnabledForWebIDL"] -+ undefined setFontSpacingSeed(unsigned long seed); -+}; -+ - // Used to assign marks to appear on the scrollbar when - // finding on a page. - partial interface Window { diff --git a/dom/workers/WorkerPrivate.h b/dom/workers/WorkerPrivate.h index 7f56e1c86d..80b050c08b 100644 --- a/dom/workers/WorkerPrivate.h @@ -604,324 +319,6 @@ index 7f56e1c86d..80b050c08b 100644 void MemoryPressure(); void UpdateContextOptions(const JS::ContextOptions& aContextOptions); -diff --git a/gfx/src/nsFontMetrics.cpp b/gfx/src/nsFontMetrics.cpp -index a52e3a56d8..3025f57da5 100644 ---- a/gfx/src/nsFontMetrics.cpp -+++ b/gfx/src/nsFontMetrics.cpp -@@ -22,9 +22,13 @@ - #include "nsStyleConsts.h" // for StyleHyphens::None - #include "mozilla/Assertions.h" // for MOZ_ASSERT - #include "mozilla/UniquePtr.h" // for UniquePtr -+#include "mozilla/dom/Document.h" // for Document -+#include "mozilla/dom/BrowsingContext.h" // for BrowsingContext -+#include "nsPIDOMWindow.h" // for nsPIDOMWindowInner - - class gfxUserFontSet; - using namespace mozilla; -+using namespace mozilla::dom; - - namespace { - -@@ -34,17 +38,39 @@ class AutoTextRun { - - AutoTextRun(const nsFontMetrics* aMetrics, DrawTarget* aDrawTarget, - const char* aString, uint32_t aLength) { -+ // Extract user context ID via BrowsingContext (canonical source) -+ uint32_t userContextId = 0; -+ if (aMetrics->mPresContext) { -+ if (Document* doc = aMetrics->mPresContext->Document()) { -+ if (nsPIDOMWindowInner* win = doc->GetInnerWindow()) { -+ if (BrowsingContext* bc = win->GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ } -+ } -+ } - mTextRun = aMetrics->GetThebesFontGroup()->MakeTextRun( - reinterpret_cast(aString), aLength, aDrawTarget, - aMetrics->AppUnitsPerDevPixel(), ComputeFlags(aMetrics), -- nsTextFrameUtils::Flags(), nullptr); -+ nsTextFrameUtils::Flags(), nullptr, userContextId); - } - - AutoTextRun(const nsFontMetrics* aMetrics, DrawTarget* aDrawTarget, - const char16_t* aString, uint32_t aLength) { -+ // Extract user context ID via BrowsingContext (canonical source) -+ uint32_t userContextId = 0; -+ if (aMetrics->mPresContext) { -+ if (Document* doc = aMetrics->mPresContext->Document()) { -+ if (nsPIDOMWindowInner* win = doc->GetInnerWindow()) { -+ if (BrowsingContext* bc = win->GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ } -+ } -+ } - mTextRun = aMetrics->GetThebesFontGroup()->MakeTextRun( - aString, aLength, aDrawTarget, aMetrics->AppUnitsPerDevPixel(), -- ComputeFlags(aMetrics), nsTextFrameUtils::Flags(), nullptr); -+ ComputeFlags(aMetrics), nsTextFrameUtils::Flags(), nullptr, userContextId); - } - - gfxTextRun* get() const { return mTextRun.get(); } -diff --git a/gfx/src/nsFontMetrics.h b/gfx/src/nsFontMetrics.h -index df44559b26..d829aa8ff2 100644 ---- a/gfx/src/nsFontMetrics.h -+++ b/gfx/src/nsFontMetrics.h -@@ -299,6 +299,10 @@ class nsFontMetrics final { - bool AllowForceGDIClassic() const { return mAllowForceGDIClassic; } - #endif - -+ // Pointer to the pres context for which this fontMetrics object was -+ // created. -+ nsPresContext* MOZ_NON_OWNING_REF mPresContext; -+ - private: - // Private destructor, to discourage deletion outside of Release(): - ~nsFontMetrics(); -@@ -306,9 +310,6 @@ class nsFontMetrics final { - const nsFont mFont; - RefPtr mFontGroup; - RefPtr const mLanguage; -- // Pointer to the pres context for which this fontMetrics object was -- // created. -- nsPresContext* MOZ_NON_OWNING_REF mPresContext; - const int32_t mP2A; - - // The font orientation (horizontal or vertical) for which these metrics -diff --git a/gfx/thebes/gfxFont.cpp b/gfx/thebes/gfxFont.cpp -index 39a0a013de..ce1c233d7d 100644 ---- a/gfx/thebes/gfxFont.cpp -+++ b/gfx/thebes/gfxFont.cpp -@@ -18,6 +18,7 @@ - - #include "gfxGlyphExtents.h" - #include "gfxPlatform.h" -+#include "mozilla/dom/FontSpacingSeedManager.h" - #include "gfxTextRun.h" - #include "nsGkAtoms.h" - -@@ -3308,9 +3309,10 @@ bool gfxFont::ProcessShapedWordInternal( - const T* aText, uint32_t aLength, uint32_t aHash, Script aRunScript, - nsAtom* aLanguage, bool aVertical, int32_t aAppUnitsPerDevUnit, - gfx::ShapedTextFlags aFlags, RoundingFlags aRounding, -- gfxTextPerfMetrics* aTextPerf GFX_MAYBE_UNUSED, Func aCallback) { -+ gfxTextPerfMetrics* aTextPerf GFX_MAYBE_UNUSED, -+ uint32_t aUserContextId, Func aCallback) { - WordCacheKey key(aText, aLength, aHash, aRunScript, aLanguage, -- aAppUnitsPerDevUnit, aFlags, aRounding); -+ aAppUnitsPerDevUnit, aFlags, aRounding, aUserContextId); - { - // If we have a word cache, attempt to look up the word in it. - AutoReadLock lock(mLock); -@@ -3341,6 +3343,8 @@ bool gfxFont::ProcessShapedWordInternal( - NS_WARNING("failed to create gfxShapedWord - expect missing text"); - return false; - } -+ // Propagate user context ID for HarfBuzz shaping/logging. -+ newShapedWord->SetUserContextId(aUserContextId); - DebugOnly ok = ShapeText(aText, 0, aLength, aRunScript, aLanguage, - aVertical, aRounding, newShapedWord.get()); - NS_WARNING_ASSERTION(ok, "failed to shape word - expect garbled text"); -@@ -3397,7 +3401,8 @@ bool gfxFont::WordCacheKey::HashPolicy::match(const Key& aKey, - if (aKey.mLength != aLookup.mLength || aKey.mFlags != aLookup.mFlags || - aKey.mRounding != aLookup.mRounding || - aKey.mAppUnitsPerDevUnit != aLookup.mAppUnitsPerDevUnit || -- aKey.mScript != aLookup.mScript || aKey.mLanguage != aLookup.mLanguage) { -+ aKey.mScript != aLookup.mScript || aKey.mLanguage != aLookup.mLanguage || -+ aKey.mUserContextId != aLookup.mUserContextId) { - return false; - } - -@@ -3434,7 +3439,7 @@ bool gfxFont::ProcessSingleSpaceShapedWord( - return ProcessShapedWordInternal( - &space, 1, gfxShapedWord::HashMix(0, ' '), Script::LATIN, - /* aLanguage = */ nullptr, aVertical, aAppUnitsPerDevUnit, aFlags, -- aRounding, nullptr, aCallback); -+ aRounding, nullptr, 0 /* pbid */, aCallback); - } - - bool gfxFont::ShapeText(const uint8_t* aText, uint32_t aOffset, -@@ -3788,6 +3793,7 @@ bool gfxFont::SplitAndInitTextRun( - bool processed = ProcessShapedWordInternal( - aString + wordStart, length, hash, aRunScript, aLanguage, vertical, - appUnitsPerDevUnit, wordFlags, rounding, tp, -+ aTextRun->GetUserContextId(), - [&](gfxShapedWord* aShapedWord) { - aTextRun->CopyGlyphDataFrom(aShapedWord, aRunStart + wordStart); - }); -@@ -3813,6 +3819,7 @@ bool gfxFont::SplitAndInitTextRun( - &boundary, 1, gfxShapedWord::HashMix(0, boundary), aRunScript, - aLanguage, vertical, appUnitsPerDevUnit, - flags | gfx::ShapedTextFlags::TEXT_IS_8BIT, rounding, tp, -+ aTextRun->GetUserContextId(), - [&](gfxShapedWord* aShapedWord) { - aTextRun->CopyGlyphDataFrom(aShapedWord, aRunStart + i); - if (boundary == ' ') { -diff --git a/gfx/thebes/gfxFont.h b/gfx/thebes/gfxFont.h -index 25022d4b5b..fe65662b77 100644 ---- a/gfx/thebes/gfxFont.h -+++ b/gfx/thebes/gfxFont.h -@@ -746,6 +746,10 @@ class gfxShapedText { - - virtual ~gfxShapedText() = default; - -+ // Optional accessor overridden by gfxTextRun to expose the private browsing ID. -+ // Default returns 0 for non-textrun shaped text objects. -+ virtual uint32_t GetUserContextId() const { return 0; } -+ - /** - * This class records the information associated with a character in the - * input string. It's optimized for the case where there is one glyph -@@ -1335,6 +1339,11 @@ class gfxShapedWord final : public gfxShapedText { - // allocated via malloc. - void operator delete(void* p) { free(p); } - -+ // User Context ID plumbing for shaping-time access. -+ // HarfBuzz shaper will query this via gfxShapedText::GetUserContextId(). -+ void SetUserContextId(uint32_t aId) { mUserContextId = aId; } -+ uint32_t GetUserContextId() const override { return mUserContextId; } -+ - const CompressedGlyph* GetCharacterGlyphs() const override { - return &mCharGlyphsStorage[0]; - } -@@ -1418,6 +1427,9 @@ class gfxShapedWord final : public gfxShapedText { - // With multithreaded shaping, this may be updated by any thread. - std::atomic mAgeCounter; - -+ // User Context ID carried with the shaped word for shaper access. -+ uint32_t mUserContextId = 0; -+ - // The mCharGlyphsStorage array is actually a variable-size member; - // when the ShapedWord is created, its size will be increased as necessary - // to allow the proper number of glyphs to be stored. -@@ -2155,7 +2167,8 @@ class gfxFont { - int32_t aAppUnitsPerDevUnit, - mozilla::gfx::ShapedTextFlags aFlags, - RoundingFlags aRounding, -- gfxTextPerfMetrics* aTextPerf, Func aCallback); -+ gfxTextPerfMetrics* aTextPerf, -+ uint32_t aUserContextId, Func aCallback); - - // whether a given feature is included in feature settings from both the - // font and the style. aFeatureOn set if resolved feature value is non-zero -@@ -2185,12 +2198,13 @@ class gfxFont { - int32_t mAppUnitsPerDevUnit; - PLDHashNumber mHashKey; - bool mTextIs8Bit; -+ uint32_t mUserContextId; - RoundingFlags mRounding; - - WordCacheKey(const uint8_t* aText, uint32_t aLength, uint32_t aStringHash, - Script aScriptCode, nsAtom* aLanguage, - int32_t aAppUnitsPerDevUnit, ShapedTextFlags aFlags, -- RoundingFlags aRounding) -+ RoundingFlags aRounding, uint32_t aUserContextId = 0) - : mLength(aLength), - mFlags(aFlags), - mScript(aScriptCode), -@@ -2198,8 +2212,9 @@ class gfxFont { - mAppUnitsPerDevUnit(aAppUnitsPerDevUnit), - mHashKey(aStringHash + static_cast(aScriptCode) + - aAppUnitsPerDevUnit * 0x100 + uint16_t(aFlags) * 0x10000 + -- int(aRounding) + (aLanguage ? aLanguage->hash() : 0)), -+ int(aRounding) + (aLanguage ? aLanguage->hash() : 0) + aUserContextId * 0x1000000), - mTextIs8Bit(true), -+ mUserContextId(aUserContextId), - mRounding(aRounding) { - NS_ASSERTION(aFlags & ShapedTextFlags::TEXT_IS_8BIT, - "8-bit flag should have been set"); -@@ -2209,7 +2224,7 @@ class gfxFont { - WordCacheKey(const char16_t* aText, uint32_t aLength, uint32_t aStringHash, - Script aScriptCode, nsAtom* aLanguage, - int32_t aAppUnitsPerDevUnit, ShapedTextFlags aFlags, -- RoundingFlags aRounding) -+ RoundingFlags aRounding, uint32_t aUserContextId = 0) - : mLength(aLength), - mFlags(aFlags), - mScript(aScriptCode), -@@ -2217,8 +2232,9 @@ class gfxFont { - mAppUnitsPerDevUnit(aAppUnitsPerDevUnit), - mHashKey(aStringHash + static_cast(aScriptCode) + - aAppUnitsPerDevUnit * 0x100 + uint16_t(aFlags) * 0x10000 + -- int(aRounding)), -+ int(aRounding) + aUserContextId * 0x1000000), - mTextIs8Bit(false), -+ mUserContextId(aUserContextId), - mRounding(aRounding) { - // We can NOT assert that TEXT_IS_8BIT is false in aFlags here, - // because this might be an 8bit-only word from a 16-bit textrun, -diff --git a/gfx/thebes/gfxHarfBuzzShaper.cpp b/gfx/thebes/gfxHarfBuzzShaper.cpp -index e41408f51c..448fcd58fd 100644 ---- a/gfx/thebes/gfxHarfBuzzShaper.cpp -+++ b/gfx/thebes/gfxHarfBuzzShaper.cpp -@@ -16,6 +16,10 @@ - - #include "harfbuzz/hb.h" - #include "harfbuzz/hb-ot.h" -+#include -+#include -+#include "MaskConfig.hpp" -+#include "mozilla/dom/FontSpacingSeedManager.h" - - #include - -@@ -1465,6 +1469,50 @@ bool gfxHarfBuzzShaper::ShapeText(const char16_t* aText, uint32_t aOffset, - - hb_shape(mHBFont, mBuffer, features.Elements(), features.Length()); - -+ // Resolve seed from manager using user context ID. -+ // seed == 0 means no perturbation (consistent with AudioFingerprintManager). -+ uint32_t pbid = aShapedText ? aShapedText->GetUserContextId() : 0; -+ uint32_t seed = mozilla::dom::FontSpacingSeedManager::GetSeed(pbid); -+ -+ if (seed != 0) { -+ // Generate a random float [0, 0.1] to offset the letter spacing -+ seed = (seed * 1103515245 + 12345) & 0x7fffffff; -+ float randomFloat = (static_cast(seed) / 2147483647.0f) * 0.1f; -+ hb_position_t spacing = FloatToFixed(randomFloat); -+ -+ uint32_t glyphCount; -+ hb_glyph_position_t* glyphPositions = -+ hb_buffer_get_glyph_positions(mBuffer, &glyphCount); -+ -+ // Perturb ADVANCES only. -+ // -+ // x_offset/y_offset are the positioning offsets GPOS uses to place a glyph -+ // relative to the pen -- most importantly to park a combining mark over -+ // its base. They do not contribute to the measured width, so they add -+ // nothing to the metric we are perturbing, but layout never sees them: -+ // gfxTextRun stores advances, while painting honours the offsets. Feeding -+ // a running total into them paints every glyph a further `spacing` past -+ // where layout placed it and detaches every combining mark from its base. -+ // Scripts whose marks carry a zero advance -- Thai, Lao, Arabic, -+ // Devanagari, Hebrew -- collapse into stacked glyphs, while Latin merely -+ // looks slightly loose (daijro/camoufox#741). -+ // -+ // Zero-advance glyphs are skipped for the same reason: a combining mark is -+ // not a separate character, and widening it pushes the following base away -+ // and strands the mark. Every advancing glyph still gets +spacing, so the -+ // measured width -- the actual fingerprinting signal -- is perturbed -+ // exactly as before. -+ for (uint32_t i = 0; i < glyphCount; ++i) { -+ if (aVertical) { -+ if (glyphPositions[i].y_advance != 0) { -+ glyphPositions[i].y_advance -= spacing; -+ } -+ } else if (glyphPositions[i].x_advance != 0) { -+ glyphPositions[i].x_advance += spacing; -+ } -+ } -+ } -+ - if (isRightToLeft) { - hb_buffer_reverse(mBuffer); - } -@@ -1770,3 +1818,4 @@ nsresult gfxHarfBuzzShaper::SetGlyphsFromRun(gfxShapedText* aShapedText, - - return NS_OK; - } -+ diff --git a/gfx/thebes/gfxPlatformFontList.cpp b/gfx/thebes/gfxPlatformFontList.cpp index b3c22e2bb1..6bdcd2a57c 100644 --- a/gfx/thebes/gfxPlatformFontList.cpp @@ -935,7 +332,7 @@ index b3c22e2bb1..6bdcd2a57c 100644 private: diff --git a/gfx/thebes/gfxTextRun.cpp b/gfx/thebes/gfxTextRun.cpp -index fc1293be42..45641bd8b3 100644 +index fc1293be42..cac43a8b1f 100644 --- a/gfx/thebes/gfxTextRun.cpp +++ b/gfx/thebes/gfxTextRun.cpp @@ -24,6 +24,7 @@ @@ -956,39 +353,7 @@ index fc1293be42..45641bd8b3 100644 #ifdef XP_WIN # include "gfxWindowsPlatform.h" -@@ -144,25 +148,27 @@ void* gfxTextRun::AllocateStorageForTextRun(size_t aSize, uint32_t aLength) { - already_AddRefed gfxTextRun::Create( - const gfxTextRunFactory::Parameters* aParams, uint32_t aLength, - gfxFontGroup* aFontGroup, gfx::ShapedTextFlags aFlags, -- nsTextFrameUtils::Flags aFlags2) { -+ nsTextFrameUtils::Flags aFlags2, uint32_t aUserContextId) { - void* storage = AllocateStorageForTextRun(sizeof(gfxTextRun), aLength); - if (!storage) { - return nullptr; - } - -- RefPtr result = -- new (storage) gfxTextRun(aParams, aLength, aFontGroup, aFlags, aFlags2); -+ RefPtr result = new (storage) -+ gfxTextRun(aParams, aLength, aFontGroup, aFlags, aFlags2, aUserContextId); - return result.forget(); - } - - gfxTextRun::gfxTextRun(const gfxTextRunFactory::Parameters* aParams, - uint32_t aLength, gfxFontGroup* aFontGroup, - gfx::ShapedTextFlags aFlags, -- nsTextFrameUtils::Flags aFlags2) -+ nsTextFrameUtils::Flags aFlags2, -+ uint32_t aUserContextId) - : gfxShapedText(aLength, aFlags, aParams->mAppUnitsPerDevUnit), - mUserData(aParams->mUserData), - mFontGroup(aFontGroup), - mFlags2(aFlags2), -+ mUserContextId(aUserContextId), - mShapingState(eShapingState_Normal) { - NS_ASSERTION(mAppUnitsPerDevUnit > 0, "Invalid app unit scale"); - NS_ADDREF(mFontGroup); -@@ -699,7 +705,8 @@ void gfxTextRun::DrawEmphasisMarks( +@@ -699,7 +703,8 @@ void gfxTextRun::DrawEmphasisMarks( gfxContext* aContext, gfxTextRun* aMark, gfxFloat aMarkAdvance, gfx::Point aPt, Range aRange, const PropertyProvider* aProvider, mozilla::gfx::PaletteCache& aPaletteCache) const { @@ -998,7 +363,7 @@ index fc1293be42..45641bd8b3 100644 EmphasisMarkDrawParams params(aContext, aPaletteCache); params.mark = aMark; -@@ -1841,6 +1848,18 @@ gfxFontGroup::gfxFontGroup(FontVisibilityProvider* aFontVisibilityProvider, +@@ -1841,6 +1846,18 @@ gfxFontGroup::gfxFontGroup(FontVisibilityProvider* aFontVisibilityProvider, mFontVariantEmoji(aVariantEmoji) { // We don't use SetUserFontSet() here, as we want to unconditionally call // EnsureFontList() rather than only do UpdateUserFonts() if it changed. @@ -1017,60 +382,7 @@ index fc1293be42..45641bd8b3 100644 } gfxFontGroup::~gfxFontGroup() { -@@ -2453,12 +2472,12 @@ already_AddRefed gfxFontGroup::MakeHyphenTextRun( - RefPtr font = GetFirstValidFont(uint32_t(hyphen)); - if (font->HasCharacter(hyphen)) { - return MakeTextRun(&hyphen, 1, aDrawTarget, aAppUnitsPerDevUnit, aFlags, -- nsTextFrameUtils::Flags(), nullptr); -+ nsTextFrameUtils::Flags(), nullptr, mUserContextId); - } - - static const uint8_t dash = '-'; - return MakeTextRun(&dash, 1, aDrawTarget, aAppUnitsPerDevUnit, aFlags, -- nsTextFrameUtils::Flags(), nullptr); -+ nsTextFrameUtils::Flags(), nullptr, mUserContextId); - } - - gfxFloat gfxFontGroup::GetHyphenWidth( -@@ -2479,7 +2498,7 @@ template - already_AddRefed gfxFontGroup::MakeTextRun( - const T* aString, uint32_t aLength, const Parameters* aParams, - gfx::ShapedTextFlags aFlags, nsTextFrameUtils::Flags aFlags2, -- gfxMissingFontRecorder* aMFR) { -+ gfxMissingFontRecorder* aMFR, uint32_t aUserContextId) { - if (aLength == 0) { - return MakeEmptyTextRun(aParams, aFlags, aFlags2); - } -@@ -2498,8 +2517,13 @@ already_AddRefed gfxFontGroup::MakeTextRun( - return MakeBlankTextRun(aString, aLength, aParams, aFlags, aFlags2); - } - -+ // If caller didn't provide an ID, default to the group's cached value. -+ if (aUserContextId == 0) { -+ aUserContextId = mUserContextId; -+ } -+ - RefPtr textRun = -- gfxTextRun::Create(aParams, aLength, this, aFlags, aFlags2); -+ gfxTextRun::Create(aParams, aLength, this, aFlags, aFlags2, aUserContextId); - if (!textRun) { - return nullptr; - } -@@ -2515,11 +2539,11 @@ already_AddRefed gfxFontGroup::MakeTextRun( - template already_AddRefed gfxFontGroup::MakeTextRun( - const uint8_t* aString, uint32_t aLength, const Parameters* aParams, - gfx::ShapedTextFlags aFlags, nsTextFrameUtils::Flags aFlags2, -- gfxMissingFontRecorder* aMFR); -+ gfxMissingFontRecorder* aMFR, uint32_t aUserContextId); - template already_AddRefed gfxFontGroup::MakeTextRun( - const char16_t* aString, uint32_t aLength, const Parameters* aParams, - gfx::ShapedTextFlags aFlags, nsTextFrameUtils::Flags aFlags2, -- gfxMissingFontRecorder* aMFR); -+ gfxMissingFontRecorder* aMFR, uint32_t aUserContextId); - - // ComputeRanges instantiation (used by - // gfxPlatformFontList::ListFontsUsedForString). -@@ -2583,8 +2607,9 @@ static Script ResolveScriptForLang(const nsAtom* aLanguage, Script aDefault) { +@@ -2583,8 +2600,9 @@ static Script ResolveScriptForLang(const nsAtom* aLanguage, Script aDefault) { static LangScriptCache sCache; static RWLock sLock("LangScriptCache lock"); @@ -1082,7 +394,7 @@ index fc1293be42..45641bd8b3 100644 { // Try to use a cached value without taking an exclusive lock. -@@ -3418,7 +3443,7 @@ already_AddRefed gfxFontGroup::FindFontForChar( +@@ -3418,7 +3436,7 @@ already_AddRefed gfxFontGroup::FindFontForChar( font = FindFallbackFaceForChar(ff, aCh, aNextCh, presentation); if (font) { if (CheckCandidate(font, @@ -1092,136 +404,18 @@ index fc1293be42..45641bd8b3 100644 } } diff --git a/gfx/thebes/gfxTextRun.h b/gfx/thebes/gfxTextRun.h -index 85aecf5fdd..5fdfe522fc 100644 +index 85aecf5fdd..79f4df71a4 100644 --- a/gfx/thebes/gfxTextRun.h +++ b/gfx/thebes/gfxTextRun.h -@@ -485,13 +485,14 @@ class gfxTextRun : public gfxShapedText { - void ClearFlagBits(nsTextFrameUtils::Flags aFlags) { mFlags2 &= ~aFlags; } - const gfxSkipChars& GetSkipChars() const { return mSkipChars; } - gfxFontGroup* GetFontGroup() const { return mFontGroup; } -+ uint32_t GetUserContextId() const override { return mUserContextId; } - - // Call this, don't call "new gfxTextRun" directly. This does custom - // allocation and initialization - static already_AddRefed Create( - const gfxTextRunFactory::Parameters* aParams, uint32_t aLength, - gfxFontGroup* aFontGroup, mozilla::gfx::ShapedTextFlags aFlags, -- nsTextFrameUtils::Flags aFlags2); -+ nsTextFrameUtils::Flags aFlags2, uint32_t aUserContextId = 0); - - // The text is divided into GlyphRuns as necessary. (In the vast majority - // of cases, a gfxTextRun contains just a single GlyphRun.) -@@ -801,7 +802,7 @@ class gfxTextRun : public gfxShapedText { - */ - gfxTextRun(const gfxTextRunFactory::Parameters* aParams, uint32_t aLength, - gfxFontGroup* aFontGroup, mozilla::gfx::ShapedTextFlags aFlags, -- nsTextFrameUtils::Flags aFlags2); -+ nsTextFrameUtils::Flags aFlags2, uint32_t aUserContextId = 0); - - // Whether we need to fetch actual glyph extents from the fonts. - bool NeedsGlyphExtents() const; -@@ -893,6 +894,8 @@ class gfxTextRun : public gfxShapedText { - nsTextFrameUtils::Flags - mFlags2; // additional flags (see also gfxShapedText::mFlags) - -+ uint32_t mUserContextId; // user context ID for font spacing seed -+ - bool mDontSkipDrawing; // true if the text run must not skip drawing, even if - // waiting for a user font download, e.g. because we - // are using it to draw canvas text -@@ -970,7 +973,8 @@ class gfxFontGroup final : public gfxTextRunFactory { - const Parameters* aParams, - mozilla::gfx::ShapedTextFlags aFlags, - nsTextFrameUtils::Flags aFlags2, -- gfxMissingFontRecorder* aMFR); -+ gfxMissingFontRecorder* aMFR, -+ uint32_t aUserContextId = 0); - - /** - * Textrun creation helper for clients that don't want to pass -@@ -982,10 +986,11 @@ class gfxFontGroup final : public gfxTextRunFactory { - int32_t aAppUnitsPerDevUnit, - mozilla::gfx::ShapedTextFlags aFlags, - nsTextFrameUtils::Flags aFlags2, -- gfxMissingFontRecorder* aMFR) { -+ gfxMissingFontRecorder* aMFR, -+ uint32_t aUserContextId = 0) { - gfxTextRunFactory::Parameters params = { - aRefDrawTarget, nullptr, nullptr, nullptr, 0, aAppUnitsPerDevUnit}; -- return MakeTextRun(aString, aLength, ¶ms, aFlags, aFlags2, aMFR); -+ return MakeTextRun(aString, aLength, ¶ms, aFlags, aFlags2, aMFR, aUserContextId); - } - - // Get the (possibly-cached) width of the hyphen character. -@@ -1401,6 +1406,8 @@ class gfxFontGroup final : public gfxTextRunFactory { +@@ -1401,6 +1401,8 @@ class gfxFontGroup final : public gfxTextRunFactory { uint32_t mFontListGeneration = 0; // platform font list generation for this // fontgroup -+ uint32_t mUserContextId = 0; // user context ID for font spacing seed ++ uint32_t mUserContextId = 0; // selects the per-context font list + /** * Textrun creation short-cuts for special cases where we don't need to * call a font shaper to generate glyphs. -diff --git a/layout/base/nsLayoutUtils.cpp b/layout/base/nsLayoutUtils.cpp -index 585561b2a0..1e76b87091 100644 ---- a/layout/base/nsLayoutUtils.cpp -+++ b/layout/base/nsLayoutUtils.cpp -@@ -144,6 +144,7 @@ - #include "nsIFrameInlines.h" - #include "nsIImageLoadingContent.h" - #include "nsIInterfaceRequestorUtils.h" -+#include "mozilla/dom/BrowsingContext.h" - #include "nsIWidget.h" - #include "nsListControlFrame.h" - #include "nsMenuPopupFrame.h" -@@ -1212,6 +1213,31 @@ int32_t nsLayoutUtils::DoCompareTreePosition(const nsIFrame* aFrame1, - nonCommonAncestor ? aCommonAncestor : nullptr); - } - -+/* static */ uint32_t -+nsLayoutUtils::GetUserContextId(nsIFrame* aFrame) -+{ -+ if (!aFrame || !aFrame->GetContent()) { -+ return 0; -+ } -+ -+ mozilla::dom::Document* doc = aFrame->GetContent()->GetComposedDoc(); -+ if (!doc) { -+ return 0; -+ } -+ -+ nsPIDOMWindowInner* win = doc->GetInnerWindow(); -+ if (!win) { -+ return 0; -+ } -+ -+ mozilla::dom::BrowsingContext* bc = win->GetBrowsingContext(); -+ if (!bc) { -+ return 0; -+ } -+ -+ return bc->OriginAttributesRef().mUserContextId; -+} -+ - // static - int32_t nsLayoutUtils::DoCompareTreePosition( - const nsIFrame* aFrame1, const nsIFrame* aFrame2, -diff --git a/layout/base/nsLayoutUtils.h b/layout/base/nsLayoutUtils.h -index b453efb3c7..0bbe0f36f7 100644 ---- a/layout/base/nsLayoutUtils.h -+++ b/layout/base/nsLayoutUtils.h -@@ -439,6 +439,11 @@ class nsLayoutUtils { - */ - static nsIFrame* GetLastSibling(nsIFrame* aFrame); - -+ /** -+ * Get the user context ID from a frame's document context -+ */ -+ static uint32_t GetUserContextId(nsIFrame* aFrame); -+ - /** - * FindSiblingViewFor locates the child of aParentView that aFrame's - * view should be inserted 'above' (i.e., before in sibling view diff --git a/layout/base/nsPresContext.cpp b/layout/base/nsPresContext.cpp index b2394c6106..da755e553c 100644 --- a/layout/base/nsPresContext.cpp @@ -1237,186 +431,6 @@ index b2394c6106..da755e553c 100644 void nsPresContext::GetUserPreferences() { if (!GetPresShell()) { // No presshell means nothing to do here. We'll do this when we -diff --git a/layout/generic/MathMLTextRunFactory.cpp b/layout/generic/MathMLTextRunFactory.cpp -index 22adbaf9bc..e685fd4feb 100644 ---- a/layout/generic/MathMLTextRunFactory.cpp -+++ b/layout/generic/MathMLTextRunFactory.cpp -@@ -9,6 +9,8 @@ - #include "mozilla/ComputedStyleInlines.h" - #include "mozilla/StaticPrefs_mathml.h" - #include "mozilla/intl/UnicodeScriptCodes.h" -+#include "mozilla/dom/BrowsingContext.h" -+#include "nsPIDOMWindow.h" - #include "nsDeviceContext.h" - #include "nsFontMetrics.h" - #include "nsStyleConsts.h" -@@ -635,6 +637,18 @@ void MathMLTextRunFactory::RebuildTextRun( - newFontGroup = fontGroup; - } - -+ // Extract user context ID via BrowsingContext (canonical source) -+ uint32_t userContextId = 0; -+ if (length && styles[0]->mPresContext) { -+ if (mozilla::dom::Document* doc = styles[0]->mPresContext->Document()) { -+ if (nsPIDOMWindowInner* win = doc->GetInnerWindow()) { -+ if (mozilla::dom::BrowsingContext* bc = win->GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ } -+ } -+ } -+ - if (mInnerTransformingTextRunFactory) { - transformedChild = mInnerTransformingTextRunFactory->MakeTextRun( - convertedString.BeginReading(), convertedString.Length(), &innerParams, -@@ -644,7 +658,7 @@ void MathMLTextRunFactory::RebuildTextRun( - } else { - cachedChild = newFontGroup->MakeTextRun( - convertedString.BeginReading(), convertedString.Length(), &innerParams, -- flags, nsTextFrameUtils::Flags(), aMFR); -+ flags, nsTextFrameUtils::Flags(), aMFR, userContextId); - child = cachedChild.get(); - } - if (!child) { -diff --git a/layout/generic/nsTextFrame.cpp b/layout/generic/nsTextFrame.cpp -index 6c3615f90c..448aed22e7 100644 ---- a/layout/generic/nsTextFrame.cpp -+++ b/layout/generic/nsTextFrame.cpp -@@ -2344,10 +2344,11 @@ static already_AddRefed GetHyphenTextRun(nsTextFrame* aTextFrame, - return fontGroup->MakeHyphenTextRun(dt, flags, appPerDev); - } - auto* missingFonts = aTextFrame->PresContext()->MissingFontRecorder(); -+ uint32_t userContextId = nsLayoutUtils::GetUserContextId(aTextFrame); - const NS_ConvertUTF8toUTF16 hyphenStr(hyphenateChar.AsString().AsString()); - return fontGroup->MakeTextRun(hyphenStr.BeginReading(), hyphenStr.Length(), - dt, appPerDev, flags, nsTextFrameUtils::Flags(), -- missingFonts); -+ missingFonts, userContextId); - } - - already_AddRefed BuildTextRunsScanner::BuildTextRunForFrames( -@@ -2698,6 +2699,9 @@ already_AddRefed BuildTextRunsScanner::BuildTextRunForFrames( - "We didn't cover all the characters in the text run!"); - } - -+ // Get user context ID for font spacing seed -+ uint32_t userContextId = nsLayoutUtils::GetUserContextId(firstFrame); -+ - RefPtr textRun; - gfxTextRunFactory::Parameters params = { - mDrawTarget, -@@ -2715,7 +2719,7 @@ already_AddRefed BuildTextRunsScanner::BuildTextRunForFrames( - std::move(styles), true); - } else { - textRun = fontGroup->MakeTextRun(text, transformedLength, ¶ms, flags, -- flags2, mMissingFonts); -+ flags2, mMissingFonts, userContextId); - } - } else { - const uint8_t* text = static_cast(textPtr); -@@ -2726,7 +2730,7 @@ already_AddRefed BuildTextRunsScanner::BuildTextRunForFrames( - std::move(styles), true); - } else { - textRun = fontGroup->MakeTextRun(text, transformedLength, ¶ms, flags, -- flags2, mMissingFonts); -+ flags2, mMissingFonts, userContextId); - } - } - if (!textRun) { -@@ -5585,6 +5589,7 @@ static already_AddRefed GenerateTextRunForEmphasisMarks( - - RefPtr dt = CreateReferenceDrawTarget(aFrame); - auto appUnitsPerDevUnit = aFrame->PresContext()->AppUnitsPerDevPixel(); -+ uint32_t userContextId = nsLayoutUtils::GetUserContextId(aFrame); - gfx::ShapedTextFlags flags = - nsLayoutUtils::GetTextRunOrientFlagsForStyle(aComputedStyle); - if (flags == gfx::ShapedTextFlags::TEXT_ORIENT_VERTICAL_MIXED) { -@@ -5593,7 +5598,8 @@ static already_AddRefed GenerateTextRunForEmphasisMarks( - } - return aFontGroup->MakeTextRun(string.get(), string.Length(), dt, - appUnitsPerDevUnit, flags, -- nsTextFrameUtils::Flags(), nullptr); -+ nsTextFrameUtils::Flags(), nullptr, -+ userContextId); - } - - static nsRubyFrame* FindFurthestInlineRubyAncestor(nsTextFrame* aFrame) { -diff --git a/layout/generic/nsTextRunTransformations.cpp b/layout/generic/nsTextRunTransformations.cpp -index f1f0c2ef92..59b59112f1 100644 ---- a/layout/generic/nsTextRunTransformations.cpp -+++ b/layout/generic/nsTextRunTransformations.cpp -@@ -908,6 +908,10 @@ void nsCaseTransformTextRunFactory::RebuildTextRun( - RefPtr cachedChild; - gfxTextRun* child; - -+ // Text transformation contexts don't have direct access to document/frame context, -+ // so we cannot extract private browsing ID. Use 0 (default context). -+ uint32_t privateBrowsingId = 0; -+ - if (mInnerTransformingTextRunFactory) { - transformedChild = mInnerTransformingTextRunFactory->MakeTextRun( - convertedString.BeginReading(), convertedString.Length(), &innerParams, -@@ -917,7 +921,7 @@ void nsCaseTransformTextRunFactory::RebuildTextRun( - } else { - cachedChild = fontGroup->MakeTextRun( - convertedString.BeginReading(), convertedString.Length(), &innerParams, -- flags, nsTextFrameUtils::Flags(), aMFR); -+ flags, nsTextFrameUtils::Flags(), aMFR, privateBrowsingId); - child = cachedChild.get(); - } - if (!child) { -diff --git a/layout/mathml/nsMathMLChar.cpp b/layout/mathml/nsMathMLChar.cpp -index befaf7fed8..2f818fd962 100644 ---- a/layout/mathml/nsMathMLChar.cpp -+++ b/layout/mathml/nsMathMLChar.cpp -@@ -19,6 +19,8 @@ - #include "mozilla/StaticPrefs_mathml.h" - #include "mozilla/UniquePtr.h" - #include "mozilla/dom/Document.h" -+#include "mozilla/dom/BrowsingContext.h" -+#include "nsPIDOMWindow.h" - #include "mozilla/gfx/2D.h" - #include "mozilla/intl/UnicodeScriptCodes.h" - #include "nsCSSRendering.h" -@@ -271,7 +273,7 @@ already_AddRefed nsUnicodeTable::MakeTextRun( - "nsUnicodeTable can only access glyphs by code point"); - return aFontGroup->MakeTextRun(&aGlyph.code, 1, aDrawTarget, - aAppUnitsPerDevPixel, gfx::ShapedTextFlags(), -- nsTextFrameUtils::Flags(), nullptr); -+ nsTextFrameUtils::Flags(), nullptr, 0); // TODO: Extract private browsing ID - } - - // An instance of nsOpenTypeTable is associated with one gfxFontEntry that -@@ -343,7 +345,7 @@ void nsOpenTypeTable::UpdateCache(DrawTarget* aDrawTarget, - if (mCharCache != aChar) { - RefPtr textRun = - aFontGroup->MakeTextRun(&aChar, 1, aDrawTarget, aAppUnitsPerDevPixel, -- mFlags, nsTextFrameUtils::Flags(), nullptr); -+ mFlags, nsTextFrameUtils::Flags(), nullptr, 0); - const gfxTextRun::CompressedGlyph& data = textRun->GetCharacterGlyphs()[0]; - if (data.IsSimpleGlyph()) { - mGlyphID = data.GetSimpleGlyph(); -@@ -1216,10 +1218,19 @@ nsresult nsMathMLChar::StretchInternal( - flags |= gfx::ShapedTextFlags::TEXT_IS_RTL; - } - -+ uint32_t userContextId = 0; -+ if (mozilla::dom::Document* doc = presContext->Document()) { -+ if (nsPIDOMWindowInner* win = doc->GetInnerWindow()) { -+ if (mozilla::dom::BrowsingContext* bc = win->GetBrowsingContext()) { -+ userContextId = bc->OriginAttributesRef().mUserContextId; -+ } -+ } -+ } -+ - mGlyphs[0] = fm->GetThebesFontGroup()->MakeTextRun( - static_cast(mData.get()), len, aDrawTarget, - presContext->AppUnitsPerDevPixel(), flags, nsTextFrameUtils::Flags(), -- presContext->MissingFontRecorder()); -+ presContext->MissingFontRecorder(), userContextId); - aDesiredStretchSize = MeasureTextRun(aDrawTarget, mGlyphs[0].get()); - - bool maxWidth = aStretchFlags.contains(MathMLStretchFlag::MaxWidth); diff --git a/toolkit/components/resistfingerprinting/FontVisibilityProvider.h b/toolkit/components/resistfingerprinting/FontVisibilityProvider.h index 2f71aae0f9..c29d5ad5b5 100644 --- a/toolkit/components/resistfingerprinting/FontVisibilityProvider.h diff --git a/patches/browser-init.patch b/patches/browser-init.patch index 63ef3a84b..38699d342 100644 --- a/patches/browser-init.patch +++ b/patches/browser-init.patch @@ -1,10 +1,10 @@ diff --git a/browser/base/content/browser-init.js b/browser/base/content/browser-init.js -index 1cfa4b8497..bdf8d7202e 100644 +index 0aa4baea8e..3c964b7ad5 100644 --- a/browser/base/content/browser-init.js +++ b/browser/base/content/browser-init.js -@@ -3,6 +3,16 @@ - * License, v. 2.0. If a copy of the MPL was not distributed with this - * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ +@@ -44,6 +44,16 @@ var gSerialDeviceObserver = { + }, + }; +const { AddonManager } = ChromeUtils.importESModule( + "resource://gre/modules/AddonManager.sys.mjs", @@ -19,7 +19,7 @@ index 1cfa4b8497..bdf8d7202e 100644 let _resolveDelayedStartup; var delayedStartupPromise = new Promise(resolve => { _resolveDelayedStartup = resolve; -@@ -72,7 +82,7 @@ var gBrowserInit = { +@@ -157,7 +167,7 @@ var gBrowserInit = { updateBookmarkToolbarVisibility(); // Set a sane starting width/height for all resolutions on new profiles. @@ -28,7 +28,7 @@ index 1cfa4b8497..bdf8d7202e 100644 // When the fingerprinting resistance is enabled, making sure that we don't // have a maximum window to interfere with generating rounded window dimensions. document.documentElement.setAttribute("sizemode", "normal"); -@@ -310,6 +320,22 @@ var gBrowserInit = { +@@ -385,6 +395,22 @@ var gBrowserInit = { // Update UI if browser is under remote control. gRemoteControl.updateVisualCue(); @@ -51,7 +51,7 @@ index 1cfa4b8497..bdf8d7202e 100644 // If we are given a tab to swap in, take care of it before first paint to // avoid an about:blank flash. let tabToAdopt = this.getTabToAdopt(); -@@ -345,6 +371,33 @@ var gBrowserInit = { +@@ -448,6 +474,33 @@ var gBrowserInit = { } } @@ -85,8 +85,8 @@ index 1cfa4b8497..bdf8d7202e 100644 // Wait until chrome is painted before executing code not critical to making the window visible this._boundDelayedStartup = this._delayedStartup.bind(this); window.addEventListener("MozAfterPaint", this._boundDelayedStartup); -@@ -366,9 +417,177 @@ var gBrowserInit = { - )?.removeAttribute("key"); +@@ -475,9 +528,177 @@ var gBrowserInit = { + } } + // Set default size @@ -132,8 +132,8 @@ index 1cfa4b8497..bdf8d7202e 100644 + browser.style.setProperty('box-sizing', 'content-box'); + + // Hijack the inner window size -+ let innerWidth = ChromeUtils.camouGetInt("window.innerWidth") || ChromeUtils.camouGetInt("document.body.clientWidth"); -+ let innerHeight = ChromeUtils.camouGetInt("window.innerHeight") || ChromeUtils.camouGetInt("document.body.clientHeight"); ++ let innerWidth = ChromeUtils.camouGetInt("window.innerWidth"); ++ let innerHeight = ChromeUtils.camouGetInt("window.innerHeight"); + + if (innerWidth || innerHeight) { + let win_inner_style = document.createElement('style'); diff --git a/patches/fingerprint-injection.patch b/patches/fingerprint-injection.patch index 406a70dad..a4cdcc74c 100644 --- a/patches/fingerprint-injection.patch +++ b/patches/fingerprint-injection.patch @@ -1,8 +1,8 @@ diff --git a/browser/app/moz.build b/browser/app/moz.build -index a23bb7812a..58942ccb13 100644 +index 28a9650680..9251ac4547 100644 --- a/browser/app/moz.build +++ b/browser/app/moz.build -@@ -182,6 +182,9 @@ for icon in ("firefox", "document", "newwindow", "newtab", "pbmode", "document_p +@@ -186,6 +186,9 @@ for icon in ("firefox", "document", "newwindow", "newtab", "pbmode", "document_p icon, ) @@ -12,43 +12,11 @@ index a23bb7812a..58942ccb13 100644 if CONFIG["MOZ_ASAN"] or CONFIG["MOZ_DEBUG"]: WINCONSOLE = True else: -diff --git a/dom/base/Element.cpp b/dom/base/Element.cpp -index 101590d8c8..f004930b20 100644 ---- a/dom/base/Element.cpp -+++ b/dom/base/Element.cpp -@@ -12,6 +12,8 @@ - - #include "mozilla/dom/Element.h" - -+#include "MaskConfig.hpp" -+ - #include - - #include -@@ -1006,6 +1008,18 @@ nsRect Element::GetClientAreaRect() { - Document* doc = OwnerDoc(); - nsPresContext* presContext = doc->GetPresContext(); - -+ if (doc->GetBodyElement() == this) { -+ if (auto conf = MaskConfig::GetInt32Rect( -+ "document.body.clientLeft", "document.body.clientTop", -+ "document.body.clientWidth", "document.body.clientHeight")) { -+ if (conf.has_value()) { -+ auto values = conf.value(); -+ return nsRect(values[0] * 60, values[1] * 60, values[2] * 60, -+ values[3] * 60); -+ } -+ } -+ } -+ - // We can avoid a layout flush if this is the scrolling element of the - // document, we have overlay scrollbars, and we aren't embedded in another - // document diff --git a/dom/base/moz.build b/dom/base/moz.build -index cd9090cda3..37a053579e 100644 +index e25ac44451..13839534ad 100644 --- a/dom/base/moz.build +++ b/dom/base/moz.build -@@ -634,3 +634,6 @@ GeneratedFile( +@@ -647,3 +647,6 @@ GeneratedFile( "/servo/components/style/properties/counted_unknown_properties.py", ], ) @@ -57,10 +25,10 @@ index cd9090cda3..37a053579e 100644 +LOCAL_INCLUDES += ["/camoucfg"] \ No newline at end of file diff --git a/dom/base/nsGlobalWindowInner.cpp b/dom/base/nsGlobalWindowInner.cpp -index 330a707789..241f87780b 100644 +index 4bc6688f9a..693779ca08 100644 --- a/dom/base/nsGlobalWindowInner.cpp +++ b/dom/base/nsGlobalWindowInner.cpp -@@ -5,6 +5,7 @@ +@@ -3,6 +3,7 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ #include "nsGlobalWindowInner.h" @@ -68,7 +36,7 @@ index 330a707789..241f87780b 100644 #include #include -@@ -3439,6 +3440,8 @@ void nsGlobalWindowInner::SetName(const nsAString& aName, +@@ -3704,6 +3705,8 @@ void nsGlobalWindowInner::SetName(const nsAString& aName, } double nsGlobalWindowInner::GetInnerWidth(ErrorResult& aError) { @@ -77,7 +45,7 @@ index 330a707789..241f87780b 100644 FORWARD_TO_OUTER_OR_THROW(GetInnerWidthOuter, (aError), aError, 0); } -@@ -3450,6 +3453,8 @@ nsresult nsGlobalWindowInner::GetInnerWidth(double* aWidth) { +@@ -3715,6 +3718,8 @@ nsresult nsGlobalWindowInner::GetInnerWidth(double* aWidth) { } double nsGlobalWindowInner::GetInnerHeight(ErrorResult& aError) { @@ -86,7 +54,7 @@ index 330a707789..241f87780b 100644 // We ignore aCallerType; we only have that argument because some other things // called by GetReplaceableWindowCoord need it. If this ever changes, fix // nsresult nsGlobalWindowInner::GetInnerHeight(double* aInnerWidth) -@@ -3466,12 +3471,18 @@ nsresult nsGlobalWindowInner::GetInnerHeight(double* aHeight) { +@@ -3731,6 +3736,12 @@ nsresult nsGlobalWindowInner::GetInnerHeight(double* aHeight) { int32_t nsGlobalWindowInner::GetOuterWidth(CallerType aCallerType, ErrorResult& aError) { @@ -99,13 +67,7 @@ index 330a707789..241f87780b 100644 FORWARD_TO_OUTER_OR_THROW(GetOuterWidthOuter, (aCallerType, aError), aError, 0); } - - int32_t nsGlobalWindowInner::GetOuterHeight(CallerType aCallerType, - ErrorResult& aError) { - FORWARD_TO_OUTER_OR_THROW(GetOuterHeightOuter, (aCallerType, aError), aError, - 0); - } -@@ -3486,11 +3497,13 @@ double nsGlobalWindowInner::ScreenEdgeSlopY() const { +@@ -3751,11 +3762,13 @@ double nsGlobalWindowInner::ScreenEdgeSlopY() const { int32_t nsGlobalWindowInner::GetScreenX(CallerType aCallerType, ErrorResult& aError) { @@ -119,7 +81,7 @@ index 330a707789..241f87780b 100644 FORWARD_TO_OUTER_OR_THROW(GetScreenYOuter, (aCallerType, aError), aError, 0); } -@@ -3524,6 +3537,8 @@ static nsPresContext* GetPresContextForRatio(Document* aDoc) { +@@ -3789,6 +3802,8 @@ static nsPresContext* GetPresContextForRatio(Document* aDoc) { double nsGlobalWindowInner::GetDevicePixelRatio(CallerType aCallerType, ErrorResult& aError) { ENSURE_ACTIVE_DOCUMENT(aError, 0.0); @@ -128,80 +90,20 @@ index 330a707789..241f87780b 100644 RefPtr presContext = GetPresContextForRatio(mDoc); if (NS_WARN_IF(!presContext)) { -@@ -3594,26 +3609,38 @@ already_AddRefed nsGlobalWindowInner::MatchMedia( - } - - int32_t nsGlobalWindowInner::GetScrollMinX(ErrorResult& aError) { -+ if (auto value = MaskConfig::GetInt32("window.scrollMinX")) -+ return value.value(); - FORWARD_TO_OUTER_OR_THROW(GetScrollBoundaryOuter, (eSideLeft), aError, 0); - } - - int32_t nsGlobalWindowInner::GetScrollMinY(ErrorResult& aError) { -+ if (auto value = MaskConfig::GetInt32("window.scrollMinY")) -+ return value.value(); - FORWARD_TO_OUTER_OR_THROW(GetScrollBoundaryOuter, (eSideTop), aError, 0); - } - - int32_t nsGlobalWindowInner::GetScrollMaxX(ErrorResult& aError) { -+ if (auto value = MaskConfig::GetInt32("window.scrollMaxX")) -+ return value.value(); - FORWARD_TO_OUTER_OR_THROW(GetScrollBoundaryOuter, (eSideRight), aError, 0); - } - - int32_t nsGlobalWindowInner::GetScrollMaxY(ErrorResult& aError) { -+ if (auto value = MaskConfig::GetInt32("window.scrollMaxY")) -+ return value.value(); - FORWARD_TO_OUTER_OR_THROW(GetScrollBoundaryOuter, (eSideBottom), aError, 0); - } - - double nsGlobalWindowInner::GetScrollX(ErrorResult& aError) { -+ if (auto value = MaskConfig::GetDouble("screen.pageXOffset")) -+ return value.value(); - FORWARD_TO_OUTER_OR_THROW(GetScrollXOuter, (), aError, 0); - } - - double nsGlobalWindowInner::GetScrollY(ErrorResult& aError) { -+ if (auto value = MaskConfig::GetDouble("screen.pageYOffset")) -+ return value.value(); - FORWARD_TO_OUTER_OR_THROW(GetScrollYOuter, (), aError, 0); - } - -diff --git a/dom/base/nsHistory.cpp b/dom/base/nsHistory.cpp -index e2fd8e6389..e5ddf4e08c 100644 ---- a/dom/base/nsHistory.cpp -+++ b/dom/base/nsHistory.cpp -@@ -5,6 +5,7 @@ - * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ - - #include "nsHistory.h" -+#include "MaskConfig.hpp" - - #include "jsapi.h" - #include "mozilla/RefPtr.h" -@@ -64,6 +65,8 @@ JSObject* nsHistory::WrapObject(JSContext* aCx, - } - - uint32_t nsHistory::GetLength(ErrorResult& aRv) const { -+ if (auto value = MaskConfig::GetUint32("window.history.length")) -+ return value.value(); - nsCOMPtr win(do_QueryReferent(mInnerWindow)); - if (!win || !win->HasActiveDocument()) { - aRv.Throw(NS_ERROR_DOM_SECURITY_ERR); diff --git a/dom/base/nsScreen.cpp b/dom/base/nsScreen.cpp -index 306ab35772..7f7728a6cc 100644 +index de9a6a4d11..78849001c5 100644 --- a/dom/base/nsScreen.cpp +++ b/dom/base/nsScreen.cpp -@@ -6,6 +6,8 @@ +@@ -4,6 +4,8 @@ #include "nsScreen.h" +#include "MaskConfig.hpp" + #include "mozilla/GeckoBindings.h" + #include "mozilla/dom/BrowsingContextBinding.h" #include "mozilla/dom/Document.h" - #include "mozilla/dom/DocumentInlines.h" -@@ -40,6 +42,10 @@ NS_IMPL_CYCLE_COLLECTION_INHERITED(nsScreen, DOMEventTargetHelper, +@@ -44,6 +46,10 @@ NS_IMPL_CYCLE_COLLECTION_INHERITED(nsScreen, DOMEventTargetHelper, mScreenOrientation) int32_t nsScreen::PixelDepth() { @@ -212,7 +114,7 @@ index 306ab35772..7f7728a6cc 100644 // Return 24 to prevent fingerprinting. if (ShouldResistFingerprinting(RFPTarget::ScreenPixelDepth)) { return 24; -@@ -89,6 +95,12 @@ CSSIntRect nsScreen::GetRect() { +@@ -99,6 +105,12 @@ CSSIntRect nsScreen::GetRect() { } CSSIntRect nsScreen::GetAvailRect() { @@ -225,79 +127,19 @@ index 306ab35772..7f7728a6cc 100644 // Return window inner rect to prevent fingerprinting. if (ShouldResistFingerprinting(RFPTarget::ScreenAvailRect)) { return GetTopWindowInnerRectForRFP(); -diff --git a/dom/battery/BatteryManager.cpp b/dom/battery/BatteryManager.cpp -index 6322093fd9..264cc91bd1 100644 ---- a/dom/battery/BatteryManager.cpp -+++ b/dom/battery/BatteryManager.cpp -@@ -9,6 +9,7 @@ - #include - #include - -+#include "MaskConfig.hpp" - #include "Constants.h" - #include "mozilla/DOMEventTargetHelper.h" - #include "mozilla/Hal.h" -@@ -53,6 +54,9 @@ JSObject* BatteryManager::WrapObject(JSContext* aCx, - - bool BatteryManager::Charging() const { - MOZ_ASSERT(NS_IsMainThread()); -+ if (auto value = MaskConfig::GetBool("battery:charging"); value.has_value()) -+ return value.value(); -+ - // For testing, unable to report the battery status information - if (Preferences::GetBool("dom.battery.test.default", false)) { - return true; -@@ -69,6 +73,8 @@ bool BatteryManager::Charging() const { - - double BatteryManager::DischargingTime() const { - MOZ_ASSERT(NS_IsMainThread()); -+ if (auto value = MaskConfig::GetDouble("battery:dischargingTime")) -+ return value.value(); - // For testing, unable to report the battery status information - if (Preferences::GetBool("dom.battery.test.default", false)) { - return std::numeric_limits::infinity(); -@@ -86,6 +92,8 @@ double BatteryManager::DischargingTime() const { - - double BatteryManager::ChargingTime() const { - MOZ_ASSERT(NS_IsMainThread()); -+ if (auto value = MaskConfig::GetDouble("battery:chargingTime")) -+ return value.value(); - // For testing, unable to report the battery status information - if (Preferences::GetBool("dom.battery.test.default", false)) { - return 0.0; -@@ -103,6 +111,7 @@ double BatteryManager::ChargingTime() const { - - double BatteryManager::Level() const { - MOZ_ASSERT(NS_IsMainThread()); -+ if (auto value = MaskConfig::GetDouble("battery:level")) return value.value(); - // For testing, unable to report the battery status information - if (Preferences::GetBool("dom.battery.test.default")) { - return 1.0; -diff --git a/dom/battery/moz.build b/dom/battery/moz.build -index 3a90c93c01..91d673039b 100644 ---- a/dom/battery/moz.build -+++ b/dom/battery/moz.build -@@ -21,3 +21,6 @@ FINAL_LIBRARY = "xul" - - MOCHITEST_CHROME_MANIFESTS += ["test/chrome.toml"] - MOCHITEST_MANIFESTS += ["test/mochitest.toml"] -+ -+# DOM Mask -+LOCAL_INCLUDES += ["/camoucfg"] -\ No newline at end of file diff --git a/dom/workers/WorkerNavigator.cpp b/dom/workers/WorkerNavigator.cpp -index 622724b529..c9a65ff013 100644 +index e9ebd3e01b..c6f64371f2 100644 --- a/dom/workers/WorkerNavigator.cpp +++ b/dom/workers/WorkerNavigator.cpp -@@ -5,6 +5,7 @@ +@@ -3,6 +3,7 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ #include "mozilla/dom/WorkerNavigator.h" +#include "MaskConfig.hpp" - #include - -@@ -103,6 +104,9 @@ JSObject* WorkerNavigator::WrapObject(JSContext* aCx, + #include "ErrorList.h" + #include "MainThreadUtils.h" +@@ -106,6 +107,9 @@ JSObject* WorkerNavigator::WrapObject(JSContext* aCx, } bool WorkerNavigator::GlobalPrivacyControl() const { @@ -307,7 +149,7 @@ index 622724b529..c9a65ff013 100644 bool gpcStatus = StaticPrefs::privacy_globalprivacycontrol_enabled(); if (!gpcStatus) { JSObject* jso = GetWrapper(); -@@ -125,6 +129,8 @@ void WorkerNavigator::SetLanguages(const nsTArray& aLanguages) { +@@ -128,6 +132,8 @@ void WorkerNavigator::SetLanguages(const nsTArray& aLanguages) { void WorkerNavigator::GetAppVersion(nsString& aAppVersion, CallerType aCallerType, ErrorResult& aRv) const { @@ -316,7 +158,7 @@ index 622724b529..c9a65ff013 100644 WorkerPrivate* workerPrivate = GetCurrentThreadWorkerPrivate(); MOZ_ASSERT(workerPrivate); -@@ -147,6 +153,8 @@ void WorkerNavigator::GetAppVersion(nsString& aAppVersion, +@@ -150,6 +156,8 @@ void WorkerNavigator::GetAppVersion(nsString& aAppVersion, void WorkerNavigator::GetPlatform(nsString& aPlatform, CallerType aCallerType, ErrorResult& aRv) const { @@ -325,7 +167,7 @@ index 622724b529..c9a65ff013 100644 WorkerPrivate* workerPrivate = GetCurrentThreadWorkerPrivate(); MOZ_ASSERT(workerPrivate); -@@ -207,6 +215,8 @@ class GetUserAgentRunnable final : public WorkerMainThreadRunnable { +@@ -210,6 +218,8 @@ class GetUserAgentRunnable final : public WorkerMainThreadRunnable { void WorkerNavigator::GetUserAgent(nsString& aUserAgent, CallerType aCallerType, ErrorResult& aRv) const { @@ -334,7 +176,7 @@ index 622724b529..c9a65ff013 100644 WorkerPrivate* workerPrivate = GetCurrentThreadWorkerPrivate(); MOZ_ASSERT(workerPrivate); -@@ -218,6 +228,8 @@ void WorkerNavigator::GetUserAgent(nsString& aUserAgent, CallerType aCallerType, +@@ -221,6 +231,8 @@ void WorkerNavigator::GetUserAgent(nsString& aUserAgent, CallerType aCallerType, } uint64_t WorkerNavigator::HardwareConcurrency() const { @@ -344,10 +186,10 @@ index 622724b529..c9a65ff013 100644 MOZ_ASSERT(rts); diff --git a/dom/workers/moz.build b/dom/workers/moz.build -index 2f2948a729..b8ad10403f 100644 +index f5ec1301a7..294a1c7a03 100644 --- a/dom/workers/moz.build +++ b/dom/workers/moz.build -@@ -114,3 +114,6 @@ MARIONETTE_MANIFESTS += ["test/marionette/manifest.toml"] +@@ -110,3 +110,6 @@ MARIONETTE_MANIFESTS += ["test/marionette/manifest.toml"] XPCSHELL_TESTS_MANIFESTS += ["test/xpcshell/xpcshell.toml"] BROWSER_CHROME_MANIFESTS += ["test/browser.toml"] diff --git a/patches/librewolf/1550_1549.diff.opt b/patches/librewolf/1550_1549.diff.opt deleted file mode 100644 index 2827e6744..000000000 --- a/patches/librewolf/1550_1549.diff.opt +++ /dev/null @@ -1,33 +0,0 @@ -=== modified file 'debian/patches/unity-menubar.patch' ---- debian/patches/unity-menubar.patch 2022-04-21 08:09:23 +0000 -+++ debian/patches/unity-menubar.patch 2022-05-18 06:18:20 +0000 -@@ -1068,7 +1068,7 @@ - + - + OnOpen(); - + --+ mOpenDelayTimer = do_CreateInstance(NS_TIMER_CONTRACTID); -++ mOpenDelayTimer = NS_NewTimer(); - + if (!mOpenDelayTimer) { - + return; - + } -@@ -5118,8 +5118,8 @@ - --- a/widget/gtk/components.conf - +++ b/widget/gtk/components.conf - @@ -76,6 +76,14 @@ Classes = [ -- 'headers': ['/widget/gtk/nsApplicationChooser.h'], -- 'processes': ProcessSelector.MAIN_PROCESS_ONLY, -+ 'headers': ['/widget/gtk/nsUserIdleServiceGTK.h'], -+ 'constructor': 'nsUserIdleServiceGTK::GetInstance', - }, - + { - + 'cid': '{0b3fe5aa-bc72-4303-85ae-76365df1251d}', -@@ -5131,7 +5131,7 @@ - + }, - ] - -- if defined('MOZ_X11'): -+ if defined('NS_PRINTING'): - --- a/xpfe/appshell/AppWindow.cpp - +++ b/xpfe/appshell/AppWindow.cpp - @@ -80,7 +80,7 @@ - diff --git a/patches/librewolf/arm.patch.opt b/patches/librewolf/arm.patch.opt deleted file mode 100644 index 0e0d64861..000000000 --- a/patches/librewolf/arm.patch.opt +++ /dev/null @@ -1,12 +0,0 @@ -diff --git a/js/src/wasm/WasmSignalHandlers.cpp b/js/src/wasm/WasmSignalHandlers.cpp -index f8977a6..34f52fc 100644 ---- a/js/src/wasm/WasmSignalHandlers.cpp -+++ b/js/src/wasm/WasmSignalHandlers.cpp -@@ -243,7 +243,7 @@ using mozilla::DebugOnly; - // If you run into compile problems on a tier-3 platform, you can disable the - // emulation here. - --#if defined(__linux__) && defined(__arm__) -+#if 0 && defined(__linux__) && defined(__arm__) - # define WASM_EMULATE_ARM_UNALIGNED_FP_ACCESS - #endif diff --git a/patches/librewolf/bootstrap-without-vcs.patch.opt b/patches/librewolf/bootstrap-without-vcs.patch.opt deleted file mode 100644 index b4295b80c..000000000 --- a/patches/librewolf/bootstrap-without-vcs.patch.opt +++ /dev/null @@ -1,179 +0,0 @@ -# LibreWolf bootstrap-without-vcs.patch -# -# Author: Malte Jürgens -# Description: Allow mach bootstrapping without a VCS checkout -# Last Updated: 2023-03-15 -# License: MPL 2.0 -# -# This patch allows you to use `./mach bootstrap` without a VCS checkout. -# You can use that command to bootstrap a Firefox build environment. -# This patch works by adding a stub `LocalRepository`, which suprisingly -# is enough to make the bootstrapping process work. This may break other -# things in mach, but we don't use those. ---- a/python/mozboot/mozboot/bootstrap.py -+++ b/python/mozboot/mozboot/bootstrap.py -@@ -628,10 +628,7 @@ def current_firefox_checkout(env, hg: Optional[Path] = None): - break - path = path.parent - -- raise UserError( -- "Could not identify the root directory of your checkout! " -- "Are you running `mach bootstrap` in an hg or git clone?" -- ) -+ return ("local", Path.cwd()) - - - def update_git_tools(git: Optional[Path], root_state_dir: Path): ---- a/python/mozversioncontrol/mozversioncontrol/__init__.py -+++ b/python/mozversioncontrol/mozversioncontrol/__init__.py -@@ -744,7 +744,30 @@ class GitRepository(Repository): - self._run("config", name, value) - - -+class LocalRepository(Repository): -+ -+ def __init__(self, path): -+ super(LocalRepository, self).__init__(path, tool="true") -+ -+ @property -+ def head_ref(self): -+ return "" -+ -+ def get_outgoing_files(self): -+ return [] -+ -+ def get_changed_files(self): -+ return [] -+ -+ def get_tracked_files_finder(self): -+ files = [os.path.relpath(os.path.join(dp, f), self.path).replace("\\","/") for dp, dn, fn in os.walk(self.path) for f in fn] -+ files.sort() -+ return FileListFinder(files) -+ -+ -+ -+ - def get_repository_object( - path: Optional[Union[str, Path]], hg="hg", git="git", src="src" - ): - """Get a repository object for the repository at `path`. -@@ -757,7 +780,7 @@ def get_repository_object(path: Optional[Union[str, Path]], hg="hg", git="git"): - elif (path / ".git").exists(): - return GitRepository(path, git=git) - else: -- raise InvalidRepoPath(f"Unknown VCS, or not a source checkout: {path}") -+ return LocalRepository(path) - - - def get_repository_from_build_config(config): -@@ -781,6 +804,8 @@ def get_repository_from_build_config(config): - return HgRepository(Path(config.topsrcdir), hg=config.substs["HG"]) - elif flavor == "git": - return GitRepository(Path(config.topsrcdir), git=config.substs["GIT"]) -+ elif flavor == "local": -+ return LocalRepository(config.topsrcdir) - elif flavor == "src": - return SrcRepository(Path(config.topsrcdir), src=config.substs["SRC"]) - else: ---- a/third_party/python/mozilla_repo_urls/mozilla_repo_urls/parser.py -+++ b/third_party/python/mozilla_repo_urls/mozilla_repo_urls/parser.py -@@ -9,22 +9,7 @@ for i, platform in enumerate(ADDITIONAL_PLATFORMS): - giturlparse.platforms.PLATFORMS.insert(i, platform) - - --_SUPPORTED_PLATFORMS = ("hgmo", "github") -- -- --SUPPORTED_HOSTS = tuple( -- sorted( -- [ -- host -- for domains in [ -- platform[1].DOMAINS -- for platform in giturlparse.platforms.PLATFORMS -- if platform[0] in _SUPPORTED_PLATFORMS -- ] -- for host in domains -- ] -- ) --) -+SUPPORTED_HOSTS = ("hg.mozilla.org", "github.com", "gitlab.com") - - - def parse(url_string): ---- a/third_party/python/taskcluster_taskgraph/taskgraph/util/vcs.py -+++ b/third_party/python/taskcluster_taskgraph/taskgraph/util/vcs.py -@@ -495,6 +495,64 @@ class GitRepository(Repository): - raise - - -+class LocalRepository(Repository): -+ @property -+ def tool(self): -+ return "true" -+ -+ @property -+ def head_rev(self) -> str: -+ return "" -+ -+ @property -+ def base_rev(self): -+ return "" -+ -+ @property -+ def branch(self): -+ return "" -+ -+ @property -+ def all_remote_names(self): -+ return "" -+ -+ @property -+ def default_remote_name(self): -+ return "" -+ -+ @property -+ def remote_name(self): -+ return "" -+ -+ @property -+ def default_branch(self): -+ return "" -+ -+ def get_url(self, remote=None): -+ return "" -+ -+ def get_commit_message(self, revision=None): -+ raise Exception("Unimplemented") -+ -+ def get_changed_files(self, diff_filter, mode="unstaged", rev=None, base_rev=None): -+ raise Exception("Unimplemented") -+ -+ def get_outgoing_files(self, diff_filter, upstream): -+ raise Exception("Unimplemented") -+ -+ def working_directory_clean(self, untracked=False, ignored=False): -+ raise Exception("Unimplemented") -+ -+ def update(self, ref): -+ raise Exception("Unimplemented") -+ -+ def find_latest_common_revision(self, base_ref_or_rev, head_rev): -+ raise Exception("Unimplemented") -+ -+ def does_revision_exist_locally(self, revision): -+ raise Exception("Unimplemented") -+ -+ - def get_repository(path): - """Get a repository object for the repository at `path`. - If `path` is not a known VCS repository, raise an exception. -@@ -505,7 +563,7 @@ def get_repository(path): - elif os.path.exists(os.path.join(path, ".git")): - return GitRepository(path) - -- raise RuntimeError("Current directory is neither a git or hg repository") -+ return LocalRepository(path) - - - def find_hg_revision_push_info(repository, revision): diff --git a/patches/no-css-animations.patch b/patches/no-css-animations.patch index c9dfb3100..8d8848d83 100644 --- a/patches/no-css-animations.patch +++ b/patches/no-css-animations.patch @@ -1,5 +1,5 @@ diff --git a/dom/animation/AnimationEffect.cpp b/dom/animation/AnimationEffect.cpp -index e92fdc9268..5f3fa86940 100644 +index e92fdc9268..9c04f00200 100644 --- a/dom/animation/AnimationEffect.cpp +++ b/dom/animation/AnimationEffect.cpp @@ -11,6 +11,7 @@ @@ -10,7 +10,7 @@ index e92fdc9268..5f3fa86940 100644 namespace mozilla::dom { -@@ -121,10 +122,19 @@ ComputedTiming AnimationEffect::GetComputedTimingAt( +@@ -121,10 +122,20 @@ ComputedTiming AnimationEffect::GetComputedTimingAt( // Always return the same object to benefit from return-value optimization. ComputedTiming result; @@ -19,19 +19,20 @@ index e92fdc9268..5f3fa86940 100644 MOZ_ASSERT(aTiming.Duration().ref() >= zeroDuration, "Iteration duration should be positive"); - result.mDuration = aTiming.Duration().ref(); -+ // Camoufox: finite CSS animations complete instantly so Playwright doesn't -+ // wait on them. `disableInstantAnimations` restores normal animation timing. -+ if (MaskConfig::GetBool("disableInstantAnimations") || -+ result.mActiveDuration == StickyTimeDuration::Forever()) { -+ result.mDuration = aTiming.Duration().ref(); -+ } else { ++ // Camoufox: `instantAnimations` completes finite animations at once so ++ // Playwright never waits on them. Off by default: a page reads the zero ++ // duration back through getComputedTiming(). ++ if (MaskConfig::GetBool("instantAnimations") && ++ result.mActiveDuration != StickyTimeDuration::Forever()) { + result.mDuration = zeroDuration; + result.mActiveDuration = zeroDuration; ++ } else { ++ result.mDuration = aTiming.Duration().ref(); + } } MOZ_ASSERT(aTiming.Iterations() >= 0.0 && !std::isnan(aTiming.Iterations()), -@@ -137,7 +147,6 @@ ComputedTiming AnimationEffect::GetComputedTimingAt( +@@ -137,7 +148,6 @@ ComputedTiming AnimationEffect::GetComputedTimingAt( "ValidateIterationStart"); result.mIterationStart = aTiming.IterationStart(); diff --git a/patches/patch-dependencies.md b/patches/patch-dependencies.md index 323305861..1aa0d550b 100644 --- a/patches/patch-dependencies.md +++ b/patches/patch-dependencies.md @@ -1,24 +1,61 @@ # Patch Dependencies -Quick reference for which patches depend on shared infrastructure. +Quick reference for the shared infrastructure patches build on. `scripts/patch.py` +applies every `patches/**/*.patch` in order of file name, so the +`playwright/0-*` and `playwright/1-*` patches go first and the rest follow +alphabetically. The dependencies below are compile-time: a patch applies without +them, but the tree will not build. ## camoucfg (MaskConfig) -Most patches read config via `MaskConfig::GetBool()`, `MaskConfig::GetString()`, etc. from `/camoucfg`. Any patch that adds `LOCAL_INCLUDES += ["/camoucfg"]` to a `moz.build` file depends on `config.patch` being applied first (which provides the `camoucfg` directory). - -### Patches using MaskConfig - -| Patch | Config keys | What it does | -|-------|-------------|--------------| -| `media-codec-spoofing.patch` | `media:spoof_codecs` | Bypasses `PDMFactory::Supports()` checks in `MP4Decoder` and `MatroskaDecoder` so `canPlayType()`/`isTypeSupported()` don't leak system codec libraries | -| `navigator-spoofing.patch` | Various `navigator:*` keys | Per-context navigator property spoofing | -| `geolocation-spoofing.patch` | `geo:*` keys | Geolocation coordinate spoofing | -| `locale-spoofing.patch` | `locale:*` keys | Language/locale spoofing | +`additions/camoucfg/MaskConfig.hpp` reads the spoofing config (`CAMOU_CONFIG` / +`camoufox.cfg`) through `MaskConfig::GetBool()`, `GetString()`, `GetUint32()` +and friends. `scripts/copy-additions.sh` copies it into the source tree before +any patch applies. A patch that calls MaskConfig from a directory whose +`moz.build` does not already see `/camoucfg` must add +`LOCAL_INCLUDES += ["/camoucfg"]` itself. + +Config keys are declared in `settings/properties.json`; a key a patch reads must +be listed there. + +### Patches that read config + +| Patch | Config keys | +|-------|-------------| +| `audio-context-spoofing.patch` | `AudioContext:outputLatency` | +| `audio-fingerprint-manager.patch` | `audio:seed` | +| `chromeutil.patch` | `debug` | +| `fingerprint-injection.patch` | `navigator.*`, `screen.*`, `window.*` | +| `font-hijacker.patch` | `navigator.platform` | +| `font-system-fonts-css2.patch` | `navigator.platform`, `window.devicePixelRatio` | +| `force-default-pointer.patch` | `navigator.maxTouchPoints` | +| `geolocation-spoofing.patch` | `geolocation:*` | +| `global-style-sheets.patch` | `disableTheming` | +| `locale-spoofing.patch` | `locale:*`, `navigator.language` | +| `media-codec-spoofing.patch` | `media:spoof_codecs` (bypasses `PDMFactory::Supports()` in `MP4Decoder`/`MatroskaDecoder` so `canPlayType()`/`isTypeSupported()` don't leak system codec libraries) | +| `media-device-spoofing.patch` | `mediaDevices:*` | +| `navigator-spoofing.patch` | `navigator.*`, `timezone` | +| `network-patches.patch` | `headers.*`, `navigator.userAgent` | +| `no-css-animations.patch` | `instantAnimations` | +| `screen-spoofing.patch` | `screen.width`, `screen.height` | +| `system-ui-font-spoofing.patch` | `navigator.platform` | +| `timezone-spoofing.patch` | `timezone` | +| `touchscreen-fingerprint-spoofing.patch` | `navigator.maxTouchPoints` | +| `voice-spoofing.patch` | `voices:*` | +| `webgl-spoofing.patch` | `webGl:*` | +| `webrtc-ip-spoofing.patch` | `webrtc:ipv4`, `webrtc:ipv6`, `navigator.platform` | + +To regenerate the list: `grep -l 'MaskConfig::' patches/*.patch`. ## RoverfoxStorageManager -Per-context patches that use cross-process storage depend on `cross-process-storage.patch`. +Per-context values set from Playwright (audio seed, WebRTC IP, timezone, +screen, navigator, voices, ...) are kept in `RoverfoxStorageManager`, which +`anti-font-fingerprinting.patch` adds under `dom/base/`. Its cross-process +put/get IPC lives in `cross-process-storage.patch`. Any patch that uses the +storage manager needs both. ## Playwright -All patches should be applied after `0-playwright.patch` and `1-leak-fixes.patch`. +Everything else is written against a tree that already has +`playwright/0-playwright.patch` and `playwright/1-leak-fixes.patch` applied. diff --git a/patches/playwright/README.md b/patches/playwright/README.md index d3899e359..c5a3b55b8 100644 --- a/patches/playwright/README.md +++ b/patches/playwright/README.md @@ -3,4 +3,4 @@ | File | Purpose | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `0-playwright.patch` | Playwright's upstream patches. Needs to be kept up to date with [this file](https://github.com/microsoft/playwright/blob/main/browser_patches/firefox/patches/bootstrap.diff). Will branch off if upstream is out of date. | -| `1-leak-fixes.patch` | Undos certain patches from `0-playwright.patch`. | +| `1-leak-fixes.patch` | Undoes certain patches from `0-playwright.patch`. | diff --git a/patches/timezone-spoofing.patch b/patches/timezone-spoofing.patch index 9fdf1486e..8d3f3675f 100644 --- a/patches/timezone-spoofing.patch +++ b/patches/timezone-spoofing.patch @@ -134,10 +134,10 @@ index 0000000000..888303c315 + +#endif // mozilla_dom_TimezoneManager_h diff --git a/dom/base/moz.build b/dom/base/moz.build -index 743be1950f..922a79af86 100644 +index 56e0798c8b..eab5894010 100644 --- a/dom/base/moz.build +++ b/dom/base/moz.build -@@ -285,6 +285,7 @@ EXPORTS.mozilla.dom += [ +@@ -284,6 +284,7 @@ EXPORTS.mozilla.dom += [ "TimeoutBudgetManager.h", "TimeoutHandler.h", "TimeoutManager.h", @@ -145,7 +145,7 @@ index 743be1950f..922a79af86 100644 "TreeIterator.h", "TreeOrderedArray.h", "TreeOrderedArrayInlines.h", -@@ -503,6 +504,7 @@ UNIFIED_SOURCES += [ +@@ -501,6 +502,7 @@ UNIFIED_SOURCES += [ "TimeoutExecutor.cpp", "TimeoutHandler.cpp", "TimeoutManager.cpp", @@ -154,20 +154,20 @@ index 743be1950f..922a79af86 100644 "UIDirectionManager.cpp", "UserActivation.cpp", diff --git a/dom/base/nsGlobalWindowInner.cpp b/dom/base/nsGlobalWindowInner.cpp -index 6444fccd9c..c0b2671d5b 100644 +index fb9c5c69d3..d4453fda3f 100644 --- a/dom/base/nsGlobalWindowInner.cpp +++ b/dom/base/nsGlobalWindowInner.cpp -@@ -249,6 +249,8 @@ +@@ -248,6 +248,8 @@ + #include "nsICookieService.h" #include "nsID.h" #include "nsIDOMStorageManager.h" - #include "FontSpacingSeedManager.h" +#include "TimezoneManager.h" +#include "js/Date.h" #include "nsDocShell.h" #include "mozilla/OriginAttributes.h" #include "nsIDOMXULControlElement.h" -@@ -7781,6 +7783,50 @@ void nsGlobalWindowInner::SetFontSpacingSeed(uint32_t seed, ErrorResult& aRv) { - } +@@ -7763,6 +7765,50 @@ IntlUtils* nsGlobalWindowInner::GetIntlUtils(ErrorResult& aError) { + return mIntlUtils; } +void nsGlobalWindowInner::SetTimezone(const nsAString& timezone, ErrorResult& aRv) { @@ -218,17 +218,18 @@ index 6444fccd9c..c0b2671d5b 100644 MOZ_ASSERT(aSharedWorker); MOZ_ASSERT(!mSharedWorkers.Contains(aSharedWorker)); diff --git a/dom/base/nsGlobalWindowInner.h b/dom/base/nsGlobalWindowInner.h -index bb70b2b8fe..bfde19786c 100644 +index 43600d66bf..1f9f283026 100644 --- a/dom/base/nsGlobalWindowInner.h +++ b/dom/base/nsGlobalWindowInner.h -@@ -685,6 +685,7 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, +@@ -683,6 +683,8 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, - // Font spacing seed for privacy-preserving font fingerprinting - void SetFontSpacingSeed(uint32_t seed, mozilla::ErrorResult& aRv); -+ void SetTimezone(const nsAString& timezone, mozilla::ErrorResult& aRv); + mozilla::dom::IntlUtils* GetIntlUtils(mozilla::ErrorResult& aRv); ++ void SetTimezone(const nsAString& timezone, mozilla::ErrorResult& aRv); ++ void StoreSharedWorker(mozilla::dom::SharedWorker* aSharedWorker); + void ForgetSharedWorker(mozilla::dom::SharedWorker* aSharedWorker); diff --git a/dom/base/nsGlobalWindowOuter.cpp b/dom/base/nsGlobalWindowOuter.cpp index dd0124f7e9..549e0869d0 100644 --- a/dom/base/nsGlobalWindowOuter.cpp @@ -306,10 +307,10 @@ index 21d69bdfed..fa2a7e3be3 100644 MOZ_ASSERT(NS_IsMainThread()); diff --git a/dom/webidl/Window.webidl b/dom/webidl/Window.webidl -index 6448765fb8..f01e8011cc 100644 +index 162c95ba3d..b20d7c1f6f 100644 --- a/dom/webidl/Window.webidl +++ b/dom/webidl/Window.webidl -@@ -958,6 +958,12 @@ partial interface Window { +@@ -952,6 +952,12 @@ partial interface Window { undefined setSpeechVoices(DOMString voices); }; diff --git a/patches/voice-spoofing.patch b/patches/voice-spoofing.patch index 4d4a2687c..732c752c3 100644 --- a/patches/voice-spoofing.patch +++ b/patches/voice-spoofing.patch @@ -1,27 +1,29 @@ diff --git a/dom/media/webspeech/synth/moz.build b/dom/media/webspeech/synth/moz.build +index ca6ffacb21..fb661fd69e 100644 --- a/dom/media/webspeech/synth/moz.build +++ b/dom/media/webspeech/synth/moz.build -@@ -63,2 +63,5 @@ +@@ -62,3 +62,6 @@ LOCAL_INCLUDES += [ + "/dom/base", "ipc", ] + +# DOM Mask +LOCAL_INCLUDES += ['/camoucfg'] - diff --git a/dom/media/webspeech/synth/nsSynthVoiceRegistry.cpp b/dom/media/webspeech/synth/nsSynthVoiceRegistry.cpp -index e5a1353d6b..4a4a5b080b 100644 +index 3648239545..759e61837b 100644 --- a/dom/media/webspeech/synth/nsSynthVoiceRegistry.cpp +++ b/dom/media/webspeech/synth/nsSynthVoiceRegistry.cpp -@@ -26,7 +26,8 @@ +@@ -23,7 +23,9 @@ + #include "nsServiceManagerUtils.h" #include "nsSpeechTask.h" #include "nsString.h" ++#include "nsThreadUtils.h" +#include "MaskConfig.hpp" using mozilla::intl::LocaleService; - namespace mozilla::dom { - -@@ -169,6 +171,20 @@ + #undef LOG +@@ -165,6 +167,20 @@ nsSynthVoiceRegistry* nsSynthVoiceRegistry::GetInstance() { // Start up all speech synth services. NS_CreateServicesFromCategory(NS_SPEECH_SYNTH_STARTED, nullptr, NS_SPEECH_SYNTH_STARTED); @@ -42,7 +44,7 @@ index e5a1353d6b..4a4a5b080b 100644 } } -@@ -305,6 +320,21 @@ nsSynthVoiceRegistry::AddVoice(nsISpeechService* aService, +@@ -301,6 +317,21 @@ nsSynthVoiceRegistry::AddVoice(nsISpeechService* aService, return NS_ERROR_NOT_AVAILABLE; } @@ -64,7 +66,24 @@ index e5a1353d6b..4a4a5b080b 100644 return AddVoiceImpl(aService, aUri, aName, aLang, aLocalService, aQueuesUtterances); } -@@ -779,6 +796,35 @@ void nsSynthVoiceRegistry::SpeakImpl(VoiceData* aVoice, nsSpeechTask* aTask, +@@ -373,6 +404,16 @@ nsSynthVoiceRegistry::NotifyVoicesChanged() { + + NS_IMETHODIMP + nsSynthVoiceRegistry::NotifyVoicesError(const nsAString& aError) { ++ // While Camoufox manages the voice list, a host speech backend failing to ++ // start (no speech-dispatcher on a Linux host) says nothing about the ++ // spoofed voices. Forwarding it would make every page error its queued ++ // utterances, which a Windows or macOS identity never does. ++ if (MaskConfig::MVoices()) { ++ LOG(LogLevel::Debug, ("nsSynthVoiceRegistry::NotifyVoicesError ignored: " ++ "the voice list is spoofed")); ++ return NS_OK; ++ } ++ + if (XRE_IsParentProcess()) { + nsTArray ssplist; + GetAllSpeechSynthActors(ssplist); +@@ -775,6 +816,30 @@ void nsSynthVoiceRegistry::SpeakImpl(VoiceData* aVoice, nsSpeechTask* aTask, NS_ConvertUTF16toUTF8(aText).get(), NS_ConvertUTF16toUTF8(aVoice->mUri).get(), aRate, aPitch)); @@ -72,26 +91,21 @@ index e5a1353d6b..4a4a5b080b 100644 + if (auto voices = MaskConfig::MVoices()) { + for (const auto& [lang, name, uri, isDefault, isLocal] : voices.value()) { + if (NS_ConvertUTF8toUTF16(uri).Equals(aVoice->mUri)) { -+ printf_stderr("Tried to speak a fake voice: %s", -+ NS_ConvertUTF16toUTF8(aVoice->mUri).get()); ++ // A spoofed voice has no engine behind it. Speak it silently for as ++ // long as the text takes at ~150 words per minute, so start and end ++ // arrive when they would from a real voice. + aTask->Init(); -+ // If fake completion is disabled, throw an error -+ if (!MaskConfig::GetBool("voices:fakeCompletion")) { -+ aTask->DispatchError(0, 0); -+ return; -+ } -+ float charsPerSecond; -+ if (auto value = -+ MaskConfig::GetDouble("voices:fakeCompletion:charsPerSecond")) { -+ charsPerSecond = value.value(); -+ } else { -+ charsPerSecond = 12.5f; -+ } -+ // Return a fake success with a speach rate of 150wpm + aTask->DispatchStart(); -+ float fakeElapsedTime = -+ static_cast(aText.Length()) / (charsPerSecond * aRate); -+ aTask->DispatchEnd(fakeElapsedTime, aText.Length()); ++ const float elapsed = ++ static_cast(aText.Length()) / (12.5f * aRate); ++ RefPtr task = aTask; ++ const uint32_t length = aText.Length(); ++ NS_DelayedDispatchToCurrentThread( ++ NS_NewRunnableFunction("CamouFakeSpeechEnd", ++ [task, elapsed, length]() { ++ (void)task->DispatchEnd(elapsed, length); ++ }), ++ static_cast(elapsed * 1000)); + return; + } + } diff --git a/patches/webrtc-ip-spoofing.patch b/patches/webrtc-ip-spoofing.patch index bc8136145..2d27cda7a 100644 --- a/patches/webrtc-ip-spoofing.patch +++ b/patches/webrtc-ip-spoofing.patch @@ -262,10 +262,10 @@ index 0000000000..c9810a06d2 + +#endif // mozilla_dom_WebRTCIPManager_h diff --git a/dom/base/moz.build b/dom/base/moz.build -index b718f44036..52a61b0ca1 100644 +index ba674a00d9..ef7c9bdff2 100644 --- a/dom/base/moz.build +++ b/dom/base/moz.build -@@ -297,10 +297,15 @@ EXPORTS.mozilla.dom += [ +@@ -296,10 +296,15 @@ EXPORTS.mozilla.dom += [ "VideoFrameProvider.h", "ViewportMetaData.h", "VisualViewport.h", @@ -282,10 +282,10 @@ index b718f44036..52a61b0ca1 100644 # in unified builds (it includes RoverfoxStorageManager.h which can affect # alphabetically-later files like BarProps.cpp) diff --git a/dom/base/nsGlobalWindowInner.cpp b/dom/base/nsGlobalWindowInner.cpp -index 007e8f5eae..7cdaf2767c 100644 +index 740c59ed8b..4b7d3d1297 100644 --- a/dom/base/nsGlobalWindowInner.cpp +++ b/dom/base/nsGlobalWindowInner.cpp -@@ -335,6 +335,10 @@ +@@ -334,6 +334,10 @@ #include "xpcprivate.h" #include "xpcpublic.h" @@ -296,7 +296,7 @@ index 007e8f5eae..7cdaf2767c 100644 #ifdef NS_PRINTING # include "nsIPrintSettings.h" #endif -@@ -7826,6 +7830,42 @@ void nsGlobalWindowInner::SetTimezone(const nsAString& timezone, ErrorResult& aR +@@ -7810,6 +7814,42 @@ void nsGlobalWindowInner::SetTimezone(const nsAString& timezone, ErrorResult& aR } } @@ -340,11 +340,11 @@ index 007e8f5eae..7cdaf2767c 100644 MOZ_ASSERT(aSharedWorker); MOZ_ASSERT(!mSharedWorkers.Contains(aSharedWorker)); diff --git a/dom/base/nsGlobalWindowInner.h b/dom/base/nsGlobalWindowInner.h -index 17e00f408f..d709b07f7d 100644 +index 9199f08fdf..66f858aad5 100644 --- a/dom/base/nsGlobalWindowInner.h +++ b/dom/base/nsGlobalWindowInner.h -@@ -687,6 +687,10 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, - void SetFontSpacingSeed(uint32_t seed, mozilla::ErrorResult& aRv); +@@ -685,6 +685,10 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, + void SetTimezone(const nsAString& timezone, mozilla::ErrorResult& aRv); + // WebRTC IP addresses for privacy-preserving IP spoofing @@ -1117,10 +1117,10 @@ index d32e08c2b0..40a750196a 100644 +# DOM Mask +LOCAL_INCLUDES += ["/camoucfg"] diff --git a/dom/webidl/Window.webidl b/dom/webidl/Window.webidl -index 3a13d4963f..6b4c7101fa 100644 +index 2471488e16..b7833efded 100644 --- a/dom/webidl/Window.webidl +++ b/dom/webidl/Window.webidl -@@ -964,6 +964,17 @@ partial interface Window { +@@ -958,6 +958,17 @@ partial interface Window { undefined setTimezone(DOMString timezone); }; diff --git a/patches/window-setter-seal.patch b/patches/window-setter-seal.patch index fbb45c388..4ab4f34c1 100644 --- a/patches/window-setter-seal.patch +++ b/patches/window-setter-seal.patch @@ -13,10 +13,11 @@ index 6ec4dc5265..d8a5a5aac0 100644 uint32_t userContextId = 0; if (BrowsingContext* bc = win->GetBrowsingContext()) { diff --git a/dom/base/ChromeUtils.cpp b/dom/base/ChromeUtils.cpp -index 8657d3282b..d8889edb6a 100644 +index 9a04e43224..fb7f231799 100644 --- a/dom/base/ChromeUtils.cpp +++ b/dom/base/ChromeUtils.cpp -@@ -6,5 +6,9 @@ +@@ -5,6 +5,10 @@ + #include "ChromeUtils.h" #include "MaskConfig.hpp" +// Camoufox: for CamouSealFingerprintSetters below. @@ -39,7 +40,7 @@ index 8657d3282b..d8889edb6a 100644 + "setScreenDimensions", "setScreenColorDepth", + "setWebGLVendor", "setWebGLRenderer", + "setWebRTCIPv4", "setWebRTCIPv6", -+ "setFontList", "setFontSpacingSeed", ++ "setFontList", + "setAudioFingerprintSeed", "setSpeechVoices", + "setTimezone", +}; @@ -102,7 +103,7 @@ index 8657d3282b..d8889edb6a 100644 bool ChromeUtils::ShouldResistFingerprinting( GlobalObject& aGlobal, JSRFPTarget aTarget, diff --git a/dom/base/ChromeUtils.h b/dom/base/ChromeUtils.h -index e32ee77dfd..8747d75c44 100644 +index f1de113a05..0937a561f3 100644 --- a/dom/base/ChromeUtils.h +++ b/dom/base/ChromeUtils.h @@ -361,6 +361,12 @@ class ChromeUtils { @@ -132,20 +133,6 @@ index 438bad576d..5fd40e4e48 100644 uint32_t id = 0; if (BrowsingContext* bc = win->GetBrowsingContext()) { id = bc->OriginAttributesRef().mUserContextId; -diff --git a/dom/base/FontSpacingSeedManager.cpp b/dom/base/FontSpacingSeedManager.cpp -index e07de3e753..f16422badf 100644 ---- a/dom/base/FontSpacingSeedManager.cpp -+++ b/dom/base/FontSpacingSeedManager.cpp -@@ -76,6 +76,9 @@ FontSpacingSeedManager::IsFunctionEnabledForWebIDL(JSContext* aCx, JSObject* aOb - if (!win) { - return false; - } -+ // Camoufox: sealed once this window's init scripts have run, so page -+ // script never sees the setter (FrameTree -> camouSealFingerprintSetters). -+ if (win->CamouSettersSealed()) return false; - - uint32_t userContextId = 0; - if (BrowsingContext* bc = win->GetBrowsingContext()) { diff --git a/dom/base/NavigatorManager.cpp b/dom/base/NavigatorManager.cpp index 37a1713159..e53e97de25 100644 --- a/dom/base/NavigatorManager.cpp @@ -229,10 +216,10 @@ index 0119707221..511e92e1f2 100644 if (BrowsingContext* bc = win->GetBrowsingContext()) { id = bc->OriginAttributesRef().mUserContextId; diff --git a/dom/base/TimezoneManager.cpp b/dom/base/TimezoneManager.cpp -index ffbc624040..548ff14a78 100644 +index 206a027918..cd35b265ad 100644 --- a/dom/base/TimezoneManager.cpp +++ b/dom/base/TimezoneManager.cpp -@@ -56,6 +56,9 @@ TimezoneManager::IsFunctionEnabledForWebIDL(JSContext* aCx, JSObject* aObj) { +@@ -75,6 +75,9 @@ TimezoneManager::IsFunctionEnabledForWebIDL(JSContext* aCx, JSObject* aObj) { if (!win) { return false; } @@ -291,11 +278,11 @@ index 8834c93e4b..fa807ad165 100644 uint32_t userContextId = 0; if (BrowsingContext* bc = win->GetBrowsingContext()) { diff --git a/dom/base/nsGlobalWindowInner.h b/dom/base/nsGlobalWindowInner.h -index d709b07f7d..95184adcef 100644 +index 66f858aad5..65ccfc6ea5 100644 --- a/dom/base/nsGlobalWindowInner.h +++ b/dom/base/nsGlobalWindowInner.h -@@ -687,6 +687,36 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, - void SetFontSpacingSeed(uint32_t seed, mozilla::ErrorResult& aRv); +@@ -685,6 +685,36 @@ class nsGlobalWindowInner final : public mozilla::dom::EventTarget, + void SetTimezone(const nsAString& timezone, mozilla::ErrorResult& aRv); + // Camoufox: the window.setXxx() fingerprint setters are configuration API @@ -332,7 +319,7 @@ index d709b07f7d..95184adcef 100644 void SetWebRTCIPv4(const nsAString& ipv4, mozilla::ErrorResult& aRv); void SetWebRTCIPv6(const nsAString& ipv6, mozilla::ErrorResult& aRv); diff --git a/dom/chrome-webidl/ChromeUtils.webidl b/dom/chrome-webidl/ChromeUtils.webidl -index 4c15c49fef..33aa276db2 100644 +index 31c1212350..5f655f6714 100644 --- a/dom/chrome-webidl/ChromeUtils.webidl +++ b/dom/chrome-webidl/ChromeUtils.webidl @@ -938,6 +938,20 @@ partial namespace ChromeUtils { diff --git a/pythonlib/LICENSE b/pythonlib/LICENSE new file mode 100644 index 000000000..be612e85c --- /dev/null +++ b/pythonlib/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2024-2026 daijro and the Camoufox contributors + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/pythonlib/README.md b/pythonlib/README.md index 10c65726d..d66e9b2ec 100644 --- a/pythonlib/README.md +++ b/pythonlib/README.md @@ -7,7 +7,7 @@ > [!NOTE] -> All the the latest documentation is avaliable [here](https://camoufox.com/python). +> All the latest documentation is available [here](https://camoufox.com/python). --- @@ -33,18 +33,10 @@ The `geoip` parameter is optional, but heavily recommended if you are using prox Next, download the Camoufox browser: -**Windows** - ```bash camoufox fetch ``` -**MacOS & Linux** - -```bash -python3 -m camoufox fetch -``` - To uninstall, run `camoufox remove`. --- @@ -75,7 +67,7 @@ camoufox gui --- -## CLI Mananger +## CLI Manager #### Demonstration @@ -246,13 +238,12 @@ Display the Python package version, active browser version, channel, and update ```bash > camoufox version Python Packages - Camoufox v0.5.0 + Camoufox v0.5.6 fpgen v1.3.0 - Apify Fingerprints v0.10.0 - Playwright v1.57.1.dev0+g732639b35.d20251217 + Playwright v1.62.0 Browser - Active official/stable/135.0.1-beta.24 - Current browser v135.0.1-beta.24 + Active official/stable/152.0.4-beta.31 + Current browser v152.0.4-beta.31 Installed Yes Latest in official/stable? Yes Last Sync 2026-03-07 00:23 @@ -303,4 +294,4 @@ Launch a remote Playwright server. ## Usage -All of the latest stable documentation is avaliable at [camoufox.com/python](https://camoufox.com/python). +All of the latest stable documentation is available at [camoufox.com/python](https://camoufox.com/python). diff --git a/pythonlib/camoufox/__main__.py b/pythonlib/camoufox/__main__.py index f11326318..de1dd11c7 100644 --- a/pythonlib/camoufox/__main__.py +++ b/pythonlib/camoufox/__main__.py @@ -36,7 +36,6 @@ remove_version, save_config, save_repo_cache, - set_active, ) from .pkgman import ( INSTALL_DIR, @@ -681,7 +680,7 @@ def _list_installed(show_paths: bool): rprint(" └── Not configured", fg="yellow") -def _list_all(_show_paths: bool): +def _list_all(show_paths: bool): """ List all available versions from synced repos """ @@ -721,6 +720,8 @@ def _list_all(_show_paths: bool): click.secho(" (installed, active)", fg="green", bold=True, nl=False) else: click.secho(" (installed)", fg="green", nl=False) + if show_paths: + click.secho(f" -> {inst.path}", fg="bright_black", nl=False) click.echo() @@ -866,7 +867,6 @@ def packages(self): self._header("Python Packages") self._pkg("Camoufox", "camoufox") self._pkg("fpgen", "fpgen") - self._pkg("Apify Fingerprints", "apify_fingerprint_datapoints") self._pkg("Playwright", "playwright") def browser(self): @@ -952,7 +952,7 @@ def geoip(self): self._header("GeoIP") if not ALLOW_GEOIP: - # geoip2 package not installed + # maxminddb not installed self._row("Status", "Not supported (install camoufox[geoip])", style="dim") else: mmdb_path = get_mmdb_path() diff --git a/pythonlib/camoufox/__version__.py b/pythonlib/camoufox/__version__.py index cfad362ad..726abb4ac 100644 --- a/pythonlib/camoufox/__version__.py +++ b/pythonlib/camoufox/__version__.py @@ -29,9 +29,3 @@ class CONSTRAINTS: # actually break get moved. PLAYWRIGHT_BROWSER_FLOORS = (((1, 61), 'beta.30'),) - @staticmethod - def as_range() -> str: - """ - Returns the version range as a string. - """ - return f">={CONSTRAINTS.MIN_VERSION}, <{CONSTRAINTS.MAX_VERSION}" diff --git a/pythonlib/camoufox/_warnings.py b/pythonlib/camoufox/_warnings.py index 5df2e878f..b9bde9ad5 100644 --- a/pythonlib/camoufox/_warnings.py +++ b/pythonlib/camoufox/_warnings.py @@ -1,13 +1,37 @@ import inspect +import platform import warnings +from importlib.metadata import PackageNotFoundError, version from pathlib import Path -from typing import Optional +from typing import Optional, Type from camoufox.pkgman import load_yaml WARNINGS_DATA = load_yaml('warnings.yml') +def _warn_from_caller(message: str, category: Type[Warning]) -> None: + """Attribute the warning to the first frame outside this package, so it + points at the user's call rather than at camoufox internals.""" + current_module = Path(__file__).parent + frame = inspect.currentframe() + while frame: + if not Path(frame.f_code.co_filename).is_relative_to(current_module): + break + frame = frame.f_back + + if frame: + warnings.warn_explicit( + message, + category=category, + filename=frame.f_code.co_filename, + lineno=frame.f_lineno, + ) + return + + warnings.warn(message, category=category) + + class LeakWarning(RuntimeWarning): """ Raised when a the user has a setting enabled that can cause detection. @@ -23,22 +47,45 @@ def warn(warning_key: str, i_know_what_im_doing: Optional[bool] = None) -> None: return if i_know_what_im_doing is not None: warning += '\nIf this is intentional, pass `i_know_what_im_doing=True`.' + _warn_from_caller(warning, LeakWarning) - # Get caller information - current_module = Path(__file__).parent - frame = inspect.currentframe() - while frame: - if not Path(frame.f_code.co_filename).is_relative_to(current_module): - break - frame = frame.f_back - - if frame: - warnings.warn_explicit( - warning, - category=LeakWarning, - filename=frame.f_code.co_filename, - lineno=frame.f_lineno, - ) - return - warnings.warn(warning, category=LeakWarning) +def _browser_version() -> str: + from camoufox.exceptions import CamoufoxNotInstalled + from camoufox.pkgman import installed_verstr + + try: + return installed_verstr() + except CamoufoxNotInstalled: + return 'not installed' + + +class FallbackWarning(RuntimeWarning): + """ + Raised when part of an identity could not be drawn and a substitute was used. + """ + + @staticmethod + def warn(what: str, instead: str, error: Exception, identity: Optional[str] = None) -> None: + """ + Warns that `what` failed with `error` and the identity uses `instead`, + with a block of versions and the error for the user to paste into an issue. + """ + try: + camoufox_version = version('camoufox') + except PackageNotFoundError: + camoufox_version = 'source checkout' + lines = [ + f'camoufox: {camoufox_version}', + f'browser: {_browser_version()}', + f'os: {platform.platform()}', + f'python: {platform.python_version()}', + f'error: {type(error).__name__}: {error}', + ] + if identity: + lines.append(f'identity: {identity}') + report = '\n'.join(f' {line}' for line in lines) + _warn_from_caller( + WARNINGS_DATA['fallback'].format(what=what, instead=instead, report=report), + FallbackWarning, + ) diff --git a/pythonlib/camoufox/async_api.py b/pythonlib/camoufox/async_api.py index 7a60fea99..52cb49892 100644 --- a/pythonlib/camoufox/async_api.py +++ b/pythonlib/camoufox/async_api.py @@ -1,9 +1,6 @@ import asyncio -import json as _json -import urllib.request from functools import partial -from typing import Any, Dict, List, Optional, Union, overload -from urllib.parse import urlparse +from typing import Any, Dict, Optional, Tuple, Union, overload from playwright.async_api import ( Browser, @@ -16,6 +13,7 @@ from camoufox.virtdisplay import VirtualDisplay from .fingerprints import generate_context_fingerprint +from .ip import Proxy, proxy_exit_geo from .utils import ( async_attach_vd, attach_no_viewport_default, @@ -178,31 +176,9 @@ async def _launch( return await async_attach_vd(browser, virtual_display) -def _proxy_url_with_creds(proxy: Dict[str, str]) -> str: - """Builds a proxy URL string with embedded credentials.""" - parsed = urlparse(proxy.get("server", "")) - user = proxy.get("username", "") - pwd = proxy.get("password", "") - if user and pwd: - return f"{parsed.scheme}://{user}:{pwd}@{parsed.netloc}" - return proxy.get("server", "") - - -async def _resolve_proxy_geo(proxy: Dict[str, str]) -> Dict[str, Optional[str]]: - """Queries ip-api.com through the proxy for the exit IP and timezone.""" - proxy_url = _proxy_url_with_creds(proxy) - - def _fetch() -> Dict[str, Optional[str]]: - handler = urllib.request.ProxyHandler({"http": proxy_url, "https": proxy_url}) - opener = urllib.request.build_opener(handler) - try: - with opener.open("http://ip-api.com/json?fields=query,timezone", timeout=10) as resp: - data = _json.loads(resp.read()) - return {"ip": data.get("query") or None, "timezone": data.get("timezone") or None} - except Exception: - return {"ip": None, "timezone": None} - - return await asyncio.get_event_loop().run_in_executor(None, _fetch) +async def _resolve_proxy_geo(proxy: Dict[str, str]) -> Tuple[str, str]: + """The proxy's exit IP and timezone, looked up off the event loop.""" + return await asyncio.to_thread(proxy_exit_geo, Proxy(**proxy).as_string()) async def AsyncNewContext( @@ -219,27 +195,31 @@ async def AsyncNewContext( """ Creates a new browser context with a unique fingerprint identity. - Each context gets its own real fingerprint preset (navigator, screen, WebGL, fonts, etc.) - with unique seeds for audio, canvas, and font spacing noise. All values are applied + Each context gets its own identity (navigator, screen, WebGL, fonts, voices), + drawn by fpgen unless a preset is given, with its own audio noise seed. All values are applied via addInitScript so they self-destruct before page scripts can detect them. Parameters: browser: A Browser instance from AsyncNewBrowser or AsyncCamoufox. - preset: A specific fingerprint preset dict to use. If None, picks randomly. - os: Target OS for preset selection ("windows", "macos", "linux"). - ff_version: Firefox version string for UA patching. - webrtc_ip: IPv4 address to spoof for WebRTC ICE candidates. + preset: A fingerprint preset dict to use. If None, fpgen draws a new identity. + os: Target OS for the drawn identity ("windows", "macos", "linux"). + ff_version: Firefox major version to claim in the UA. Defaults to the browser's own. + webrtc_ip: IPv4 or IPv6 address to spoof for WebRTC ICE candidates. proxy: Per-context proxy (Playwright format: {"server": "...", "username": "...", "password": "..."}). + Unless webrtc_ip and timezone_id are both given, they are looked up from the + proxy's exit IP; InvalidIP is raised if that lookup fails. geolocation: Per-context geolocation ({"latitude": float, "longitude": float}). **context_kwargs: Additional Playwright new_context() options. """ + # The drawn UA carries fpgen's Firefox version, which must not disagree with + # the browser the page is actually talking to. + ff_version = ff_version or browser.version.split('.', 1)[0] + # Auto-derive WebRTC IP and timezone from proxy's exit IP when not explicitly provided if proxy and (not webrtc_ip or "timezone_id" not in context_kwargs): - geo = await _resolve_proxy_geo(proxy) - if not webrtc_ip: - webrtc_ip = geo["ip"] - if "timezone_id" not in context_kwargs and geo["timezone"]: - context_kwargs["timezone_id"] = geo["timezone"] + exit_ip, timezone = await _resolve_proxy_geo(proxy) + webrtc_ip = webrtc_ip or exit_ip + context_kwargs.setdefault("timezone_id", timezone) fp = await asyncio.get_event_loop().run_in_executor( None, diff --git a/pythonlib/camoufox/coherence.py b/pythonlib/camoufox/coherence.py index bc0f055bc..2708fac93 100644 --- a/pythonlib/camoufox/coherence.py +++ b/pythonlib/camoufox/coherence.py @@ -1,9 +1,9 @@ """Whole-identity coherence: the checks that look at more than one field. Camoufox assembles an identity from several pools -- the navigator and screen -from the fingerprint generator, the GPU from `webgl_data.db`, fonts and voices -from its own catalogues, media devices from `media-devices.json`. Each pool is -sampled on its own, so a combination that no machine has ever had can be built +from the fingerprint generator, the GPU from fpgen's WebGL records, fonts and +voices from its own catalogues, media devices from `media-devices.json`. Each +pool is sampled on its own, so a combination that no machine has ever had can be built out of individually plausible parts: an Apple M1 with 2 cores, a Mac reporting a Braswell Atom GPU, a Linux identity whose platform says armv81 while its user agent says x86_64. @@ -43,10 +43,15 @@ # Linux reports 1, or 2 under HiDPI, with GNOME fractional scaling giving the # 1.25/1.5/1.75 steps. Values outside these (1.818, 1.09, 1.36) are scraped # artefacts -- a browser zoom level folded into the ratio, not a display mode. +# +# Ascending tuples, not sets: the repair keeps the first of two equally near +# steps, and a frozenset literal iterates in a different order when compiled +# than when loaded from a .pyc -- so a set made the first launch repair an +# identity differently from every later one. PLAUSIBLE_DPR = { - 'win': frozenset({1, 1.25, 1.5, 1.75, 2, 2.5, 3}), - 'mac': frozenset({1, 2}), - 'lin': frozenset({1, 1.25, 1.5, 1.75, 2}), + 'win': (1, 1.25, 1.5, 1.75, 2, 2.5, 3), + 'mac': (1, 2), + 'lin': (1, 1.25, 1.5, 1.75, 2), } # colorDepth: Firefox reports 24, or 30 on a deep-colour display. macOS defaults @@ -78,9 +83,9 @@ # GPU strings that are not possible on macOS. Firefox on a Mac reports Apple # Silicon as "Apple M1, or similar", and Intel Macs as an Intel Iris/UHD/HD -# 4000-6000 part; ANGLE is Windows-only (Direct3D), and these two rows in -# webgl_data.db are a Braswell Atom IGP and a desktop PC card, neither of which -# shipped in any Mac. +# 4000-6000 part; ANGLE is Windows-only (Direct3D), and the other two, which +# fpgen records from macOS, are a Braswell Atom IGP and a desktop PC card, +# neither of which shipped in any Mac. _NOT_A_MAC_GPU = ('ANGLE', 'Intel(R) HD Graphics 400', 'Radeon R9 200 Series', 'llvmpipe') @@ -128,7 +133,7 @@ def _repair_apple_silicon_cores(config: Dict[str, Any], target_os: str) -> None: def gpu_fits_os(renderer: Optional[str], target_os: str) -> bool: """Whether this renderer string is one the OS can report. - Used both to check a finished identity and to filter `webgl_data.db` before + Used both to check a finished identity and to filter fpgen's GPUs before sampling, so the two can never disagree about what a Mac may claim. """ renderer = str(renderer or '') diff --git a/pythonlib/camoufox/exceptions.py b/pythonlib/camoufox/exceptions.py index 3cdbd6718..292e992e6 100644 --- a/pythonlib/camoufox/exceptions.py +++ b/pythonlib/camoufox/exceptions.py @@ -38,14 +38,6 @@ class UnsupportedOS(Exception): ... -class UnknownProperty(Exception): - """ - Raised when the property is unknown. - """ - - ... - - class InvalidPropertyType(Exception): """ Raised when the property type is invalid. @@ -62,22 +54,6 @@ class InvalidAddonPath(FileNotFoundError): ... -class InvalidDebugPort(ValueError): - """ - Raised when the debug port is invalid. - """ - - ... - - -class MissingDebugPort(ValueError): - """ - Raised when the debug port is missing. - """ - - ... - - class LocaleError(Exception): """ Raised when the locale is invalid. @@ -141,7 +117,7 @@ class UnknownLanguage(InvalidLocale): class NotInstalledGeoIPExtra(ImportError): """ - Raised when the geoip2 module is not installed. + Raised when the maxminddb module is not installed. """ ... @@ -149,7 +125,7 @@ class NotInstalledGeoIPExtra(ImportError): class NonFirefoxFingerprint(Exception): """ - Raised when a passed Browserforge fingerprint is invalid. + Raised when a passed fingerprint is not a Firefox fingerprint. """ ... diff --git a/pythonlib/camoufox/fingerprint-presets-v150.json b/pythonlib/camoufox/fingerprint-presets-v150.json index 44d1de0d1..c0557052f 100644 --- a/pythonlib/camoufox/fingerprint-presets-v150.json +++ b/pythonlib/camoufox/fingerprint-presets-v150.json @@ -2881,76 +2881,6 @@ "Zuzana:cs-CZ:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "MacIntel", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1280, - "height": 800, - "colorDepth": 30, - "availWidth": 1280, - "availHeight": 800, - "devicePixelRatio": 2 - }, - "webgl": { - "unmaskedVendor": "Intel Inc.", - "unmaskedRenderer": "Intel 945GM, or similar" - }, - "speechVoices": [ - "Alex:en-US:local", - "Alice:it-IT:local", - "Alva:sv-SE:local", - "Amelie:fr-CA:local", - "Anna:de-DE:local", - "Carmit:he-IL:local", - "Damayanti:id-ID:local", - "Daniel:en-GB:local", - "Diego:es-AR:local", - "Ellen:nl-BE:local", - "Fiona:en-scotland:local", - "Fred:en-US:local", - "Ioana:ro-RO:local", - "Joana:pt-PT:local", - "Jorge:es-ES:local", - "Juan:es-MX:local", - "Kanya:th-TH:local", - "Karen:en-AU:local", - "Kyoko:ja-JP:local", - "Laura:sk-SK:local", - "Lekha:hi-IN:local", - "Luca:it-IT:local", - "Luciana:pt-BR:local", - "Maged:ar-SA:local", - "Mariska:hu-HU:local", - "Mei-Jia:zh-TW:local", - "Melina:el-GR:local", - "Milena:ru-RU:local", - "Moira:en-IE:local", - "Monica:es-ES:local", - "Nora:nb-NO:local", - "Paulina:es-MX:local", - "Rishi:en-IN:local", - "Samantha:en-US:local", - "Sara:da-DK:local", - "Satu:fi-FI:local", - "Sin-ji:zh-HK:local", - "Tessa:en-ZA:local", - "Thomas:fr-FR:local", - "Ting-Ting:zh-CN:local", - "Veena:en-IN:local", - "Victoria:en-US:local", - "Xander:nl-NL:local", - "Yelda:tr-TR:local", - "Yuna:ko-KR:local", - "Yuri:ru-RU:local", - "Zosia:pl-PL:local", - "Zuzana:cs-CZ:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -6895,76 +6825,6 @@ "Zuzana:cs-CZ:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "MacIntel", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1920, - "height": 1080, - "colorDepth": 30, - "availWidth": 1920, - "availHeight": 984, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Intel Inc.", - "unmaskedRenderer": "Intel 945GM, or similar" - }, - "speechVoices": [ - "Alex:en-US:local", - "Alice:it-IT:local", - "Alva:sv-SE:local", - "Amelie:fr-CA:local", - "Anna:de-DE:local", - "Carmit:he-IL:local", - "Damayanti:id-ID:local", - "Daniel:en-GB:local", - "Diego:es-AR:local", - "Ellen:nl-BE:local", - "Fiona:en-scotland:local", - "Fred:en-US:local", - "Ioana:ro-RO:local", - "Joana:pt-PT:local", - "Jorge:es-ES:local", - "Juan:es-MX:local", - "Kanya:th-TH:local", - "Karen:en-AU:local", - "Kyoko:ja-JP:local", - "Laura:sk-SK:local", - "Lekha:hi-IN:local", - "Luca:it-IT:local", - "Luciana:pt-BR:local", - "Maged:ar-SA:local", - "Mariska:hu-HU:local", - "Mei-Jia:zh-TW:local", - "Melina:el-GR:local", - "Milena:ru-RU:local", - "Moira:en-IE:local", - "Monica:es-ES:local", - "Nora:nb-NO:local", - "Paulina:es-MX:local", - "Rishi:en-IN:local", - "Samantha:en-US:local", - "Sara:da-DK:local", - "Satu:fi-FI:local", - "Sin-ji:zh-HK:local", - "Tessa:en-ZA:local", - "Thomas:fr-FR:local", - "Ting-Ting:zh-CN:local", - "Veena:en-IN:local", - "Victoria:en-US:local", - "Xander:nl-NL:local", - "Yelda:tr-TR:local", - "Yuna:ko-KR:local", - "Yuri:ru-RU:local", - "Zosia:pl-PL:local", - "Zuzana:cs-CZ:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:149.0) Gecko/20100101 Firefox/149.0", @@ -8256,33 +8116,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Win32", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 728, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (AMD)", - "unmaskedRenderer": "ANGLE (AMD, Radeon R9 200 Series Direct3D11 vs_4_0 ps_4_0), or similar" - }, - "speechVoices": [ - "Microsoft Helena - Spanish (Spain):es-ES:local", - "Microsoft Laura - Spanish (Spain):es-ES:local", - "Microsoft Pablo - Spanish (Spain):es-ES:local", - "Microsoft Helena Desktop - Spanish (Spain):es-ES:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -8534,33 +8367,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Win32", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1920, - "height": 1080, - "colorDepth": 24, - "availWidth": 1920, - "availHeight": 1040, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Intel)", - "unmaskedRenderer": "ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_4_1 ps_4_1), or similar" - }, - "speechVoices": [ - "Microsoft Helena - Spanish (Spain):es-ES:local", - "Microsoft Laura - Spanish (Spain):es-ES:local", - "Microsoft Pablo - Spanish (Spain):es-ES:local", - "Microsoft Helena Desktop - Spanish (Spain):es-ES:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -9588,32 +9394,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Win32", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1299, - "availHeight": 768, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Intel)", - "unmaskedRenderer": "ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_4_1 ps_4_1), or similar" - }, - "speechVoices": [ - "Microsoft Raul - Spanish (Mexico):es-MX:local", - "Microsoft Sabina - Spanish (Mexico):es-MX:local", - "Microsoft Sabina Desktop - Spanish (Mexico):es-MX:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -9810,29 +9590,6 @@ "Microsoft David Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Win32", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1680, - "height": 1050, - "colorDepth": 24, - "availWidth": 1680, - "availHeight": 1010, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (NVIDIA)", - "unmaskedRenderer": "ANGLE (NVIDIA, NVIDIA GeForce 8800 GTX Direct3D11 vs_4_0 ps_4_0), or similar" - }, - "speechVoices": [ - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -9887,26 +9644,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Win32", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 720, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Intel)", - "unmaskedRenderer": "ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_4_1 ps_4_1), or similar" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", @@ -10893,33 +10630,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Win32", - "hardwareConcurrency": 8, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1280, - "height": 800, - "colorDepth": 24, - "availWidth": 1280, - "availHeight": 752, - "devicePixelRatio": 1.5 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Unknown)", - "unmaskedRenderer": "ANGLE (Unknown, Generic Renderer Direct3D11 vs_5_0 ps_5_0), or similar" - }, - "speechVoices": [ - "Microsoft Hortense - French (France):fr-FR:local", - "Microsoft Julie - French (France):fr-FR:local", - "Microsoft Paul - French (France):fr-FR:local", - "Microsoft Hortense Desktop - French:fr-FR:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", @@ -10976,33 +10686,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Win32", - "hardwareConcurrency": 8, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 720, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Unknown)", - "unmaskedRenderer": "ANGLE (Unknown, Adreno (TM) 650 Direct3D11 vs_5_0 ps_5_0), or similar" - }, - "speechVoices": [ - "Microsoft David - English (United States):en-US:local", - "Microsoft Mark - English (United States):en-US:local", - "Microsoft Zira - English (United States):en-US:local", - "Microsoft David Desktop - English (United States):en-US:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", @@ -11436,85 +11119,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Win32", - "hardwareConcurrency": 12, - "maxTouchPoints": 10 - }, - "screen": { - "width": 1440, - "height": 960, - "colorDepth": 24, - "availWidth": 1440, - "availHeight": 912, - "devicePixelRatio": 2 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Unknown)", - "unmaskedRenderer": "ANGLE (Unknown, Generic Renderer Direct3D11 vs_5_0 ps_5_0), or similar" - }, - "speechVoices": [ - "Microsoft David - English (United States):en-US:local", - "Microsoft Mark - English (United States):en-US:local", - "Microsoft Zira - English (United States):en-US:local", - "Microsoft David Desktop - English (United States):en-US:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Win32", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 728, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Intel)", - "unmaskedRenderer": "ANGLE (Intel, Intel(R) HD Graphics Direct3D11 vs_4_0 ps_4_0), or similar" - }, - "speechVoices": [ - "Microsoft Raul - Spanish (Mexico):es-MX:local", - "Microsoft Sabina - Spanish (Mexico):es-MX:local", - "Microsoft Sabina Desktop - Spanish (Mexico):es-MX:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Win32", - "hardwareConcurrency": 8, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1536, - "height": 864, - "colorDepth": 24, - "availWidth": 1536, - "availHeight": 816, - "devicePixelRatio": 1.25 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Unknown)", - "unmaskedRenderer": "ANGLE (Unknown, Adreno (TM) 650 Direct3D11 vs_5_0 ps_5_0), or similar" - }, - "speechVoices": [ - "Microsoft Daniel - Portuguese (Brazil):pt-BR:local", - "Microsoft Maria - Portuguese (Brazil):pt-BR:local", - "Microsoft Maria Desktop - Portuguese(Brazil):pt-BR:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:149.0) Gecko/20100101 Firefox/149.0", @@ -12126,26 +11730,6 @@ } ], "linux": [ - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 8, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1920, - "height": 1080, - "colorDepth": 24, - "availWidth": 1920, - "availHeight": 1080, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "NVIDIA Corporation", - "unmaskedRenderer": "NVIDIA GeForce 8800 GTX, or similar" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -12466,26 +12050,6 @@ "unmaskedRenderer": "NVIDIA GeForce GTX 980, or similar" } }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1368, - "height": 768, - "colorDepth": 24, - "availWidth": 1368, - "availHeight": 728, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Mesa", - "unmaskedRenderer": "GeForce 8800 GTX, or similar" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -12506,128 +12070,6 @@ "unmaskedRenderer": "Intel(R) HD Graphics, or similar" } }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 717, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Intel Open Source Technology Center", - "unmaskedRenderer": "Intel(R) HD Graphics 400, or similar" - }, - "speechVoices": [ - "Catalan:ca:remote", - "Bishnupriya Manipuri:bpy:remote", - "Nahuatl (Classical):nci:remote", - "Dutch:nl:remote", - "Azerbaijani:az:remote", - "Kyrgyz:ky:remote", - "English (Great Britain):en:remote", - "Chinese (Cantonese):yue:remote", - "English (Lancaster):en:remote", - "Vietnamese (Northern):vi:remote", - "Amharic:am:remote", - "Gaelic (Irish):ga:remote", - "Latvian:lv:remote", - "Swahili:sw:remote", - "Afrikaans:af:remote", - "Malay:ms:remote", - "Tatar:tt:remote", - "Nepali:ne:remote", - "Serbian:sr:remote", - "Oriya:or:remote", - "Arabic:ar:remote", - "Interlingua:ia:remote", - "Italian:it:remote", - "Aragonese:an:remote", - "German:de:remote", - "Assamese:as:remote", - "Bengali:bn:remote", - "Georgian:ka:remote", - "Lithuanian:lt:remote", - "Danish:da:remote", - "English (Caribbean):en:remote", - "Papiamento:pap:remote", - "Chinese (Mandarin):cmn:remote", - "Bosnian:bs:remote", - "Guarani:gn:remote", - "Marathi:mr:remote", - "Persian:fa:remote", - "Hindi:hi:remote", - "Bulgarian:bg:remote", - "Lingua Franca Nova:lfn:remote", - "Hungarian:hu:remote", - "Icelandic:is:remote", - "Burmese:my:remote", - "Maltese:mt:remote", - "Esperanto:eo:remote", - "Portuguese (Portugal):pt:remote", - "Spanish (Spain):es:remote", - "Kurdish:ku:remote", - "Lojban:jbo:remote", - "Czech:cs:remote", - "Turkish:tr:remote", - "Tamil:ta:remote", - "Spanish (Latin America):es:remote", - "Indonesian:id:remote", - "Armenian (West Armenia):hy:remote", - "Kannada:kn:remote", - "Greek (Ancient):grc:remote", - "Armenian (East Armenia):hy:remote", - "Greenlandic:kl:remote", - "French (Switzerland):fr:remote", - "Norwegian Bokmål:nb:remote", - "Portuguese (Brazil):pt:remote", - "Urdu:ur:remote", - "Punjabi:pa:remote", - "Japanese:ja:remote", - "Romanian:ro:remote", - "Gujarati:gu:remote", - "Latin:la:remote", - "Polish:pl:remote", - "French (France):fr:remote", - "Slovenian:sl:remote", - "Malayalam:ml:remote", - "Russian:ru:remote", - "Croatian:hr:remote", - "Korean:ko:remote", - "Welsh:cy:remote", - "Slovak:sk:remote", - "poz/mi:mi:remote", - "English (Scotland):en:remote", - "Vietnamese (Southern):vi:remote", - "English (West Midlands):en:remote", - "Persian (Pinglish):fa:remote", - "Albanian:sq:remote", - "Finnish:fi:remote", - "English (Received Pronunciation):en:remote", - "Greek:el:remote", - "French (Belgium):fr:remote", - "Gaelic (Scottish):gd:remote", - "Telugu:te:remote", - "Konkani:kok:remote", - "Sindhi:sd:remote", - "Estonian:et:remote", - "Basque:eu:remote", - "Oromo:om:remote", - "English (America):en:remote", - "Swedish:sv:remote", - "Macedonian:mk:remote", - "Setswana:tn:remote", - "Sinhala:si:remote", - "Vietnamese (Central):vi:remote" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20100101 Firefox/150.0", @@ -12748,46 +12190,6 @@ "unmaskedRenderer": "Intel(R) HD Graphics, or similar" } }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20100101 Firefox/150.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 718, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Intel Open Source Technology Center", - "unmaskedRenderer": "Intel(R) HD Graphics 400, or similar" - } - }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 24, - "maxTouchPoints": 0 - }, - "screen": { - "width": 3072, - "height": 1728, - "colorDepth": 24, - "availWidth": 3072, - "availHeight": 1728, - "devicePixelRatio": 2 - }, - "webgl": { - "unmaskedVendor": "NVIDIA Corporation", - "unmaskedRenderer": "NVIDIA GeForce 8800 GTX, or similar" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0", @@ -13419,26 +12821,6 @@ "unmaskedRenderer": "Intel(R) HD Graphics 400, or similar" } }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 735, - "devicePixelRatio": 1 - }, - "webgl": { - "unmaskedVendor": "Mesa", - "unmaskedRenderer": "Radeon HD 5850, or similar" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0", diff --git a/pythonlib/camoufox/fingerprint-presets.json b/pythonlib/camoufox/fingerprint-presets.json index 95ba13256..f44823518 100644 --- a/pythonlib/camoufox/fingerprint-presets.json +++ b/pythonlib/camoufox/fingerprint-presets.json @@ -208,25 +208,6 @@ "Zuzana:cs-CZ:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:109.0) Gecko/20100101 Firefox/115.0", - "platform": "MacIntel", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1280, - "height": 800, - "colorDepth": 30, - "availWidth": 1280, - "availHeight": 714 - }, - "webgl": { - "unmaskedVendor": "Intel Inc.", - "unmaskedRenderer": "Intel(R) HD Graphics" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:147.0) Gecko/20100101 Firefox/147.0", @@ -337,75 +318,6 @@ "unmaskedRenderer": "Apple M1, or similar" } }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:148.0) Gecko/20100101 Firefox/148.0", - "platform": "MacIntel", - "hardwareConcurrency": 4, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1920, - "height": 1080, - "colorDepth": 30, - "availWidth": 1920, - "availHeight": 1011 - }, - "webgl": { - "unmaskedVendor": "NVIDIA Corporation", - "unmaskedRenderer": "NVIDIA GeForce GTX 480, or similar" - }, - "speechVoices": [ - "Alex:en-US:local", - "Alice:it-IT:local", - "Alva:sv-SE:local", - "Amelie:fr-CA:local", - "Anna:de-DE:local", - "Carmit:he-IL:local", - "Damayanti:id-ID:local", - "Daniel:en-GB:local", - "Diego:es-AR:local", - "Ellen:nl-BE:local", - "Fiona:en-scotland:local", - "Fred:en-US:local", - "Ioana:ro-RO:local", - "Joana:pt-PT:local", - "Jorge:es-ES:local", - "Juan:es-MX:local", - "Kanya:th-TH:local", - "Karen:en-AU:local", - "Kyoko:ja-JP:local", - "Laura:sk-SK:local", - "Lekha:hi-IN:local", - "Luca:it-IT:local", - "Luciana:pt-BR:local", - "Maged:ar-SA:local", - "Mariska:hu-HU:local", - "Mei-Jia:zh-TW:local", - "Melina:el-GR:local", - "Milena:ru-RU:local", - "Moira:en-IE:local", - "Monica:es-ES:local", - "Nora:nb-NO:local", - "Paulina:es-MX:local", - "Rishi:en-IN:local", - "Samantha:en-US:local", - "Sara:da-DK:local", - "Satu:fi-FI:local", - "Sin-ji:zh-HK:local", - "Tessa:en-ZA:local", - "Thomas:fr-FR:local", - "Ting-Ting:zh-CN:local", - "Veena:en-IN:local", - "Victoria:en-US:local", - "Xander:nl-NL:local", - "Yelda:tr-TR:local", - "Yuna:ko-KR:local", - "Yuri:ru-RU:local", - "Zosia:pl-PL:local", - "Zuzana:cs-CZ:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:147.0) Gecko/20100101 Firefox/147.0", @@ -2982,28 +2894,6 @@ "Microsoft Huihui Desktop - Chinese (Simplified):zh-CN:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0", - "platform": "Win32", - "hardwareConcurrency": 2, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1366, - "height": 768, - "colorDepth": 24, - "availWidth": 1366, - "availHeight": 728 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (Intel)", - "unmaskedRenderer": "ANGLE (Intel, Intel 945GM Direct3D11 vs_4_0 ps_4_0)" - }, - "speechVoices": [ - "Microsoft Anna - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:148.0) Gecko/20100101 Firefox/148.0", @@ -3188,45 +3078,6 @@ "Microsoft Zira Desktop - English (United States):en-US:local" ] }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:148.0) Gecko/20100101 Firefox/148.0", - "platform": "Win32", - "hardwareConcurrency": 8, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1680, - "height": 1050, - "colorDepth": 24, - "availWidth": 1680, - "availHeight": 1010 - }, - "webgl": { - "unmaskedVendor": "Google Inc. (NVIDIA)", - "unmaskedRenderer": "ANGLE (NVIDIA, NVIDIA GeForce 8800 GTX Direct3D11 vs_4_0 ps_4_0), or similar" - }, - "speechVoices": [ - "Microsoft David - English (United States):en-US:local", - "Microsoft James - English (Australia):en-AU:local", - "Microsoft Linda - English (Canada):en-CA:local", - "Microsoft Richard - English (Canada):en-CA:local", - "Microsoft George - English (United Kingdom):en-GB:local", - "Microsoft Hazel - English (United Kingdom):en-GB:local", - "Microsoft Susan - English (United Kingdom):en-GB:local", - "Microsoft Sean - English (Ireland):en-IE:local", - "Microsoft Heera - English (India):en-IN:local", - "Microsoft Ravi - English (India):en-IN:local", - "Microsoft Catherine - English (Australia):en-AU:local", - "Microsoft Mark - English (United States):en-US:local", - "Microsoft Zira - English (United States):en-US:local", - "Microsoft George - English (Great Britain):en-GB:local", - "Microsoft Sarah Mobile - English (Great Britain):en-GB:local", - "Microsoft Hazel Desktop - English (Great Britain):en-GB:local", - "Microsoft David Desktop - English (United States):en-US:local", - "Microsoft Zira Desktop - English (United States):en-US:local" - ] - }, { "navigator": { "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:148.0) Gecko/20100101 Firefox/148.0", @@ -4360,25 +4211,6 @@ "unmaskedRenderer": "NVIDIA GeForce GTX 980, or similar" } }, - { - "navigator": { - "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:148.0) Gecko/20100101 Firefox/148.0", - "platform": "Linux x86_64", - "hardwareConcurrency": 8, - "maxTouchPoints": 0 - }, - "screen": { - "width": 1920, - "height": 1080, - "colorDepth": 24, - "availWidth": 1920, - "availHeight": 1010 - }, - "webgl": { - "unmaskedVendor": "NVIDIA Corporation", - "unmaskedRenderer": "GeForce GTX 480, or similar" - } - }, { "navigator": { "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0", diff --git a/pythonlib/camoufox/fingerprints.py b/pythonlib/camoufox/fingerprints.py index c3d414e8a..698d190a7 100644 --- a/pythonlib/camoufox/fingerprints.py +++ b/pythonlib/camoufox/fingerprints.py @@ -6,11 +6,12 @@ import unicodedata from dataclasses import asdict, dataclass, is_dataclass from pathlib import Path -from random import Random, choice, randint, randrange, random, sample, shuffle +from random import Random, choice, randint, randrange from typing import Any, Dict, FrozenSet, List, Optional, Set, Tuple +from camoufox._warnings import FallbackWarning +from camoufox.ip import valid_ipv4, validate_ip from camoufox.pkgman import load_yaml -from camoufox.webgl import sample_webgl # Load the fpgen mapping file FPGEN_DATA = load_yaml('fpgen.yml') @@ -399,7 +400,10 @@ def _load_font_groups() -> Dict[str, List[Dict[str, Any]]]: try: with open(path, 'rb') as f: _FONT_GROUPS_CACHE = json.loads(f.read()) - except (OSError, ValueError): + except (OSError, ValueError) as e: + FallbackWarning.warn( + 'Reading font-groups.json', 'an OS-version base with no font additions', e + ) _FONT_GROUPS_CACHE = {} return _FONT_GROUPS_CACHE @@ -422,7 +426,10 @@ def _load_font_bases() -> Dict[str, List[Dict[str, Any]]]: try: with open(path, 'rb') as f: _FONT_BASES_CACHE = json.loads(f.read()) - except (OSError, ValueError): + except (OSError, ValueError) as e: + FallbackWarning.warn( + 'Reading font-bases.json', 'only the always-present core fonts as its OS base', e + ) _FONT_BASES_CACHE = {} return _FONT_BASES_CACHE @@ -595,44 +602,6 @@ def _generate_random_font_subset( return result -# OS voice lists loaded from voices.json, parsed into "Name:lang:type" tuples. -_OS_VOICES_CACHE: Optional[Dict[str, List[Tuple[str, str, str]]]] = None - - -def _load_os_voices() -> Dict[str, List[Tuple[str, str, str]]]: - """Load OS voice lists from voices.json as (name, lang, type) tuples. - - Each entry is "Name:lang:type" (type is "local" or "remote"). Voice names - may contain parens/commas but not colons, so a last-two-colons split is - safe. - """ - global _OS_VOICES_CACHE - if _OS_VOICES_CACHE is not None: - return _OS_VOICES_CACHE - voices_path = os.path.join(os.path.dirname(__file__), 'voices.json') - with open(voices_path, 'rb') as f: - import orjson - raw = orjson.loads(f.read()) - _OS_VOICES_CACHE = {} - for os_key, entries in raw.items(): - parsed: List[Tuple[str, str, str]] = [] - for entry in entries: - last = entry.rfind(':') - if last < 0: - continue - vtype = entry[last + 1:] - before = entry[:last] - langsep = before.rfind(':') - if langsep < 0: - continue - lang = before[langsep + 1:] - name = before[:langsep] - if name and lang: - parsed.append((name, lang, vtype)) - _OS_VOICES_CACHE[os_key] = parsed - return _OS_VOICES_CACHE - - # Essential speech voices per OS that must always be included in subsets _ESSENTIAL_VOICES_MACOS = [ 'Samantha', 'Alex', 'Fred', 'Victoria', 'Karen', 'Daniel', @@ -1305,8 +1274,8 @@ def add(item: Dict[str, Any], grp: str) -> None: # -- WebGL <-> screen coherence (#729) --------------------------------------- # -# BrowserForge picks navigator/screen; the GPU is drawn separately from -# webgl_data.db weighted only by OS. Nothing ties the two together, so the +# fpgen picks navigator/screen; the GPU is drawn separately, weighted only by +# OS (camoufox.webgl). Nothing ties the two together, so the # synthetic path can emit pairs no real machine ships -- a discrete GPU behind # a 1024x600 netbook panel. Consistency checks (Pixelscan, Fingerprint.com) # read that as masking even when every individual value is plausible alone. @@ -1344,8 +1313,8 @@ def add(item: Dict[str, Any], grp: str) -> None: 'Generic Renderer', ) -# Discrete NVIDIA, plus the AMD R5/R7/R9/RX/Vega bucket. Everything else in -# webgl_data.db reaches down into netbook territory and gets no floor at all: +# Discrete NVIDIA, plus the AMD R5/R7/R9/RX/Vega bucket. Every other GPU +# Firefox reports reaches down into netbook territory and gets no floor at all: # the "Intel(R) HD Graphics" bucket swallows the GMA 3150 netbook chipset, # "Radeon HD 3200 Graphics" is Gecko's catch-all for a bare "AMD"/"Radeon" # (the C-50/E-350 netbook APUs included), and Apple silicon drives arbitrary @@ -1457,57 +1426,6 @@ def gpu_screen_is_plausible( return width * height > _NETBOOK_MAX_PIXELS -def sample_webgl_for_screen( - target_os: str, - width: Optional[int] = None, - height: Optional[int] = None, - attempts: int = 32, - seed: Optional[int] = None, -) -> Dict[str, str]: - """Sample a WebGL profile that is coherent with the screen already chosen. - - Rejection sampling, so the GPU keeps webgl_data.db's real OS-weighted - distribution -- we only drop draws that contradict the screen. The screen - itself is left alone on purpose: it has already been reconciled with the - real display and the window box (clamp_screen_to_display, - fix_screen_no_taskbar, clamp_window_dimensions, clamp_window_position), - and widening it here to flatter the GPU would push a headful window back - off the monitor it is drawn on (#499). - - Software rasterisers are the one exception to keeping the pool's rate: a - draw that lands on llvmpipe / WARP / SwiftShader is resampled, so they - never present as the GPU (see below). That does cost fidelity -- the corpus - records them at ~1.5%, because real users do run without working drivers -- - but "no consumer machine reports llvmpipe" is a live, standard check on a - string every fingerprint script already reads, so the trade is worth it. - Reviewed against the JS-detectability bar on 2026-09-17 and kept. - - Falls back to that first draw when the pool holds nothing coherent, so an - unusual screen degrades to today's behaviour rather than raising. - """ - # A software rasteriser (llvmpipe / SwiftShader / WARP) as the presented - # GPU is what every consumer-hardware check flags first ("no consumer - # machine reports llvmpipe" -- sundial, measured 2026-09-14), so the draw - # never settles on one: keep drawing until a hardware renderer that fits - # the screen comes up, and only fall back to the first draw if the pool - # holds nothing better. - first = sample_webgl(target_os, seed=seed) - renderer = first.get('webGl:renderer') - if not is_software_renderer(renderer) and gpu_screen_is_plausible(renderer, width, height): - return first - - fallback = None if is_software_renderer(renderer) else first - for attempt in range(attempts - 1): - candidate = sample_webgl(target_os, seed=None if seed is None else seed + 1 + attempt) - renderer = candidate.get('webGl:renderer') - if is_software_renderer(renderer): - continue - if gpu_screen_is_plausible(renderer, width, height): - return candidate - fallback = fallback or candidate - return fallback or first - - def _select_presets_file(ff_version: Optional[Any] = None) -> Path: """Pick the bundled-presets file appropriate for a given Firefox version. @@ -1682,12 +1600,8 @@ def from_preset(preset: Dict, ff_version: Optional[str] = None, salt: Optional[i if webgl.get('unmaskedRenderer'): config['webGl:renderer'] = webgl['unmaskedRenderer'] - # Generate unique random seeds per launch (1 to 2^32-1, excluding 0 which is a no-op in C++) - # fonts:spacing_seed stays 0 (off): glyph-advance perturbation produces text - # widths no real machine emits (see launch_options in utils.py). - config['fonts:spacing_seed'] = 0 + # Generate a unique audio seed per launch (1 to 2^32-1, excluding 0 which is a no-op in C++) config['audio:seed'] = randint(1, 4_294_967_295) # nosec - config['canvas:seed'] = randint(1, 4_294_967_295) # nosec if preset.get('timezone'): config['timezone'] = preset['timezone'] @@ -1702,10 +1616,16 @@ def from_preset(preset: Dict, ff_version: Optional[str] = None, salt: Optional[i target_os = 'linux' else: target_os = 'macos' + preset_key = f"{config.get('navigator.userAgent')} / {config.get('webGl:renderer')}" try: config['fonts'] = _generate_random_font_subset(target_os, seed=identity_seed(config, salt)) - except Exception: - # Fallback to preset fonts if font generation fails + except (OSError, ValueError) as e: + FallbackWarning.warn( + 'Drawing the font list', + "the preset's recorded fonts" if preset.get('fonts') else "the browser's own fonts", + e, + preset_key, + ) if preset.get('fonts'): fonts = list(preset['fonts']) _ensure_marker_fonts(fonts, { @@ -1717,7 +1637,13 @@ def from_preset(preset: Dict, ff_version: Optional[str] = None, salt: Optional[i # Generate a unique random voice subset from the OS voice list try: config['voices'] = _generate_random_voice_subset(target_os, seed=identity_seed(config, salt)) - except Exception: + except (OSError, ValueError, KeyError) as e: + FallbackWarning.warn( + 'Drawing the speech voices', + "the preset's recorded voices" if preset.get('speechVoices') else "the browser's own voices", + e, + preset_key, + ) if preset.get('speechVoices'): config['voices'] = _normalize_preset_voices( preset['speechVoices'], target_os @@ -1736,9 +1662,7 @@ def _build_init_script(values: Dict[str, Any]) -> str: lines = ['(function(v) {', ' var w = window;'] setters = [ - ('fontSpacingSeed', 'setFontSpacingSeed', '{val}'), ('audioFingerprintSeed', 'setAudioFingerprintSeed', '{val}'), - ('canvasSeed', 'setCanvasSeed', '{val}'), ('navigatorPlatform', 'setNavigatorPlatform', '{val}'), ('navigatorOscpu', 'setNavigatorOscpu', '{val}'), ('navigatorUserAgent', 'setNavigatorUserAgent', '{val}'), @@ -1783,8 +1707,10 @@ def _build_init_script(values: Dict[str, Any]) -> str: # WebRTC IP ip = values.get('webrtcIP') if ip: + validate_ip(ip) + fn_name = 'setWebRTCIPv4' if valid_ipv4(ip) else 'setWebRTCIPv6' lines.append( - f' if (typeof w.setWebRTCIPv4 === "function") w.setWebRTCIPv4({_json.dumps(ip)});' + f' if (typeof w.{fn_name} === "function") w.{fn_name}({_json.dumps(ip)});' ) else: lines.append( @@ -1827,7 +1753,7 @@ def generate_context_fingerprint( Generate fingerprint values for a single per-context identity. Returns a dict with init_script (JS string) and context_options (Playwright options). - By default, uses BrowserForge for infinite unique synthetic fingerprints. + By default, fpgen generates a unique synthetic fingerprint. Pass a preset dict to use a real fingerprint preset instead. Parameters: @@ -1838,8 +1764,7 @@ def generate_context_fingerprint( normalize_locale() and injected into config. Also sets context_options['locale'] for Playwright. config_overrides: Dict of CAMOU_CONFIG keys to override after config - is built but before init_script is rendered. Useful for disabling - perturbation (e.g. {'fonts:spacing_seed': 0}). + is built but before init_script is rendered (e.g. {'audio:seed': 7}). """ if preset is not None: # Use real fingerprint preset @@ -1856,9 +1781,7 @@ def generate_context_fingerprint( _salt = identity_salt() # Add seeds (the generator doesn't produce these) - config.setdefault('fonts:spacing_seed', 0) # perturbation off; see utils.launch_options config.setdefault('audio:seed', randint(1, 4_294_967_295)) # nosec - config.setdefault('canvas:seed', randint(1, 4_294_967_295)) # nosec # Determine target OS from platform for font/voice generation plat = config.get('navigator.platform', '') @@ -1868,21 +1791,27 @@ def generate_context_fingerprint( elif 'Linux' in plat or 'linux' in plat: os_name = 'linux' - # Add fonts (BrowserForge doesn't generate these) + # Add fonts (fpgen.yml does not map these yet) if 'fonts' not in config: try: config['fonts'] = _generate_random_font_subset(os_name, seed=identity_seed(config, _salt)) - except Exception: - pass + except (OSError, ValueError) as e: + FallbackWarning.warn( + 'Drawing the font list', "the browser's launch-time fonts", e, + config.get('navigator.userAgent'), + ) - # Add voices (BrowserForge doesn't generate these) + # Add voices (fpgen.yml does not map these yet) if 'voices' not in config: try: config['voices'] = _generate_random_voice_subset(os_name, seed=identity_seed(config, _salt)) - except Exception: - pass + except (OSError, ValueError, KeyError) as e: + FallbackWarning.warn( + 'Drawing the speech voices', "the browser's launch-time voices", e, + config.get('navigator.userAgent'), + ) - # Derive oscpu if BrowserForge didn't provide it + # Derive oscpu if the fingerprint didn't provide it if 'navigator.oscpu' not in config: plat = config.get('navigator.platform', '') if plat == 'MacIntel': @@ -1892,8 +1821,11 @@ def generate_context_fingerprint( elif 'Linux' in plat or 'linux' in plat: config['navigator.oscpu'] = 'Linux x86_64' - # Sample WebGL vendor/renderer from database (BrowserForge doesn't generate these) + # Draw the GPU and its WebGL data (fpgen.yml does not map these) if not config.get('webGl:vendor') or not config.get('webGl:renderer'): + # Not at the top: camoufox.webgl imports this module. + from .webgl import sample_webgl_for_screen + _os_map = {'macos': 'mac', 'linux': 'lin', 'windows': 'win'} _target_os = _os_map.get(os or '', None) if not _target_os: @@ -1904,21 +1836,18 @@ def generate_context_fingerprint( _target_os = 'lin' else: _target_os = 'mac' - try: - # Same coherence treatment launch_options applies (#729): lift - # netbook geometry, then keep the GPU consistent with whatever - # screen this identity ended up with. This path has no real - # display to reconcile against, so the floor is unconditional. - raise_screen_to_modern_floor(config) - webgl_fp = sample_webgl_for_screen( - _target_os, config.get('screen.width'), config.get('screen.height') - ) - webgl_fp.pop('webGl2Enabled', None) - config.update(webgl_fp) - except Exception: - pass + # Same coherence treatment launch_options applies (#729): lift + # netbook geometry, then keep the GPU consistent with whatever + # screen this identity ended up with. This path has no real + # display to reconcile against, so the floor is unconditional. + raise_screen_to_modern_floor(config) + webgl_fp = sample_webgl_for_screen( + _target_os, config.get('screen.width'), config.get('screen.height') + ) + webgl_fp.pop('webGl2Enabled') + config.update(webgl_fp) - # Build source dicts from BrowserForge config for init_values + # Build source dicts from the fingerprint config for init_values nav = { 'platform': config.get('navigator.platform'), 'hardwareConcurrency': config.get('navigator.hardwareConcurrency'), @@ -1953,9 +1882,7 @@ def generate_context_fingerprint( # Build the values dict for the init script (works for both paths) init_values: Dict[str, Any] = { - 'fontSpacingSeed': config.get('fonts:spacing_seed'), 'audioFingerprintSeed': config.get('audio:seed'), - 'canvasSeed': config.get('canvas:seed'), 'navigatorPlatform': nav.get('platform'), 'navigatorOscpu': config.get('navigator.oscpu'), 'navigatorUserAgent': config.get('navigator.userAgent'), @@ -2012,7 +1939,7 @@ def _cast_to_properties( ff_version: Optional[str] = None, ) -> None: """ - Casts Browserforge fingerprints to Camoufox config properties. + Casts a generated fingerprint to Camoufox config properties. """ for key, data in bf_dict.items(): # Ignore non-truthy values diff --git a/pythonlib/camoufox/fontprobe.py b/pythonlib/camoufox/fontprobe.py deleted file mode 100644 index 316569abb..000000000 --- a/pythonlib/camoufox/fontprobe.py +++ /dev/null @@ -1,297 +0,0 @@ -"""What fonts are actually installed on THIS machine. - -Camoufox ships its own font bundle and spoofs the claimed OS's font list from -it, so nothing in a normal launch depends on the host's fonts. This module is -the diagnostic half: it answers "what does this machine really have", which is -what a user needs to know when deciding whether a claimed identity is plausible -here, and what `camoufox fonts` reports. - -Enumeration is per platform because the authoritative list is: - - Linux fontconfig (`fc-list`), which is what Gecko itself asks - Windows the font registry, plus the per-user font directory that - Windows 10 1809 and later install into without touching HKLM - macOS the three font directories CoreText reads - -The result is cached under the Camoufox cache directory and invalidated by the -(path, mtime, size) of every directory scanned, so a call after the first costs -a handful of stats. -""" - -import json -import os -import platform -import subprocess -import sys -import time -from typing import Any, Dict, Iterable, List, Optional, Set, Tuple - -CACHE_VERSION = 1 -FONT_EXT = (".ttf", ".otf", ".ttc", ".otc", ".dfont", ".pfb") - -# fc-list can be slow on a machine with thousands of fonts, and a launch should -# never hang on it. -PROBE_TIMEOUT_S = 20 - - -def host_os() -> str: - """'linux', 'macos' or 'windows' -- the keys the font data is filed under.""" - return {"Linux": "linux", "Darwin": "macos", "Windows": "windows"}.get( - platform.system(), "linux") - - -def normalise(name: str) -> str: - """The form two family names are compared in. - - Case and surrounding space only. Nothing clever: "Segoe UI" and - "Segoe UI Semibold" are different families to DirectWrite and must stay - different here, or a host with one would be credited with the other. - """ - return " ".join(name.split()).lower() - - -# -------------------------------------------------------------------------- -# per-platform enumeration -# -------------------------------------------------------------------------- - -def _linux_font_dirs() -> List[str]: - dirs = ["/usr/share/fonts", "/usr/local/share/fonts", - os.path.expanduser("~/.local/share/fonts"), - os.path.expanduser("~/.fonts")] - return [d for d in dirs if os.path.isdir(d)] - - -def _windows_font_dirs() -> List[str]: - dirs = [os.path.join(os.environ.get("WINDIR", r"C:\Windows"), "Fonts")] - local = os.environ.get("LOCALAPPDATA") - if local: - dirs.append(os.path.join(local, "Microsoft", "Windows", "Fonts")) - return [d for d in dirs if os.path.isdir(d)] - - -def _macos_font_dirs() -> List[str]: - dirs = ["/System/Library/Fonts", "/Library/Fonts", - os.path.expanduser("~/Library/Fonts"), - "/System/Library/Fonts/Supplemental"] - return [d for d in dirs if os.path.isdir(d)] - - -def font_dirs() -> List[str]: - return {"linux": _linux_font_dirs, "windows": _windows_font_dirs, - "macos": _macos_font_dirs}[host_os()]() - - -def _from_fc_list() -> Optional[Set[str]]: - """Ask fontconfig, which is the same source Gecko uses on Linux. - - Deliberately run with the caller's own environment stripped of Camoufox's - FONTCONFIG_FILE: that variable points at the bundle, and the question here - is what the HOST has. - """ - env = dict(os.environ) - env.pop("FONTCONFIG_FILE", None) - env.pop("FONTCONFIG_PATH", None) - try: - out = subprocess.run(["fc-list", "--format", "%{family}\\n"], - capture_output=True, timeout=PROBE_TIMEOUT_S, - env=env) - except (OSError, subprocess.SubprocessError): - return None - if out.returncode != 0: - return None - families: Set[str] = set() - for line in out.stdout.decode("utf-8", "replace").splitlines(): - # fontconfig prints every alias of a family, comma separated. - for name in line.split(","): - name = name.strip() - if name: - families.add(name) - return families or None - - -def _from_windows_registry() -> Optional[Set[str]]: - """The font registry: what GDI and DirectWrite enumerate.""" - try: - import winreg # noqa: WPS433 - except ImportError: - return None - families: Set[str] = set() - keys = [(winreg.HKEY_LOCAL_MACHINE, - r"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts"), - (winreg.HKEY_CURRENT_USER, - r"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts")] - for root, path in keys: - try: - handle = winreg.OpenKey(root, path) - except OSError: - continue - try: - index = 0 - while True: - try: - name, _value, _kind = winreg.EnumValue(handle, index) - except OSError: - break - index += 1 - # Values read "Segoe UI Bold (TrueType)" or - # "MS Gothic & MS PGothic & MS UI Gothic (TrueType)". - for part in name.split("(")[0].split("&"): - part = part.strip() - if part: - families.add(part) - finally: - winreg.CloseKey(handle) - return families or None - - -def _from_font_files(dirs: Iterable[str], limit: int = 0) -> Set[str]: - """Read the name table of every font file. The portable fallback.""" - try: - from fontTools.ttLib import TTCollection, TTFont # noqa: WPS433 - except ImportError: - return set() - - families: Set[str] = set() - seen = 0 - for directory in dirs: - for root, _subdirs, names in os.walk(directory): - for name in sorted(names): - if not name.lower().endswith(FONT_EXT): - continue - path = os.path.join(root, name) - seen += 1 - if limit and seen > limit: - return families - try: - if name.lower().endswith((".ttc", ".otc")): - with TTCollection(path, lazy=True) as collection: - fonts = list(collection.fonts) - else: - fonts = [TTFont(path, lazy=True, - ignoreDecompileErrors=True)] - except Exception: - continue - for font in fonts: - try: - table = font["name"] if "name" in font else None - if table is None: - continue - for name_id in (16, 1): - value = table.getDebugName(name_id) - if value: - families.add(value.strip()) - except Exception: - pass - finally: - try: - font.close() - except Exception: - pass - return families - - -# -------------------------------------------------------------------------- -# the cached inventory -# -------------------------------------------------------------------------- - -def _signature(dirs: Iterable[str]) -> str: - """Cheap invalidation: the directories and their mtimes. - - Installing or removing a font changes the mtime of the directory it is in, - which is enough -- and costs four stats instead of reading 3000 name - tables. - """ - parts = [] - for directory in sorted(dirs): - try: - stat = os.stat(directory) - parts.append("%s:%d:%d" % (directory, int(stat.st_mtime), stat.st_size)) - except OSError: - parts.append("%s:-" % directory) - return "|".join(parts) - - -def _cache_path() -> Optional[str]: - try: - from .pkgman import INSTALL_DIR # noqa: WPS433 - base = str(INSTALL_DIR) - except Exception: - base = os.path.join(os.path.expanduser("~"), ".cache", "camoufox") - try: - os.makedirs(os.path.join(base, "fontcache"), exist_ok=True) - except OSError: - return None - return os.path.join(base, "fontcache", "host-fonts.json") - - -def installed_families(refresh: bool = False, - use_cache: bool = True) -> Dict[str, Any]: - """Every font family this machine has, with where the answer came from. - - Returns {"os", "families": [...], "source", "dirs", "seconds", "cached"}. - """ - dirs = font_dirs() - signature = _signature(dirs) - path = _cache_path() if use_cache else None - - if path and not refresh and os.path.exists(path): - try: - with open(path, "rb") as fh: - cached = json.loads(fh.read()) - if (cached.get("version") == CACHE_VERSION - and cached.get("signature") == signature): - cached["cached"] = True - return cached - except (OSError, ValueError): - pass - - started = time.time() - families: Optional[Set[str]] = None - source = "files" - system = host_os() - if system == "linux": - families = _from_fc_list() - source = "fontconfig" - elif system == "windows": - families = _from_windows_registry() - source = "registry" - if not families: - families = _from_font_files(dirs) - source = "files" - - result = { - "version": CACHE_VERSION, - "signature": signature, - "os": system, - "source": source, - "dirs": dirs, - "families": sorted(families), - "seconds": round(time.time() - started, 3), - "cached": False, - } - if path: - try: - tmp = path + ".tmp%d" % os.getpid() - with open(tmp, "w") as fh: - json.dump(result, fh) - os.replace(tmp, path) - except OSError: - pass - return result - - -def family_index(inventory: Optional[Dict[str, Any]] = None) -> Set[str]: - """The installed families, normalised for comparison.""" - inventory = inventory or installed_families() - return {normalise(name) for name in inventory.get("families", [])} - - -def partition(claimed: Iterable[str], - inventory: Optional[Dict[str, Any]] = None - ) -> Tuple[List[str], List[str]]: - """Split the families an identity claims into (on this host, not on it).""" - index = family_index(inventory) - real, missing = [], [] - for name in claimed: - (real if normalise(name) in index else missing).append(name) - return real, missing diff --git a/pythonlib/camoufox/fpgen.yml b/pythonlib/camoufox/fpgen.yml index 0d46c1124..1d1fb64e9 100644 --- a/pythonlib/camoufox/fpgen.yml +++ b/pythonlib/camoufox/fpgen.yml @@ -2,9 +2,11 @@ # # fpgen (scrapfly/fingerprint-generator) replaced BrowserForge/Apify as the # generator. The field set below is the one BrowserForge used to supply -- the -# rest of what fpgen carries (fonts, voices, WebGL parameters, system styles, -# permissions, RTC capabilities, audio) is NOT mapped here yet; Camoufox still -# draws those from its own catalogues. Wiring those through is the follow-up. +# rest of what fpgen carries (fonts, voices, system styles, permissions, RTC +# capabilities, audio) is NOT mapped here yet; Camoufox still draws those from +# its own catalogues. Wiring those through is the follow-up. WebGL is not +# mapped here either: camoufox/webgl.py draws it from fpgen, conditioned on +# the GPU, so presets get their own GPU's parameters too. # # Shape differences from browserforge.yml: # - window geometry is its own `window` node, not part of `screen` @@ -18,13 +20,10 @@ navigator: # fpgen's UAs trail the current release; the version is rewritten to the # Camoufox Firefox version in _cast_to_properties. userAgent: navigator.userAgent - appCodeName: navigator.appCodeName - appName: navigator.appName appVersion: navigator.appVersion oscpu: navigator.oscpu platform: navigator.platform hardwareConcurrency: navigator.hardwareConcurrency - product: navigator.product # Never override productSub (#105) # deviceMemory is not in Firefox, and fpgen reports the string "undefined" # Locale is handled separately (locale:*) @@ -62,7 +61,6 @@ window: # bottom edge land (see BROWSER_CHROME_HEIGHT in coherence.py). screenX: window.screenX screenY: window.screenY - pageYOffset: screen.pageYOffset # devicePixelRatio is not mapped: any value but 1 is a spoofing tell unless # the whole geometry is scaled with it. diff --git a/pythonlib/camoufox/geolocation.py b/pythonlib/camoufox/geolocation.py index 6dca40c31..d4c1ecac3 100644 --- a/pythonlib/camoufox/geolocation.py +++ b/pythonlib/camoufox/geolocation.py @@ -119,7 +119,7 @@ def get_mmdb_path(ip_version: str = 'ipv4', config: Optional[Dict] = None) -> Pa def geoip_allowed() -> None: """ - Checks if the geoip2 module is available + Checks if the maxminddb module is available """ if not ALLOW_GEOIP: raise NotInstalledGeoIPExtra( diff --git a/pythonlib/camoufox/ip.py b/pythonlib/camoufox/ip.py index e7771e822..b9045dd9e 100644 --- a/pythonlib/camoufox/ip.py +++ b/pythonlib/camoufox/ip.py @@ -75,6 +75,34 @@ def validate_ip(ip: str) -> None: raise InvalidIP(f"Invalid IP address: {ip}") +def proxy_exit_geo(proxy: str) -> Tuple[str, str]: + """ + The exit IP of `proxy` and that IP's timezone, looked up through the proxy. + Raises InvalidIP when the lookup fails: a context that silently kept the + host's WebRTC IP and timezone behind a proxy would be a leak. + """ + try: + resp = requests.get( + "http://ip-api.com/json?fields=status,message,query,timezone", + proxies=Proxy.as_requests_proxy(proxy), + timeout=10, + ) + resp.raise_for_status() + data = resp.json() + except requests.RequestException as exception: + raise InvalidIP(f"{PROXY_LOOKUP_FAILED}: {exception}") from exception + if data.get("status") != "success" or not data.get("timezone"): + raise InvalidIP(f"{PROXY_LOOKUP_FAILED}: {data.get('message') or data}") + validate_ip(data["query"]) + return data["query"], data["timezone"] + + +PROXY_LOOKUP_FAILED = ( + "Could not look up the proxy's exit IP and timezone. Pass webrtc_ip and " + "timezone_id explicitly to skip the lookup" +) + + @lru_cache(maxsize=None) def public_ip(proxy: Optional[str] = None) -> str: """ diff --git a/pythonlib/camoufox/multiversion.py b/pythonlib/camoufox/multiversion.py index c12d34fb5..08e53e12d 100644 --- a/pythonlib/camoufox/multiversion.py +++ b/pythonlib/camoufox/multiversion.py @@ -123,14 +123,6 @@ def latest_per_build(versions: List[Dict]) -> List[Dict]: ) -def get_cached_repo_names() -> List[str]: - """ - Get list of repo names in cache - """ - cache = load_repo_cache() - return [r['name'] for r in cache.get('repos', [])] - - def get_repo_name(github_repo: str) -> str: """ Get display name for a repo from repos.yml, lowercased diff --git a/pythonlib/camoufox/pkgman.py b/pythonlib/camoufox/pkgman.py index cb45254e2..bcbefb23b 100644 --- a/pythonlib/camoufox/pkgman.py +++ b/pythonlib/camoufox/pkgman.py @@ -2,7 +2,6 @@ import os import platform import re -import shutil import sys import tempfile from dataclasses import dataclass @@ -393,13 +392,6 @@ def from_path(path: Optional[Path] = None) -> 'Version': version=version_data.get('version'), ) - @staticmethod - def is_supported_path(path: Path) -> bool: - """ - Check if the version at the given path is supported - """ - return Version.from_path(path) >= VERSION_MIN - @staticmethod def build_minmax() -> Tuple['Version', 'Version']: return Version(build=CONSTRAINTS.MIN_VERSION), Version(build=CONSTRAINTS.MAX_VERSION) @@ -629,31 +621,6 @@ def download_file(file: DownloadBuffer, url: str) -> DownloadBuffer: rprint(f'Downloading package: {url}') return webdl(url, buffer=file) - def extract_zip(self, zip_file: DownloadBuffer) -> None: - """ - Extract a zip file to the installation directory - """ - rprint(f'Extracting Camoufox: {INSTALL_DIR}') - unzip(zip_file, str(INSTALL_DIR)) - - @staticmethod - def cleanup() -> bool: - """ - Clean up the old installation - """ - if INSTALL_DIR.exists(): - rprint(f'Cleaning up cache: {INSTALL_DIR}') - shutil.rmtree(INSTALL_DIR) - return True - return False - - def set_version(self) -> None: - """ - Write version.json to INSTALL_DIR - """ - with open(INSTALL_DIR / 'version.json', 'wb') as f: - f.write(orjson.dumps({'version': self.version, 'build': self.build})) - def install(self, replace: bool = False) -> None: """ Download and install camoufox to a versioned subdirectory diff --git a/pythonlib/camoufox/server.py b/pythonlib/camoufox/server.py index c59a56b93..669a6762b 100644 --- a/pythonlib/camoufox/server.py +++ b/pythonlib/camoufox/server.py @@ -8,6 +8,7 @@ from camoufox.pkgman import LOCAL_DATA from camoufox.utils import launch_options +from camoufox.virtdisplay import VirtualDisplay LAUNCH_SCRIPT: Path = LOCAL_DATA / "launchServer.js" @@ -60,7 +61,20 @@ def launch_server(**kwargs) -> NoReturn: ) kwargs.pop(unsupported, None) - config = launch_options(**kwargs) + virtual_display = None + if kwargs.get('headless') == 'virtual': + virtual_display = VirtualDisplay(debug=kwargs.get('debug')) + kwargs['virtual_display'] = virtual_display.get() + kwargs['headless'] = False + try: + _serve(launch_options(**kwargs)) + finally: + if virtual_display: + virtual_display.kill() + + +def _serve(config: Dict[str, Any]) -> NoReturn: + """Run launchServer.js with `config` until the Node process exits.""" nodejs = get_nodejs() data = orjson.dumps(to_camel_case_dict(config)) diff --git a/pythonlib/camoufox/sync_api.py b/pythonlib/camoufox/sync_api.py index aebc15d76..5cd65d7bd 100644 --- a/pythonlib/camoufox/sync_api.py +++ b/pythonlib/camoufox/sync_api.py @@ -1,7 +1,4 @@ -import json as _json -import urllib.request -from typing import Any, Dict, List, Optional, Union, overload -from urllib.parse import urlparse +from typing import Any, Dict, Optional, Tuple, Union, overload from playwright.sync_api import ( Browser, @@ -14,6 +11,7 @@ from camoufox.virtdisplay import VirtualDisplay from .fingerprints import generate_context_fingerprint +from .ip import Proxy, proxy_exit_geo from .utils import ( attach_no_viewport_default, attach_stock_media_defaults, @@ -156,27 +154,9 @@ def NewBrowser( cpu_affinity.restore(pid, previous) -def _proxy_url_with_creds(proxy: Dict[str, str]) -> str: - """Builds a proxy URL string with embedded credentials.""" - parsed = urlparse(proxy.get("server", "")) - user = proxy.get("username", "") - pwd = proxy.get("password", "") - if user and pwd: - return f"{parsed.scheme}://{user}:{pwd}@{parsed.netloc}" - return proxy.get("server", "") - - -def _resolve_proxy_geo(proxy: Dict[str, str]) -> Dict[str, Optional[str]]: - """Queries ip-api.com through the proxy for the exit IP and timezone.""" - proxy_url = _proxy_url_with_creds(proxy) - handler = urllib.request.ProxyHandler({"http": proxy_url, "https": proxy_url}) - opener = urllib.request.build_opener(handler) - try: - with opener.open("http://ip-api.com/json?fields=query,timezone", timeout=10) as resp: - data = _json.loads(resp.read()) - return {"ip": data.get("query") or None, "timezone": data.get("timezone") or None} - except Exception: - return {"ip": None, "timezone": None} +def _resolve_proxy_geo(proxy: Dict[str, str]) -> Tuple[str, str]: + """The proxy's exit IP and timezone.""" + return proxy_exit_geo(Proxy(**proxy).as_string()) def NewContext( @@ -193,27 +173,31 @@ def NewContext( """ Creates a new browser context with a unique fingerprint identity. - Each context gets its own real fingerprint preset - with unique seeds for audio, canvas, and font spacing noise. All values are applied + Each context gets its own identity (navigator, screen, WebGL, fonts, voices), + drawn by fpgen unless a preset is given, with its own audio noise seed. All values are applied via addInitScript so they self-destruct before page scripts can detect them. Parameters: browser: A Browser instance from NewBrowser or Camoufox. - preset: A specific fingerprint preset dict to use. If None, picks randomly. - os: Target OS for preset selection ("windows", "macos", "linux"). - ff_version: Firefox version string for UA patching. - webrtc_ip: IPv4 address to spoof for WebRTC ICE candidates. + preset: A fingerprint preset dict to use. If None, fpgen draws a new identity. + os: Target OS for the drawn identity ("windows", "macos", "linux"). + ff_version: Firefox major version to claim in the UA. Defaults to the browser's own. + webrtc_ip: IPv4 or IPv6 address to spoof for WebRTC ICE candidates. proxy: Per-context proxy (Playwright format: {"server": "...", "username": "...", "password": "..."}). + Unless webrtc_ip and timezone_id are both given, they are looked up from the + proxy's exit IP; InvalidIP is raised if that lookup fails. geolocation: Per-context geolocation ({"latitude": float, "longitude": float}). **context_kwargs: Additional Playwright new_context() options. """ + # The drawn UA carries fpgen's Firefox version, which must not disagree with + # the browser the page is actually talking to. + ff_version = ff_version or browser.version.split('.', 1)[0] + # Auto-derive WebRTC IP and timezone from proxy's exit IP when not explicitly provided if proxy and (not webrtc_ip or "timezone_id" not in context_kwargs): - geo = _resolve_proxy_geo(proxy) - if not webrtc_ip: - webrtc_ip = geo["ip"] - if "timezone_id" not in context_kwargs and geo["timezone"]: - context_kwargs["timezone_id"] = geo["timezone"] + exit_ip, timezone = _resolve_proxy_geo(proxy) + webrtc_ip = webrtc_ip or exit_ip + context_kwargs.setdefault("timezone_id", timezone) fp = generate_context_fingerprint(preset=preset, os=os, ff_version=ff_version, webrtc_ip=webrtc_ip) diff --git a/pythonlib/camoufox/utils.py b/pythonlib/camoufox/utils.py index 4facf9497..485aef9af 100644 --- a/pythonlib/camoufox/utils.py +++ b/pythonlib/camoufox/utils.py @@ -8,7 +8,6 @@ from os.path import abspath from pathlib import Path from pprint import pprint -from random import randint from typing import Any, Dict, List, Literal, Optional, Tuple, Union import numpy as np @@ -23,7 +22,7 @@ InvalidPropertyType, NonFirefoxFingerprint, ) -from .fingerprints import Screen, from_fpgen, from_preset, generate_fingerprint, get_random_preset, _generate_random_font_subset, _generate_random_voice_subset, fix_navigator_arch, fix_hardware_concurrency, identity_salt, identity_seed, fix_screen_no_taskbar, clamp_screen_to_display, clamp_window_dimensions, clamp_window_position, raise_screen_to_modern_floor, sample_webgl_for_screen, set_media_devices_defaults, WINDOWS_11_MARKER_FONTS +from .fingerprints import Screen, from_fpgen, from_preset, generate_fingerprint, get_random_preset, _generate_random_font_subset, _generate_random_voice_subset, fix_navigator_arch, fix_hardware_concurrency, identity_salt, identity_seed, fix_screen_no_taskbar, clamp_screen_to_display, clamp_window_dimensions, clamp_window_position, raise_screen_to_modern_floor, set_media_devices_defaults, WINDOWS_11_MARKER_FONTS from . import coherence from .geolocation import geoip_allowed, get_geolocation from .ip import Proxy, public_ip, valid_ipv4, valid_ipv6 @@ -41,8 +40,8 @@ launch_path, ) from .virtdisplay import VirtualDisplay -from ._warnings import LeakWarning -from .webgl import sample_webgl +from ._warnings import FallbackWarning, LeakWarning +from .webgl import sample_webgl_for_screen, webgl_for_gpu ListOrString: TypeAlias = Union[Tuple[str, ...], List[str], str] @@ -309,7 +308,7 @@ def get_env_vars( } os_dir = directory_map.get(user_agent_os, user_agent_os) - # v150+ uses "fontconfig/" (matching the Go launcher); older bundles shipped "fontconfigs/". + # v150+ uses "fontconfig/"; older bundles shipped "fontconfigs/". def _bundle_path(*parts: str) -> str: if path: return str(path.parent.joinpath(*parts)) @@ -515,15 +514,6 @@ def check_valid_os(os: ListOrString) -> None: raise InvalidOS(f"Camoufox does not support the OS: '{os}'") -def _clean_locals(data: Dict[str, Any]) -> Dict[str, Any]: - """ - Gets the launch options from the locals of the function. - """ - del data['playwright'] - del data['persistent_context'] - return data - - def merge_into(target: Dict[str, Any], source: Dict[str, Any]) -> None: """ Merges new keys/values from the source dictionary into the target dictionary. @@ -568,7 +558,7 @@ def warn_manual_config(config: Dict[str, Any]) -> None: """ # Manual locale setting if is_domain_set( - config, 'navigator.language', 'navigator.languages', 'headers.Accept-Language', 'locale:' + config, 'navigator.language', 'headers.Accept-Language', 'locale:' ): LeakWarning.warn('locale', False) # Manual geolocation and timezone setting @@ -585,6 +575,8 @@ def warn_manual_config(config: Dict[str, Any]) -> None: # CSS pointer media queries and the TouchEvent interfaces. if is_domain_set(config, 'navigator.maxTouchPoints'): LeakWarning.warn('max_touch_points', False) + if config.get('instantAnimations'): + LeakWarning.warn('instant_animations', False) # Manual screen/window setting if is_domain_set(config, 'screen.', 'window.', 'document.body.'): LeakWarning.warn('viewport', False) @@ -595,8 +587,6 @@ def warn_manual_config(config: Dict[str, Any]) -> None: 'window.outerHeight', 'window.innerWidth', 'window.innerHeight', - 'document.body.clientWidth', - 'document.body.clientHeight', ) @@ -894,16 +884,16 @@ def launch_options( If not provided, a random fingerprint will be generated based on the provided `os` & `screen` constraints. fingerprint_preset (Optional[Union[bool, Dict[str, Any]]]): - Opt into using real fingerprint presets instead of BrowserForge. + Opt into using real fingerprint presets instead of fpgen. Pass `True` to use a random bundled preset, or pass a preset dict directly. - By default (None), BrowserForge is used for infinite unique fingerprints. + By default (None), fpgen generates a unique fingerprint. ff_version (Optional[int]): Firefox version to use. Defaults to the current Camoufox version. To prevent leaks, only use this for special cases. headless (Optional[bool]): Whether to run the browser in headless mode. Defaults to False. - Note: If you are running linux, passing headless='virtual' to Camoufox & AsyncCamoufox - will use Xvfb. + Note: If you are running linux, passing headless='virtual' to Camoufox, AsyncCamoufox + or launch_server will use Xvfb. main_world_eval (Optional[bool]): Whether to enable running scripts in the main world. To use this, prepend "mw:" to the script: page.evaluate("mw:" + script). @@ -931,7 +921,7 @@ def launch_options( debug (Optional[bool]): Prints the config being sent to Camoufox. virtual_display (Optional[str]): - Virtual display number. Ex: ':99'. This is handled by Camoufox & AsyncCamoufox. + Virtual display number. Ex: ':99'. This is handled by Camoufox, AsyncCamoufox and launch_server. pin_cpu_cores (Optional[bool]): Pin the browser to navigator.hardwareConcurrency cores (Linux/Windows) so the fingerprint's own core count can be kept: @@ -941,6 +931,8 @@ def launch_options( which is equally coherent, just less diverse. webgl_config (Optional[Tuple[str, str]]): Use a specific WebGL vendor/renderer pair. Passed as a tuple of (vendor, renderer). + The pair must be one fpgen has recorded from Firefox on `os` + (camoufox.webgl.firefox_gpus); any other raises ValueError. **launch_options (Dict[str, Any]): Additional Firefox launch options. """ @@ -1004,7 +996,7 @@ def launch_options( _user_set_dnt = 'navigator.doNotTrack' in config _user_set_gpc = 'navigator.globalPrivacyControl' in config _user_set_accept_encoding = 'headers.Accept-Encoding' in config - _user_set_noise_seeds = {k for k in ('audio:seed', 'canvas:seed') if k in config} + _user_set_audio_seed = 'audio:seed' in config # The salt that makes every seeded draw belong to this identity (see # fingerprints.identity_salt): stable when the caller pinned the identity @@ -1045,10 +1037,10 @@ def launch_options( # Generate a fingerprint _used_preset = False if fingerprint is not None: - # User passed a custom BrowserForge fingerprint + # User passed a custom fingerprint if not i_know_what_im_doing: check_custom_fingerprint(fingerprint) - elif fingerprint_preset is not None: + elif fingerprint_preset: # User opted into real fingerprint presets if isinstance(fingerprint_preset, dict): preset = fingerprint_preset @@ -1156,7 +1148,11 @@ def launch_options( # OS base is claimed native=(target_os in ('mac', 'win') and _host_os_key() == target_os), ) - except Exception: + except (OSError, ValueError) as e: + FallbackWarning.warn( + 'Drawing the font list', f"every font fonts.json lists for {target_os}", e, + config.get('navigator.userAgent'), + ) update_fonts(config, target_os) # Draw the identity's media devices (counts + OS-style labels/groups from @@ -1277,23 +1273,15 @@ def launch_options( if not _user_set_accept_encoding: config.pop('headers.Accept-Encoding', None) - # Set random seeds for fingerprint noise (per launch) - # Glyph-advance perturbation is OFF by default (seed 0): it moves every - # measured text width off the value the same font produces on a real - # machine (measured 2026-09-14: +1 px per ~100 glyphs, fractional deltas - # on every measureText), which is a fingerprint no stock Firefox emits. - # Pass fonts:spacing_seed explicitly to opt back in. - set_into(config, 'fonts:spacing_seed', 0) - # audio/canvas noise seeds follow the identity: a returning "same device" - # must reproduce its audio and canvas hashes (#442/#765). Derived, not - # equal, so the two streams differ; never 0 (0 disables the noise). - # A preset draws its own random seeds; they are replaced here too so a - # pinned preset reproduces them, but a seed the caller set is kept. - _ident = identity_seed(config, _identity_salt) - if 'audio:seed' not in _user_set_noise_seeds: + # The audio noise seed follows the identity: a returning "same device" must + # reproduce its audio hash (#442/#765). Never 0 (0 disables the noise). A + # preset draws its own random seed; it is replaced here too so a pinned + # preset reproduces it, but a seed the caller set is kept. There is no + # canvas seed: the browser adds no canvas noise (#528), and no glyph-spacing + # noise either (ci/tribal-rules.yml: no-glyph-spacing-noise). + if not _user_set_audio_seed: + _ident = identity_seed(config, _identity_salt) config['audio:seed'] = ((_ident * 2654435761 + 97) & 0xFFFFFFFF) or 1 - if 'canvas:seed' not in _user_set_noise_seeds: - config['canvas:seed'] = ((_ident * 40503 + 12345) & 0xFFFFFFFF) or 1 # Set geolocation if geoip: @@ -1392,10 +1380,13 @@ def launch_options( config['voices'] = _generate_random_voice_subset( os_name_v, voice_locale, seed=identity_seed(config, _identity_salt) ) - except Exception: + except (OSError, ValueError, KeyError) as e: # An empty list still blocks the host's voices (see below), so a # generation failure degrades to "no voices" rather than "all of # the host's". + FallbackWarning.warn( + 'Drawing the speech voices', 'no speech voices', e, config.get('navigator.userAgent') + ) config['voices'] = [] # Pin the block explicitly instead of relying on a non-empty list to imply @@ -1430,46 +1421,22 @@ def launch_options( LeakWarning.warn('disable_coop', i_know_what_im_doing) firefox_user_prefs['browser.tabs.remote.useCrossOriginOpenerPolicy'] = False - # Allow allow_webgl parameter for backwards compatibility - if block_webgl or launch_options.pop('allow_webgl', True) is False: + if block_webgl: firefox_user_prefs['webgl.disabled'] = True LeakWarning.warn('block_webgl', i_know_what_im_doing) else: - # If the user has provided a specific WebGL vendor/renderer pair, use it + # A pair the caller named, or the preset's own GPU, keeps its name and + # gets that device's recorded parameters. webgl_for_gpu raises for a GPU + # fpgen has never seen: the caller asked for something that does not exist. if webgl_config: - webgl_fp = sample_webgl(target_os, *webgl_config, seed=identity_seed(config, _identity_salt)) + webgl_fp = webgl_for_gpu(target_os, *webgl_config, seed=identity_seed(config, _identity_salt)) elif config.get('webGl:vendor') and config.get('webGl:renderer'): - # Preset already set vendor/renderer — sample matching WebGL params - try: - webgl_fp = sample_webgl(target_os, config['webGl:vendor'], config['webGl:renderer'], seed=identity_seed(config, _identity_salt)) - except ValueError: - # The pair is not in webgl_data.db, which holds 33 GPUs. 39 of the - # 435 bundled presets name one it does not have -- including rows - # that cannot be the OS they are filed under, e.g. a Windows - # preset claiming "ANGLE (Unknown, Adreno (TM) 650 ...)", a phone - # GPU. Raising here made launch_options() fail outright for ~9% of - # presets, and a caller passing their own preset dict had no way - # to know which pairs are supported. - # - # There is no way to keep the named GPU: the parameters, extension - # list and shader precisions all have to come from a real recorded - # device, and there is none for an unknown renderer. So draw a GPU - # that fits the screen and let it replace the pair -- the identity - # loses the preset's GPU string but stays internally coherent, - # which is the property that matters to a page reading both. - webgl_fp = sample_webgl_for_screen( - target_os, config.get('screen.width'), config.get('screen.height'), - seed=identity_seed(config, _identity_salt), - ) - # merge_into does not overwrite keys that are already set, and the - # preset set these two. Drop them, or the page would read the - # preset's renderer string with another device's parameters, - # extensions and shader precisions behind it -- a mismatch louder - # than the unknown GPU we are replacing. - config.pop('webGl:vendor', None) - config.pop('webGl:renderer', None) + webgl_fp = webgl_for_gpu( + target_os, config['webGl:vendor'], config['webGl:renderer'], + seed=identity_seed(config, _identity_salt), + ) else: - # Synthetic path: keep the GPU coherent with the screen BrowserForge + # Synthetic path: keep the GPU coherent with the screen fpgen # already picked. Sampling the two independently yields pairs no # real machine ships -- a discrete desktop GPU behind a 1024x600 # panel -- which consistency checks read as masking (#729). @@ -1493,7 +1460,7 @@ def launch_options( # Every identity passes the whole-identity checks, whatever built it: a # generated fingerprint, a bundled preset, or a config the caller wrote. # The pools are sampled independently -- navigator and screen from the - # generator, GPU from webgl_data.db, fonts and voices from their own + # generator, GPU from fpgen's WebGL records, fonts and voices from their own # catalogues -- so a machine that never existed can be assembled from parts # that are each fine on their own. See coherence.py. _incoherent = coherence.apply(config, target_os) diff --git a/pythonlib/camoufox/voices.json b/pythonlib/camoufox/voices.json deleted file mode 100644 index beab0223a..000000000 --- a/pythonlib/camoufox/voices.json +++ /dev/null @@ -1,382 +0,0 @@ -{ - "mac": [ - "Albert:en-US:local", - "Alex:en-US:local", - "Alice:it-IT:local", - "Alva:sv-SE:local", - "Aman:en-IN:local", - "Amira:ms-MY:local", - "Anna:de-DE:local", - "Aru:kk-KZ:local", - "Bad News:en-US:local", - "Bahh:en-US:local", - "Bells:en-US:local", - "Boing:en-US:local", - "Bubbles:en-US:local", - "Carmit:he-IL:local", - "Cellos:en-US:local", - "Damayanti:id-ID:local", - "Daniel:en-GB:local", - "Daria:bg-BG:local", - "Diego:es-AR:local", - "Eddy (English (UK)):en-GB:local", - "Eddy (English (US)):en-US:local", - "Eddy (Finnish (Finland)):fi-FI:local", - "Eddy (French (Canada)):fr-CA:local", - "Eddy (French (France)):fr-FR:local", - "Eddy (German (Germany)):de-DE:local", - "Eddy (Italian (Italy)):it-IT:local", - "Eddy (Portuguese (Brazil)):pt-BR:local", - "Eddy (Spanish (Mexico)):es-MX:local", - "Eddy (Spanish (Spain)):es-ES:local", - "Eddy (Chinese (China mainland)):zh-CN:local", - "Eddy (Chinese (Taiwan)):zh-TW:local", - "Eddy (Japanese (Japan)):ja-JP:local", - "Eddy (Korean (South Korea)):ko-KR:local", - "Ellen:nl-BE:local", - "Fiona:en-scotland:local", - "Flo (English (UK)):en-GB:local", - "Flo (English (US)):en-US:local", - "Flo (Finnish (Finland)):fi-FI:local", - "Flo (French (Canada)):fr-CA:local", - "Flo (French (France)):fr-FR:local", - "Flo (German (Germany)):de-DE:local", - "Flo (Italian (Italy)):it-IT:local", - "Flo (Portuguese (Brazil)):pt-BR:local", - "Flo (Spanish (Mexico)):es-MX:local", - "Flo (Spanish (Spain)):es-ES:local", - "Flo (Chinese (China mainland)):zh-CN:local", - "Flo (Chinese (Taiwan)):zh-TW:local", - "Flo (Japanese (Japan)):ja-JP:local", - "Flo (Korean (South Korea)):ko-KR:local", - "Fred:en-US:local", - "Geeta:te-IN:local", - "Good News:en-US:local", - "Grandma (English (UK)):en-GB:local", - "Grandma (English (US)):en-US:local", - "Grandma (Finnish (Finland)):fi-FI:local", - "Grandma (French (Canada)):fr-CA:local", - "Grandma (French (France)):fr-FR:local", - "Grandma (German (Germany)):de-DE:local", - "Grandma (Italian (Italy)):it-IT:local", - "Grandma (Portuguese (Brazil)):pt-BR:local", - "Grandma (Spanish (Mexico)):es-MX:local", - "Grandma (Spanish (Spain)):es-ES:local", - "Grandma (Chinese (China mainland)):zh-CN:local", - "Grandma (Chinese (Taiwan)):zh-TW:local", - "Grandma (Japanese (Japan)):ja-JP:local", - "Grandma (Korean (South Korea)):ko-KR:local", - "Grandpa (English (UK)):en-GB:local", - "Grandpa (English (US)):en-US:local", - "Grandpa (Finnish (Finland)):fi-FI:local", - "Grandpa (French (Canada)):fr-CA:local", - "Grandpa (French (France)):fr-FR:local", - "Grandpa (German (Germany)):de-DE:local", - "Grandpa (Italian (Italy)):it-IT:local", - "Grandpa (Portuguese (Brazil)):pt-BR:local", - "Grandpa (Spanish (Mexico)):es-MX:local", - "Grandpa (Spanish (Spain)):es-ES:local", - "Grandpa (Chinese (China mainland)):zh-CN:local", - "Grandpa (Chinese (Taiwan)):zh-TW:local", - "Grandpa (Japanese (Japan)):ja-JP:local", - "Grandpa (Korean (South Korea)):ko-KR:local", - "Ioana:ro-RO:local", - "Jacques:fr-FR:local", - "Jester:en-US:local", - "Joana:pt-PT:local", - "Jorge:es-ES:local", - "Juan:es-MX:local", - "Junior:en-US:local", - "Kanya:th-TH:local", - "Karen:en-AU:local", - "Kathy:en-US:local", - "Kyoko:ja-JP:local", - "Lana:hr-HR:local", - "Laura:sk-SK:local", - "Lekha:hi-IN:local", - "Lesya:uk-UA:local", - "Linh:vi-VN:local", - "Luca:it-IT:local", - "Luciana:pt-BR:local", - "Majed:ar-001:local", - "Meijia:zh-TW:local", - "Melina:el-GR:local", - "Milena:ru-RU:local", - "Moira:en-IE:local", - "Montse:ca-ES:local", - "Nora:nb-NO:local", - "Ona:lt-LT:local", - "Organ:en-US:local", - "Paulina:es-MX:local", - "Piya:bn-IN:local", - "Ralph:en-US:local", - "Reed (English (UK)):en-GB:local", - "Reed (English (US)):en-US:local", - "Reed (Finnish (Finland)):fi-FI:local", - "Reed (French (Canada)):fr-CA:local", - "Reed (German (Germany)):de-DE:local", - "Reed (Italian (Italy)):it-IT:local", - "Reed (Portuguese (Brazil)):pt-BR:local", - "Reed (Spanish (Mexico)):es-MX:local", - "Reed (Spanish (Spain)):es-ES:local", - "Reed (Chinese (China mainland)):zh-CN:local", - "Reed (Chinese (Taiwan)):zh-TW:local", - "Reed (Japanese (Japan)):ja-JP:local", - "Reed (Korean (South Korea)):ko-KR:local", - "Rishi:en-IN:local", - "Rocko (English (UK)):en-GB:local", - "Rocko (English (US)):en-US:local", - "Rocko (Finnish (Finland)):fi-FI:local", - "Rocko (French (Canada)):fr-CA:local", - "Rocko (French (France)):fr-FR:local", - "Rocko (German (Germany)):de-DE:local", - "Rocko (Italian (Italy)):it-IT:local", - "Rocko (Portuguese (Brazil)):pt-BR:local", - "Rocko (Spanish (Mexico)):es-MX:local", - "Rocko (Spanish (Spain)):es-ES:local", - "Rocko (Chinese (China mainland)):zh-CN:local", - "Rocko (Chinese (Taiwan)):zh-TW:local", - "Rocko (Japanese (Japan)):ja-JP:local", - "Rocko (Korean (South Korea)):ko-KR:local", - "Samantha:en-US:local", - "Sandy (English (UK)):en-GB:local", - "Sandy (English (US)):en-US:local", - "Sandy (Finnish (Finland)):fi-FI:local", - "Sandy (French (Canada)):fr-CA:local", - "Sandy (French (France)):fr-FR:local", - "Sandy (German (Germany)):de-DE:local", - "Sandy (Italian (Italy)):it-IT:local", - "Sandy (Portuguese (Brazil)):pt-BR:local", - "Sandy (Spanish (Mexico)):es-MX:local", - "Sandy (Spanish (Spain)):es-ES:local", - "Sandy (Chinese (China mainland)):zh-CN:local", - "Sandy (Chinese (Taiwan)):zh-TW:local", - "Sandy (Japanese (Japan)):ja-JP:local", - "Sandy (Korean (South Korea)):ko-KR:local", - "Sara:da-DK:local", - "Satu:fi-FI:local", - "Shelley (English (UK)):en-GB:local", - "Shelley (English (US)):en-US:local", - "Shelley (Finnish (Finland)):fi-FI:local", - "Shelley (French (Canada)):fr-CA:local", - "Shelley (French (France)):fr-FR:local", - "Shelley (German (Germany)):de-DE:local", - "Shelley (Italian (Italy)):it-IT:local", - "Shelley (Portuguese (Brazil)):pt-BR:local", - "Shelley (Spanish (Mexico)):es-MX:local", - "Shelley (Spanish (Spain)):es-ES:local", - "Shelley (Chinese (China mainland)):zh-CN:local", - "Shelley (Chinese (Taiwan)):zh-TW:local", - "Shelley (Japanese (Japan)):ja-JP:local", - "Shelley (Korean (South Korea)):ko-KR:local", - "Sinji:zh-HK:local", - "Soumya:kn-IN:local", - "Superstar:en-US:local", - "Tara:en-IN:local", - "Tessa:en-ZA:local", - "Thomas:fr-FR:local", - "Tina:sl-SI:local", - "Tingting:zh-CN:local", - "Trinoids:en-US:local", - "Vani:ta-IN:local", - "Veena:en-IN:local", - "Victoria:en-US:local", - "Whisper:en-US:local", - "Wobble:en-US:local", - "Xander:nl-NL:local", - "Yelda:tr-TR:local", - "Yuna:ko-KR:local", - "Yuri:ru-RU:local", - "Zarvox:en-US:local", - "Zosia:pl-PL:local", - "Zuzana:cs-CZ:local" - ], - "win": [ - "Microsoft David - English (United States):en-US:local", - "Microsoft Mark - English (United States):en-US:local", - "Microsoft Zira - English (United States):en-US:local", - "Microsoft David Desktop - English (United States):en-US:local", - "Microsoft Zira Desktop - English (United States):en-US:local", - "Microsoft Hazel - English (United Kingdom):en-GB:local", - "Microsoft Hazel Desktop - English (Great Britain):en-GB:local", - "Microsoft George - English (United Kingdom):en-GB:local", - "Microsoft Susan - English (United Kingdom):en-GB:local", - "Microsoft Catherine - English (Australia):en-AU:local", - "Microsoft James - English (Australia):en-AU:local", - "Microsoft Linda - English (Canada):en-CA:local", - "Microsoft Richard - English (Canada):en-CA:local", - "Microsoft Sean - English (Ireland):en-IE:local", - "Microsoft Heera - English (India):en-IN:local", - "Microsoft Ravi - English (India):en-IN:local", - "Microsoft Hedda - German (Germany):de-DE:local", - "Microsoft Hedda Desktop - German:de-DE:local", - "Microsoft Katja - German (Germany):de-DE:local", - "Microsoft Stefan - German (Germany):de-DE:local", - "Microsoft Hortense - French (France):fr-FR:local", - "Microsoft Hortense Desktop - French:fr-FR:local", - "Microsoft Julie - French (France):fr-FR:local", - "Microsoft Paul - French (France):fr-FR:local", - "Microsoft Helena - Spanish (Spain):es-ES:local", - "Microsoft Helena Desktop - Spanish (Spain):es-ES:local", - "Microsoft Laura - Spanish (Spain):es-ES:local", - "Microsoft Pablo - Spanish (Spain):es-ES:local", - "Microsoft Sabina - Spanish (Mexico):es-MX:local", - "Microsoft Sabina Desktop - Spanish (Mexico):es-MX:local", - "Microsoft Raul - Spanish (Mexico):es-MX:local", - "Microsoft Cosimo - Italian (Italy):it-IT:local", - "Microsoft Elsa - Italian (Italy):it-IT:local", - "Microsoft Elsa Desktop - Italian (Italy):it-IT:local", - "Microsoft Daniel - Portuguese (Brazil):pt-BR:local", - "Microsoft Maria - Portuguese (Brazil):pt-BR:local", - "Microsoft Maria Desktop - Portuguese(Brazil):pt-BR:local", - "Microsoft Helia - Portuguese (Portugal):pt-PT:local", - "Microsoft Huihui - Chinese (Simplified, PRC):zh-CN:local", - "Microsoft Huihui Desktop - Chinese (Simplified):zh-CN:local", - "Microsoft Kangkang - Chinese (Simplified, PRC):zh-CN:local", - "Microsoft Yaoyao - Chinese (Simplified, PRC):zh-CN:local", - "Microsoft Anna - English (United States):en-US:local", - "Microsoft Frank - Dutch (Netherlands):nl-NL:local", - "Microsoft Adam - Polish (Poland):pl-PL:local", - "Microsoft Paulina - Polish (Poland):pl-PL:local", - "Microsoft Paulina Desktop - Polish:pl-PL:local", - "Microsoft Jakub - Czech (Czech Republic):cs-CZ:local", - "Microsoft Stefanos - Greek (Greece):el-GR:local", - "Microsoft Asaf - Hebrew (Israel):he-IL:local", - "Microsoft Naayf - Arabic (Saudi):ar-SA:local", - "Microsoft Tolga - Turkish (Turkey):tr-TR:local", - "Microsoft Sarah Mobile - English (Great Britain):en-GB:local" - ], - "lin": [ - "Afrikaans:af:local", - "Amharic:am:local", - "Aragonese:an:local", - "Arabic:ar:local", - "Assamese:as:local", - "Azerbaijani:az:local", - "Bashkir:ba:local", - "Belarusian:be:local", - "Bulgarian:bg:local", - "Bengali:bn:local", - "Bishnupriya Manipuri:bpy:local", - "Bosnian:bs:local", - "Catalan:ca:local", - "Cherokee:chr-US-QAAA-X-WEST:local", - "Chinese (Mandarin, latin as English):cmn:local", - "Chinese (Mandarin, latin as Pinyin):cmn-LATN-PINYIN:local", - "Czech:cs:local", - "Chuvash:cv:local", - "Welsh:cy:local", - "Danish:da:local", - "German:de:local", - "Greek:el:local", - "English (Caribbean):en-029:local", - "English (Great Britain):en-GB:local", - "English (Scotland):en-GB-SCOTLAND:local", - "English (Lancaster):en-GB-X-GBCLAN:local", - "English (West Midlands):en-GB-X-GBCWMD:local", - "English (Received Pronunciation):en-GB-X-RP:local", - "English (America):en-US:local", - "English (America, New York City):en-US-NYC:local", - "Esperanto:eo:local", - "Spanish (Spain):es:local", - "Spanish (Latin America):es-419:local", - "Estonian:et:local", - "Basque:eu:local", - "Persian:fa:local", - "Persian (Pinglish):fa-LATN:local", - "Finnish:fi:local", - "French (Belgium):fr-BE:local", - "French (Switzerland):fr-CH:local", - "French (France):fr-FR:local", - "Gaelic (Irish):ga:local", - "Gaelic (Scottish):gd:local", - "Guarani:gn:local", - "Greek (Ancient):grc:local", - "Gujarati:gu:local", - "Hakka Chinese:hak:local", - "Hawaiian:haw:local", - "Hebrew:he:local", - "Hindi:hi:local", - "Croatian:hr:local", - "Haitian Creole:ht:local", - "Hungarian:hu:local", - "Armenian (East Armenia):hy:local", - "Armenian (West Armenia):hyw:local", - "Interlingua:ia:local", - "Indonesian:id:local", - "Ido:io:local", - "Icelandic:is:local", - "Italian:it:local", - "Japanese:ja:local", - "Lojban:jbo:local", - "Georgian:ka:local", - "Kazakh:kk:local", - "Greenlandic:kl:local", - "Kannada:kn:local", - "Korean:ko:local", - "Konkani:kok:local", - "Kurdish:ku:local", - "Kyrgyz:ky:local", - "Latin:la:local", - "Luxembourgish:lb:local", - "Lingua Franca Nova:lfn:local", - "Lithuanian:lt:local", - "Latgalian:ltg:local", - "Latvian:lv:local", - "Māori:mi:local", - "Macedonian:mk:local", - "Malayalam:ml:local", - "Marathi:mr:local", - "Malay:ms:local", - "Maltese:mt:local", - "Myanmar (Burmese):my:local", - "Norwegian Bokmål:nb:local", - "Nahuatl (Classical):nci:local", - "Nepali:ne:local", - "Dutch:nl:local", - "Nogai:nog:local", - "Oromo:om:local", - "Oriya:or:local", - "Punjabi:pa:local", - "Papiamento:pap:local", - "Klingon:piqd:local", - "Polish:pl:local", - "Portuguese (Portugal):pt:local", - "Portuguese (Brazil):pt-BR:local", - "Pyash:py:local", - "Lang_Belta:qdb:local", - "Quechua:qu:local", - "K'iche':quc:local", - "Quenya:qya:local", - "Romanian:ro:local", - "Russian:ru:local", - "Russian (Latvia):ru-LV:local", - "Sindhi:sd:local", - "Shan (Tai Yai):shn:local", - "Sinhala:si:local", - "Sindarin:sjn:local", - "Slovak:sk:local", - "Slovenian:sl:local", - "Lule Saami:smj:local", - "Albanian:sq:local", - "Serbian:sr:local", - "Swedish:sv:local", - "Swahili:sw:local", - "Tamil:ta:local", - "Telugu:te:local", - "Thai:th:local", - "Turkmen:tk:local", - "Setswana:tn:local", - "Turkish:tr:local", - "Tatar:tt:local", - "Uyghur:ug:local", - "Ukrainian:uk:local", - "Urdu:ur:local", - "Uzbek:uz:local", - "Vietnamese (Northern):vi:local", - "Vietnamese (Central):vi-VN-X-CENTRAL:local", - "Vietnamese (Southern):vi-VN-X-SOUTH:local", - "Chinese (Cantonese):yue:local", - "Chinese (Cantonese, latin as Jyutping):yue:local" - ] -} diff --git a/pythonlib/camoufox/warnings.yml b/pythonlib/camoufox/warnings.yml index 3c81cd531..da49b9a17 100644 --- a/pythonlib/camoufox/warnings.yml +++ b/pythonlib/camoufox/warnings.yml @@ -17,12 +17,11 @@ header-ua: >- viewport: >- Manually setting screen & window properties is not recommended. Screen dimensions are randomly generated within Camoufox - based on the provided screen constraints. See here: - https://github.com/daijro/camoufox/tree/main/pythonlib#browserforge-integration. + based on the provided screen constraints. custom_fingerprint: >- Passing your own fingerprint is not recommended. - BrowserForge fingerprints are automatically generated within Camoufox + Fingerprints are automatically generated within Camoufox based on the provided `os` and `screen` constraints. proxy_without_geoip: >- @@ -37,6 +36,11 @@ no_region: >- Because you did not pass in a locale region, Camoufox will generate one for you. This can cause suspicion if your IP does not match your locale region. +instant_animations: >- + instantAnimations makes every finite animation finish at once, so Playwright + never waits on one. A page can see it: getComputedTiming() reports a duration + of 0 where stock Firefox reports the real one. + block_webgl: >- Disabling WebGL is not recommended. Many WAFs will check if WebGL is enabled. @@ -57,3 +61,10 @@ max_touch_points: >- `(pointer: coarse)` false and `ontouchstart` absent, exactly as a real one does. The rest of your fingerprint is not adjusted to suit, so a device that claims a digitizer but reports a screen size no touchscreen laptop ships with is still inconsistent. + +fallback: |- + {what} failed, so this identity uses {instead} instead. + A substitute is not drawn to match the rest of the identity, and a page may be able to tell. + Please report this at https://github.com/daijro/camoufox/issues/new and include: + + {report} diff --git a/pythonlib/camoufox/webgl.py b/pythonlib/camoufox/webgl.py new file mode 100644 index 000000000..d31c6bc03 --- /dev/null +++ b/pythonlib/camoufox/webgl.py @@ -0,0 +1,194 @@ +"""WebGL identities, drawn from the Firefox devices fpgen has recorded. + +Everything a page can read from WebGL -- vendor, renderer, context attributes, +extensions, parameters and shader precisions, for WebGL1 and WebGL2 -- comes +from one recorded device. fpgen's `webgl` node is conditioned on the GPU, and +`webgl2` on the GPU and the `webgl` chosen for it, so a WebGL2 limit never +contradicts its WebGL1 counterpart. + +Every draw takes one `random.Random`, in a fixed order (GPU, then webgl, then +webgl2), so a seeded identity always presents the same device. +""" + +from functools import lru_cache +from random import Random +from typing import Any, Dict, FrozenSet, Optional, Tuple + +import orjson + +from .coherence import gpu_fits_os +from .fingerprints import _FPGEN_OS, gpu_screen_is_plausible, is_software_renderer + +# Extensions a release Firefox never exposes (draft extensions behind +# webgl.enable-draft-extensions, or mobile-only): fpgen's corpus carries some +# of them, and a spoofed list that names one is a tell on its own. Measured on +# stock Firefox 152.0.4 on real Windows 11 (ANGLE D3D11) 2026-09-16: none of +# these four are exposed. WEBGL_provoking_vertex is NOT in this set: stock +# Firefox on Apple GPUs and on Windows does expose it. +_NEVER_EXPOSED_EXTENSIONS = frozenset( + { + 'WEBGL_multi_draw', + 'WEBGL_clip_cull_distance', + 'EXT_texture_norm16', + 'WEBGL_compressed_texture_etc1', + } +) + +# OVR_multiview2 is a RELEASE extension whose availability depends on the +# graphics backend. On Windows, Firefox renders WebGL through ANGLE's D3D11 +# backend, which implements multiview on every D3D11 GPU: stock 152.0.4 on a +# Windows 11 host exposes it on WebGL2 (MAX_VIEWS_OVR=4, headless and headed), +# and fpgen records it on ~99% of Windows WebGL2 devices -- filtering it there +# was a leak. On Linux it depends on the host GL driver (stock on an NVIDIA box +# does not expose it), and ClientWebGLContext::IsSupported answers from the +# spoofed list without asking the host, so a Linux identity could advertise an +# extension the GPU cannot back; it stays filtered off Windows. +_HOST_DEPENDENT_EXTENSIONS = frozenset({'OVR_multiview2'}) + +# What Firefox reports under privacy.resistFingerprinting. A Camoufox identity +# presents none of RFP's other changes (UTC, rounded windows), so "Mozilla" +# beside them names a browser that does not exist. +_RFP_RENDERER = 'Mozilla' + + +def _filtered_extensions(target_os: str) -> FrozenSet[str]: + if target_os == 'win': + return _NEVER_EXPOSED_EXTENSIONS + return _NEVER_EXPOSED_EXTENSIONS | _HOST_DEPENDENT_EXTENSIONS + + +@lru_cache(maxsize=None) +def _lookup_index(node: str) -> Dict[str, str]: + """fpgen's lookup index for every value of `node`, keyed by its JSON.""" + from fpgen.utils import _lookup_possibilities + + return _lookup_possibilities(node, casefold=False) + + +def _pin(node: str, value: Any) -> Tuple[str, str]: + """Evidence fixing `node` to exactly `value`. + + A dict passed to fpgen as a condition is flattened into one condition per + leaf, and each leaf replaces the node's evidence, so only the last one + applies ("Mesa" and "AMD" Radeon HD 3200 would come back mixed). Pinning + the value's own lookup index is exact. + """ + return node, _lookup_index(node)[orjson.dumps(value).decode()] + + +@lru_cache(maxsize=None) +def _trace(target: str, target_os: str, pinned: Tuple[Tuple[str, str], ...] = ()) -> Tuple[Any, ...]: + """fpgen's distribution of `target` for Firefox on `target_os`, in its order.""" + import fpgen + + return tuple( + fpgen.trace( + target=target, + browser='Firefox', + os=_FPGEN_OS[target_os], + __evidence__={node: {index} for node, index in pinned}, + ) + ) + + +def _choose(rng: Random, results: Tuple[Any, ...]) -> Any: + return rng.choices(results, weights=[result.probability for result in results])[0].value + + +def firefox_gpus(target_os: str) -> FrozenSet[Tuple[str, str]]: + """Every (vendor, renderer) that fpgen has seen Firefox report on this OS. + + A GPU outside this set has no recorded WebGL parameters behind it, so an + identity naming it could only borrow another device's. + """ + return frozenset( + (result.value['vendor'], result.value['renderer']) + for result in _trace('gpu', target_os.lower()) + ) + + +def _context_config(prefix: str, webgl: Dict[str, Any], target_os: str) -> Dict[str, Any]: + blocked = _filtered_extensions(target_os) + return { + f'{prefix}:contextAttributes': webgl['contextAttributes'], + f'{prefix}:supportedExtensions': [ + extension for extension in webgl['supportedExtensions'] if extension not in blocked + ], + f'{prefix}:parameters': {pname: param['value'] for pname, param in webgl['params'].items()}, + f'{prefix}:shaderPrecisionFormats': { + f"{entry['shaderType']},{entry['precisionType']}": entry['shaderPrecisionFormat'] + for entry in webgl['shaderPrecisionFormats'] + }, + } + + +def to_config(webgl: Dict[str, Any], webgl2: Any, target_os: str) -> Dict[str, Any]: + """fpgen's `webgl` and `webgl2` values as Camoufox config keys. + + `webgl2` is `[]` for a device without WebGL2. + """ + config = { + 'webGl:vendor': webgl['vendor'], + 'webGl:renderer': webgl['renderer'], + **_context_config('webGl', webgl, target_os), + 'webGl2Enabled': bool(webgl2), + } + if webgl2: + config.update(_context_config('webGl2', webgl2, target_os)) + # The values are fpgen's cached objects; the caller gets its own copy. + return orjson.loads(orjson.dumps(config)) + + +def _webgl_config(target_os: str, gpu: Dict[str, str], rng: Random) -> Dict[str, Any]: + gpu_pin = _pin('gpu', gpu) + webgl = _choose(rng, _trace('webgl', target_os, (gpu_pin,))) + webgl2 = _choose(rng, _trace('webgl2', target_os, (gpu_pin, _pin('webgl', webgl)))) + return to_config(webgl, webgl2, target_os) + + +def webgl_for_gpu(target_os: str, vendor: str, renderer: str, seed: Optional[int] = None) -> Dict[str, Any]: + """The WebGL config of a device with this GPU, as Firefox on `target_os` reports it. + + Raises ValueError for a GPU fpgen has never seen Firefox report on that OS: + it has no recorded parameters, and another device's would contradict it. + """ + if (vendor, renderer) not in firefox_gpus(target_os): + raise ValueError( + f'No recorded WebGL data for vendor {vendor!r} and renderer {renderer!r} ' + f'from Firefox on {_FPGEN_OS[target_os]}. Possible pairs: ' + f'{sorted(firefox_gpus(target_os))}' + ) + return _webgl_config(target_os, {'vendor': vendor, 'renderer': renderer}, Random(seed)) + + +def sample_webgl_for_screen( + target_os: str, + width: Optional[int] = None, + height: Optional[int] = None, + seed: Optional[int] = None, +) -> Dict[str, Any]: + """Draw a GPU for a synthetic identity, weighted as fpgen records Firefox + on `target_os`, and its WebGL config. + + Only GPUs the rest of the identity can stand beside are drawn: never a + software rasteriser, a GPU the OS cannot report, the resistFingerprinting + mask, or a discrete GPU behind a netbook panel (see gpu_screen_is_plausible). + The screen is left alone: it has already been reconciled with the real + display and the window (#499). + + Software rasterisers cost fidelity -- real users do run without working + drivers -- but "no consumer machine reports llvmpipe" is a live, standard + check on a string every fingerprint script reads (sundial, 2026-09-14). + """ + candidates = tuple( + result + for result in _trace('gpu', target_os) + if not is_software_renderer(result.value['renderer']) + and result.value['renderer'] != _RFP_RENDERER + and gpu_fits_os(result.value['renderer'], target_os) + and gpu_screen_is_plausible(result.value['renderer'], width, height) + ) + if not candidates: + raise ValueError(f'No recorded {target_os} GPU fits a {width}x{height} screen') + rng = Random(seed) + return _webgl_config(target_os, _choose(rng, candidates), rng) diff --git a/pythonlib/camoufox/webgl/__init__.py b/pythonlib/camoufox/webgl/__init__.py deleted file mode 100644 index ca911a319..000000000 --- a/pythonlib/camoufox/webgl/__init__.py +++ /dev/null @@ -1,3 +0,0 @@ -from .sample import sample_webgl - -__all__ = ['sample_webgl'] diff --git a/pythonlib/camoufox/webgl/sample.py b/pythonlib/camoufox/webgl/sample.py deleted file mode 100644 index 0227ecf9b..000000000 --- a/pythonlib/camoufox/webgl/sample.py +++ /dev/null @@ -1,166 +0,0 @@ -import sqlite3 -from pathlib import Path -from typing import Dict, List, Optional, Tuple - -import numpy as np -import orjson - -from camoufox.pkgman import OS_ARCH_MATRIX - -# Get database path relative to this file -DB_PATH = Path(__file__).parent / 'webgl_data.db' - -# Extensions a release Firefox never exposes (draft extensions behind -# webgl.enable-draft-extensions, or mobile-only): some database rows carry -# them, and a spoofed list that names one is a tell on its own. Measured on -# stock Firefox 152.0.4 on real Windows 11 (ANGLE D3D11) 2026-09-16: none of -# these four are exposed. WEBGL_provoking_vertex is NOT in this set: stock -# Firefox on Apple GPUs and on Windows does expose it. -_NEVER_EXPOSED_EXTENSIONS = frozenset( - { - 'WEBGL_multi_draw', - 'WEBGL_clip_cull_distance', - 'EXT_texture_norm16', - 'WEBGL_compressed_texture_etc1', - } -) - -# OVR_multiview2 is a RELEASE extension whose availability depends on the -# graphics backend. On Windows, Firefox renders WebGL through ANGLE's D3D11 -# backend, which implements multiview on every D3D11 GPU: stock 152.0.4 on -# a Windows 11 host exposes it on WebGL2 (MAX_VIEWS_OVR=4, headless and headed), and the -# recorded corpus has it on 13 of the 15 Windows rows with WebGL2 (never on -# WebGL1) -- filtering it there was a leak. On Linux it depends on -# the host GL driver (stock on an NVIDIA box does not expose it), and -# ClientWebGLContext::IsSupported answers from the spoofed list without asking -# the host, so a Linux identity could advertise an extension the GPU cannot -# back; it stays filtered off Windows. -_HOST_DEPENDENT_EXTENSIONS = frozenset({'OVR_multiview2'}) - - -def _filtered_extensions(os: str) -> frozenset: - if os == 'win': - return _NEVER_EXPOSED_EXTENSIONS - return _NEVER_EXPOSED_EXTENSIONS | _HOST_DEPENDENT_EXTENSIONS - - -def _load_webgl_data(data_str: str, os: str) -> Dict[str, str]: - data = orjson.loads(data_str) - blocked = _filtered_extensions(os) - for key in ('webGl:supportedExtensions', 'webGl2:supportedExtensions'): - exts = data.get(key) - if isinstance(exts, list): - data[key] = [e for e in exts if e not in blocked] - return data - - -def sample_webgl( - os: str, vendor: Optional[str] = None, renderer: Optional[str] = None, seed: Optional[int] = None -) -> Dict[str, str]: - """ - Sample a random WebGL vendor/renderer combination and its data based on OS probabilities. - Optionally use a specific vendor/renderer pair. - - Args: - os: Operating system ('win', 'mac', or 'lin') - vendor: Optional specific vendor to use - renderer: Optional specific renderer to use (requires vendor to be set) - - Returns: - Dict containing WebGL data including vendor, renderer and additional parameters - - Raises: - ValueError: If invalid OS provided or no data found for OS/vendor/renderer - """ - # Check that the OS is valid (avoid SQL injection) - if os not in OS_ARCH_MATRIX: - raise ValueError(f'Invalid OS: {os}. Must be one of: win, mac, lin') - - # Connect to database - conn = sqlite3.connect(DB_PATH) - cursor = conn.cursor() - - if vendor and renderer: - # Get specific vendor/renderer pair and verify it exists for this OS - cursor.execute( - f'SELECT vendor, renderer, data, {os} FROM webgl_fingerprints ' # nosec - 'WHERE vendor = ? AND renderer = ?', - (vendor, renderer), - ) - result = cursor.fetchone() - - if not result: - raise ValueError(f'No WebGL data found for vendor "{vendor}" and renderer "{renderer}"') - - if result[3] <= 0: # Check OS-specific probability - # Get a list of possible (vendor, renderer) pairs for this OS - cursor.execute( - f'SELECT DISTINCT vendor, renderer FROM webgl_fingerprints WHERE {os} > 0' # nosec - ) - possible_pairs = cursor.fetchall() - raise ValueError( - f'Vendor "{vendor}" and renderer "{renderer}" combination not valid for {os.title()}.\n' - f'Possible pairs: {", ".join(str(pair) for pair in possible_pairs)}' - ) - - conn.close() - return _load_webgl_data(result[2], os) - - # Get all vendor/renderer pairs and their probabilities for this OS - cursor.execute( - f'SELECT vendor, renderer, data, {os} FROM webgl_fingerprints WHERE {os} > 0' # nosec - ) - results = cursor.fetchall() - conn.close() - - if not results: - raise ValueError(f'No WebGL data found for OS: {os}') - - # Drop pairs this OS cannot report before sampling. webgl_data.db weights - # each pair per OS, and its macOS column carries a Braswell Atom IGP - # ("Intel(R) HD Graphics 400") at 7.4% and a desktop PC card ("Radeon R9 200 - # Series") at 3.7% -- neither shipped in any Mac, so ~11% of macOS - # identities were drawing a GPU that would contradict the rest of the - # identity the moment a page read the renderer string beside the platform. - # Filtering here rather than repairing later keeps reported and sampled - # WebGL parameters from the same recorded device. - from ..coherence import gpu_fits_os - - coherent = [row for row in results if gpu_fits_os(row[1], os)] - if coherent: - results = coherent - - # Split into separate arrays - _, _, data_strs, probs = map(list, zip(*results)) - - # Convert probabilities to numpy array and normalize - probs_array = np.array(probs, dtype=np.float64) - probs_array = probs_array / probs_array.sum() - - # Sample based on probabilities - # Seeded so the same identity always draws the same device (#442/#765). - idx = np.random.default_rng(seed).choice(len(probs_array), p=probs_array) - - # Parse the JSON data string - return _load_webgl_data(data_strs[idx], os) - - -def get_possible_pairs() -> Dict[str, List[Tuple[str, str]]]: - """ - Get all possible (vendor, renderer) pairs for all OS, where the probability is greater than 0. - """ - # Connect to database - conn = sqlite3.connect(DB_PATH) - cursor = conn.cursor() - - # Get all vendor/renderer pairs for each OS where probability > 0 - result: Dict[str, List[Tuple[str, str]]] = {} - for os_type in OS_ARCH_MATRIX: - cursor.execute( - 'SELECT DISTINCT vendor, renderer FROM webgl_fingerprints ' - f'WHERE {os_type} > 0 ORDER BY {os_type} DESC', # nosec - ) - result[os_type] = cursor.fetchall() - - conn.close() - return result diff --git a/pythonlib/camoufox/webgl/webgl_data.db b/pythonlib/camoufox/webgl/webgl_data.db deleted file mode 100644 index 58b531de393ec5524a4cb160b79e11cc648798a3..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 270336 zcmeHwYm6k1wOGKZ3>ktNfVWJ_v}VvB2ymi6kc?zNruE_XFrWmlwjDN+XEv}$H* zW}E8i?o{{8E-5nrVZaq%D?svZLoOAEk%L(o0&M zLI32ON1nTT^V*9yD=%KVe*0$S0{NAVFOlspZv5HCpKN?<*`M?M9=gFt==c(_)pAS8WKi~bG`16qu;Lpe2k3Ua;2mXBgefaae z8~F2y_u|hbqSrt9?fCPl_u$X7@5UeZas2Vu@u&J2{)FF#KMjKVKR^Wj4_(BcAASUX z9=*W0`HP=i!>|3Z7O)l=ZGpEx`}E`0H{bKv`uewiWV_ovXvdXXo!!foXFm7RtsA$l zUAp$UZFuiQYpcTNR<`3$caz zm%sUIzx$=HY{CJ!-M%6ndYe}^Pr2pIEwM@NFvJG44qvUc+wBbEhl6W_LDJee8pQq0 zD__}++P(YH=9NKm6mM-tok1&Vw<0XzGxp-%;J$3^N3Hg){X5aC-Q58=&!cfA7E3hwrIa0&$@rq>vfYsym#{v-MpEhX6aU|7g3egX4IGZ8-gh7W%H< zw(sIZ#D07iTT#>4KtL;#A>~Xmq?*Rjs%hAYtsK(TtNC@-sMA6DURVpa*k9Ur;NV%w}4Fgu?Y@}~k9*fB1# z;~3Z2rL_aU>Y=OOpe3)kTX+rwb{jPuSchit3rlwo)?(K_U^lkB69OmDcNrSm3V}n% zD69}TA)TDKeS)B7tXHS4S6w`reo%L@fIo25zy@>!=cyWW@#LoKRc+v)Ueyl(8pg5J ztJsb1V;(@&IM8|(`%$aWt3)--yCP_KWaAngGd-Y2n?ge$3mW=Z zs=m3JZ`@&6qBo6(v1#AjnxEd9KAEat*C$i0RgIn23}Z_B%XFn;gsa+1DKU9Eg3?T;pl2d9)9?#5ZpAnol=+;56{1 z^3lLS`s`S6Gr(SWLfHJeu@|4-^lDoTegD?NhBi{@(xWd{3ti)8M#ESV1h_8p%@GCx zE{)N`QR8k2=pg7Hy}^LaVboZf)`g9x@WQ1n>4U8YWE485H{{5Gk3le)-fhu<5qJYy z{yvS`{c6qkw_sCx8GdCmiJ(Hb)9S#e9zN87I#(I~fAnvz;n)6H3s?(S3!J|NzWSvL z>rYkJA6x(ZuUzZ(+A;f6x#MnCAb0Aw4qNRgx%$6@U;Wqb{f&Q~5X*t=)+~6ALDqHk$LBmM&_MM8ku)4V`SE| zl#zMQVn&t@H}A@lp+)b?2#%Z}T_(p$3-X{pNxhnUb`c=AXAq~HwT_83H+f?YIThDD zmfc0yvCB`({uZNjy(@Fg>6%~8bHF+r-JK1mOXC+t=<*4S3HSuwe&pt7R4$uCJT9LQ z{3e!(b%?>8Q)J_vAuCJP06&XYJ)a_m$8!Q=o**oMB_(shH3zzNR(!0BomWGmjySA= zU;sbVoF1^@bmRm+cB-(3vHFaQ70&_ZV(Gj%5i6#|txG20R|Hey+F%N|C|{3=M?Q6W z+ErX_PP=}JQI3ps1*7D1#+6f?jBk~b@RV@Wz_?a9iKDMot3L6*=!rA%YBevQ14R=D zZ#0Q+^{Q-5#EwJ)Zn&q>6O5Ku(^{AD#kxdn^&Ad%`vX#{3{N^9L}>*c<#=)2IV{B-ZMK} z@NxoPt`>N?*`c%qdt6QThSmR{Kiz4MU?D8<_Gc@n)c-fmO#eT2%lHlsao<+|Z}tCH z{|_%{zdsZoa9R(L)&G~_#Z7Uk$B+|KkX!vf;L5D^|C2r?mP60Wp%c_5jROJ#tp1<; zR#yL?c1cn4mg-b3jxQ|2_!%6Q|Y3CJC7i9-n%k77n8z+{Js26?I6f9ye z+G82`0zHSw;aokrD+vxs)8&@e*~dGVH1hPFSou$Qn1L_QvzU=B13w|?Jd1CiV*Ge< zI{x{W?lxI4|8XmR;7#J+CoMlMI$J7!J`I`*Ymw94i))b=fuENlvi@8}SWo)krx4ak zarx#wDd_z4tV~M3EY2dW=4^wk7n{{*(aN0F7n7i6IoZxweK~oO)pxx{oe2dvtB-(T zhcV+UhlzN(FYZ{J)C) z&NVh?nuSuMdfv0xs2;Z$;?ym4qQLxImxq~(^fD&OoX}H2r!4;e=hiO%+(K}$_8_eV zR>1;q{l$+x?!Wmi2Ye<;sj z&~z!!ApXgbXRy>}Cx{IAajTVigj6rKGCZr7*3Np%NWbVa+BC3EF%WxHz^?%vm{p>_-S<-xA>okLRN$- ze|0sp=iF9Tn-9RB;~TI@d?-nNc|kQXh168@SWry@cFTv_a%#gF{#K9=bw$tsXd3d8 z;UR`ZM8_2KZ~(#i*8K5V{={lLAV4?t))20Iio|}|L*OxdXpT4RcpB8wpE4t!jht3) zl43F23q*e^_|G6B&;PgfQ)~2l&Og7Hd<6N`o&a3``X^Fvfn4>~>vhLrMb^^=)`8l& z@L7TPD63-F*g&Jnn^qt_8_A0kP-_)}@_blsk-RM}CK-B-01^4Z)H11+1=f^lIvTsNQhscdkpUhHq37cb&L2#wtHJZ{X|axl|lTvfcl~$ zemztkg8Zj>-&*{yaBVJSu9ZJ8ulzX^|Np(UHT=%q&o6%IdqfoU*Plpv1h`CyMb}oI zDH0q}z!^|#vkKy8I|7uviRWW4yO7nN~H% z=ThFUK0kO~7s&}=Y5%asQqcu<7m5en0P&+5RUM*_(tC)}7cMn4bonwgkPS{R(V@p! zqB&GFuphk|_M;j$i@MFCek2p8>zhesQP&{=i|PbW!{i|!K-nR=#s3!nTlqi3oL(0G zzxahU{MsLD0c(M2Sm5o-cR&8@n-`#D`PP#uzty!nH)QTUOm0=~MtgC$Q@PuypjK|> zUYxYz;mmziE?~hwGWukUfThGnCwQ#rh}9~E{8d)Ba^~);DOPrvRe*~1W<4AnB{3c; zp8dkinxSSE%S~~m7bVeQJcyG%(e9wQMGanEAWI8Y=s6}0U$V?TVh&O zD$EEJD&oFmu`B0Tx1!u9WRaX%i=t?eHvvjiXp}$OdKJp&WZhZXQHKN;+EK@|SqM8- zSv?Ecs2TpeYR&NHp`rbG)!OKS7II4=z2Y`sj(4_iSGI58yIr|?ud*_l7VXZWI&YNV zk3MgU6vZOO&_m9waxdCxy0QRu-b-+&a}eL5+5>rG9+*7D$1`9iZY$OqV~+BjasAEC zR?;HlcRU|e9&{~@l1Ap~Fv=L&X7p>f;=OC_-u*}pNvXzooE*nDlIQ{A&aWTs@5hN6 zHunc{XSdZBZCpI=qZ^e@p41P!-F7!=?y&9HVt_jeGwsDYM+eO$?(D@$oFM1^es^dY z&vDvUbBm!Y&%!KuPy0>)j3*#v-7OaLfv1rDw9WB^n>mOTZa=LD=$pUnZt z%+RP|6h)CQRh9}}zVy(-{5h=yI2kd?)0oRTfM)=gZ2mth0I&jpAprpS%z6C(h2LAl zul-qG3w-^(A4YD4n_#5>B%ND<)6fR*FKHde$)&6NpZw)-{@U+;X>{I& zGubV09=qYMWQSsi6IW|lZu`aNP8;yKdD!Y4Mh{__6Yhpo;(^nKV#sU;yWK-_lJr~M zPIF3xHXU03?)G(@1BFYhHn$!12fHxD$YKle#KrEo?tOoJe_Gz`CkJ^wHNC<8rnoiK zZu67mhez!}t4Zr`g3sp9{|eIMz5_zP>c9j=Q#+bHPwBT2>=kp}sA)~3`L5}}kt-Ch zdB+0(=?|UlbSmDx9Pj*v)&RF3JTN#&V&2fM;GgZ=d@jJB;N!rx2{+K;9@X4B?JqsW+W1c5N^Ro2LaVw?f^+atY16I+1>?U!D31 z19^3Wd_)5WqL$Q+)fyfwt+vHP9=hdY*}1detU{sRaA2KRTqZxk)pK!VzTc>mJpz`M zv`k)y={jqKeG!tO?TZGkj$UFA^!2K~$Il?9gB5)X8YHBVyKmI=?oqozSB>O_9K>q# zU)qk>v#XJksK#6_Aw3ZC@JEo~=EG-j*{&rGoyngM9dfpTGx_rZ zJ&PGw)6GkUeZ5lrd5T!ifppIpzsk*by2&9%MI6bRKnN7~Ui>iL6N}{f!vgDcN$Y3e!j&ZSO?Wp}H+_Q8 zH+!BE7&2;p3WUCZ2At17XBbGh&wzpCH$8h8D9|CkQ+$d5px|#I<;O=;Lq1aZ>(*HT zKvV1wU50}XQhL^?GUsK$Abl7C5(9kUB@%LotyPT>7MLy+98tPYJS;$a0Ajj9yLUW9 zLiij9>IRNdH@KXYug0{%vXAK;2pS%Y>C%XvC-}>S19Oo13~2e}VPMhX&@%&B*64o1 zBu*z375{KDwQAL;1KrgdtL1GAB-Fa@inZgtFp^@eA`isJUf za((00oEo-7Td%qF5VNh1D*0Cs`pKuhJl9Xhg>?UPB|SDhy#)W~h5P8D)dNzJ)8)`3 z;WBm5(*S4$7|GU57wC#tMheGxVG3+g(|~MB3C#D4^GtP4?-zHQ(bb~8LSp!ZLwhO# zfad=%KEH-v`(rI&EpR#uy#0M2LdMFEP~`sw{%xz=>g-+?Nh+nxqDUSamzS?_p{?j9 zm4556)sB*@yd_tw@;VUhf2WgZ8iDevxB17*K5K0QVdc3@WSRL@!h0|kvcf}@tpZ|{(<>mw%GoF&q-tU+(JJC* zhZ75~3EeOh2sWuD`GkaNY!(1ziVP>0FcSB3$p&+E$%b#FyU3C_Fo)aXWdTql<{^c{ zG)C&>)C_<+?(fIT)9ledX_RW?^(>q+Lyazn6#X#frBmvO$AJqn5!&YEUrNq zDLtRIcAPQ**eL%-4KEj-l^i+R@UFlI6BBsuO*YD75v;`!mEbJT1X0mfeHu~zDN(>; z<0l*PKgE!LiHKM-wLB63vcps$6w#&}KWW|{_=#?Mt{MDa>kio(+1{|)uWh8iKmHse)q!*3% zqtN?e?lXbg&cuBtwtWVI1gY28r8R>sHjmhAmT!SMH(a^US2>s)#fyJf*3PPCBgXDu}FGQls4 zDGs2Ikj~UVt-Jv+1$qwg*139emL2zb84(5VJU8Ni{3kpt4k*yGn31hGAXPFT4{*$q z3rrCMPEk#;0Dvl$n?RzZc~3+zYnZeMDg!xQo>Ne?Qpy3~AR0GvI3ydHn(LZ>88PjI zTPLfoHp+v5^yHkD2lz>w1<74-wBMCckh6||ATvxqG#aW%=hTTfh|kIjdc3VL~kh%h{vk5OMC5iUd_ z2Y)Hh2grA<0HlivwgQslVG;yU;5Cr@91x=%b)W9ODhxgf4AclF>cW@xEjEulA!o$k z=lEV&8b2?Mk2H>9+G{nE$d}i;5RtDsO(oVvP_#VUMK|CI&<(Jh@tBaKsz$TwfPg3lkGrLX^#XnimU z%*_TzDV0~jnQ&TP&7V_RKXb?|N&Xyf)R9cT9Gs^ae!!jYBFNMEG;)kbgQZLZkjtN9 z-CaC5MoS`CQ3*p6q4J4xjR9T}ra&Q=k!(yY0CNrazIMFAJ(=#gih&?_q^ zV6-zS{*zFNgWQva<>a+4K=7+BQ^83z!sZ}JJQ`@=p*~WXC@7;UJD>bGUAR zP~q~j0lI-sBdP|RDZNjKq71ojzBkJNAV%=BssJ3xvzdDVK z|J9~L8UOLmrK?~0>QBG=BhCMpLC3R1^rEW37D@~Z;`TFlFI7Hsqq3buz5A`*e&t3B z#YYGJ4cDz4v+c>vDjMEq_V4teih$qqmCc38>_0^x&1!S?^GwFeQ2SoeJ??(lCjMW& zID2w6gp8$U%r9f<34sPvjk7=HXT&o2c^Q1d<7ozelv_%=+X@l*s?$^)Q;Jx(`TjLE z&5K@2E}XfpRD&E>vcW1|>V{S@z>NjZm0H3oeexQaBz9uHow*Feca0>hA|8KemNwFq z_QsTvLF#bhIFXqidq`dV4S15?CmJgcQ_%u`xR0}40G2nuZ$KfNtP(sv1?95|5Qqnz9+T%}`vw>os_`F;4qCPUtX7 z-5*Q)OWSkwRHvkEZ?Ep^T2{H(tv!kJ->H*hoSZSi@*gq>HBg*;JbK zr-Oo|S;LzGb<)bQ{&cc3o?lGki23y(F@TEQA((<2MeYSHoD9vRW2S%9d7^<6*vueJ z1loapxL>|J(IG&S<^L7`pKDNVF;i|xyl|K)ga3c@udm_P{#XlG3s?)BlLfx|$$#K*vj7V?S~aF3=m3!-i7 z2ejeJ2u{5x!u*r$QE;c^@C z-W0AhVT@_$56FMGkh+6h;T@Siv;sYcDDhl9xibp4`4Z?3@}Kaq?w~-=Vn(*QgH-u| zllZT^?tq|N@tgs?AldH86afo>i&7#4jI(%k)+@GTTGZ|%Nu2M57)J<-^i0qW(O9fr z8lS*hlTQGq5myE7SkwAeZn!7M4tS=Is=@r_Plo0U=x$h`k7Nq}r* zDg}m7jY}A3GN$1K`U)roawSYlD+RcCnMnnRUX)q^%0mxT0BHrpB%bI7JwRM5ASdzU z4QT~xP<+tyh4t`&dAdnj0d^&z2O$0#>R+~AtqLq*wCJ()n1F$41^5BhNN8BV_;%O)kK3(JIhM!zyG6Ou;A$q2xv@S{}f*mzD=Ce*pl_4&(+cmj74$ zf3Cr?#V)y_X(s=l%gC9U=>k1VVwVj5|B=7FhF|+*EigA0c;`pI^YI^eGelVZ|Gkq^ zh}~)JCf$B_e{e|zFRB?^6*XD89`#$hmAi3gFHR~qlGZW(Gb$iHMK1ruw?6!%7CTc-j7=CTl;sSSG&mzak5XjKRdhe zbJ1}e4U}-N*L@HtFC_7PoWxiPkM3T)A04#^I1c26Jj7J1*KWmo*V?`Nk=!(v^yB0> zzL7)^5Vd^$XdnI6Hv0o?s?`>4=}cJ>u9^Zhqg^ z^rI{rJuymT&K);~^!OZtLiTQKn3gpsJ(>i1nD~)vQ z%rqijz4xc}kx6-mI34FYA;lvQ3iU!l=`72Fv&C}DV-D;_3&?g}-NSMTY0&~^W&P^Z zPZ-i;jHqq^_NqqRSgqmV>ZolsJglc%zHq~#Gz9j~_R%1oklYHG2mGwpB^?D{qegh2 ztq}^R9i$Zm_A!r~LF!0&5=nXGGyOwlBT2+YXB$i+cgo^PbOV=1H#Fsn1e5x~puC#q zYWi|v3GAshkU!Uct{@Y!-380FFPu&`RA1c~1wu~5C7;_+ef74bQ8w}1cuHjCiCQ+T zH}U-N=m2})LxArfzSHX9Wm$-f$LpdD$;>=U^cpui+nAKy7G)xXBiIKl($z3a8adM^ zmCMMPKB)pdhkQ~KdP)dbFpgi00OlFg+ARIeS)jQThdvRa*+71b3%L)-cLEIkLtsjqqoBZ{RI_oYTBLBgk!pKaIFDID3(7UC7l?Cg1BEaOu|gSl82Oh7!`Ef`I z{^;{5DFH7oBXtD4wN=Tevl-N#oa%v}nv}7B8hcSvfNafxI4!kSA;AELhx6=Ov3dck z7kD^0O5&M#n5U@YYg#t3@m+JyPlB;?A_m>ED* zpyv=>o~tJ(_m?ApCjSW!YZ40dEM{b@NtgfuWp+E6jvomvnmfG#<+d~7nz=YRII(zS zZaiBZpgr#i3}y{@7EhXRzVK$HbOo{qfUO8XfcP|8f?VJwttH@~z*YoMiT^S1m(;th z2*67-Kwbc=i0OdFaneG_z=pAz(Sn1A5D2+;=pQ8yMfLYc_A7zojG^pDb%6(gRmZ!cD15f2nr%6J%i;HMH&6+M{vhh?MafvXgR$ei1sNjRR$S81NR1}y{5L5OE=;;uYa8RdhSKS8l zOHp|am8`&p+Cw+yJB>juub5=PolwgQ!0&5gAn^cjoLr3+%hV0+=)h{)=x7*QJA*U^ zmeOZzeSpiPsQUo7CbJ9Ax2fjO7p1SLd^tdtTz=N;5~KW08tK!h90L@~@mnX^EQ4eU zxVkWKFcW(g=vmT9c4f&R<RQKo2WpKdHdmR3Q(OwX@=GLhAw|AE47jWQ;TKs140rScHfERnK*nU9LG(12bc z6(g*=9%aI)hE&L^h8P8EG^iXQFB6whOrXY&a&#>L@)!9v$HA^}Z}Mpk@~}WdMxKU^ z_E}>&*w|FMc!0Vj;{PwNJ$kte=V$v^3s?)Rjs?E9{l4|5@2o$z{>@LYk8tHy zXZLcJYjLa6O)CA?VXGY_SAXfhRet5CKmE5dV)Q3zi^w+fPx#*6+aioYcE;RF9!0w0E=XHN6y0yt%9es-Za%`YcU()>ZA zUKOlAhGCv=63y@NMXdgsklTXT$5>A{(5c4o3nTfnQO(pje!=8p#44%sToL`|E3*uq zVKhGjGweKTxfp*1T*d(!c^btSe-|*J2HfC;e&{r+)w8Gk3j-VlY%Kq8`G3p*kLLe3 z)*jth9TwA`skMN$K&A!0w!g6+-dSH?|7P>ri?=E-#=G}B$OM1Tiu>&Adlv+7w^O;> zsNi#7xrY$zxNrItGdTW9T|n;+ar~KkR;yXMzeKQq9(T^vQur6i>S*bH+}I<7Sjer_ z>?a31YNfq+=jcG_AfSY3b@sbM%ebqaavt!_TDreX5`Gh{d5z)@F2O!6ST6o^aCznU zJXpFvFWryp$0AuQ*AM6))G6{qJcvR4^yM|3Z z7O)nu7RYUZuQlGg{$!Oh{y$5VLuI*8Qx9Xu~&v#e^zXv7wD-O%thkRpDf(se5*YH96 z4U!}3-S8<;fK4FLU)A+lrhY6}D^mQ{aRz)!&Q|^sU z{XhK-0VBQ~Rwj^FCNP}+QnUl-WYT|meG6s%SDkFye;uN3)BcMbnNoUCE-e6izZS}r zDfY*cWy%JNo{N-($EQd&8u*ULM%d6PI78RhA@)2mrdW+<#*~e-sDx<_ z$uY*({QomF*cW@YVgD>hCVEJbXj_}8#kAY%qlh`i2grk<@*I>ZIL{LXV5m_z)c~(A zSu(c*?_4ss0`H7l+dSsHSsi%iGPxC4&r(L_J&PGxX1MTd*#Bw5{-=ohrkE9A0YFqL zsenwRHs`qm1jMN!Ne?g91ek>HoSMeVN=@V4!_gJqBcfk%XtW5~0_2!?-hj|br3gS` z0G1Gd5h zs^<|tA%S#k{J)L=xAFfYgw!7VfcFcm_ zE6-fJbK}yr&u!nn$tJfdSur-18!dQ@2L27#tsM88)uvPFv41X^-nXm&=-z*MPv^~_ z%^<|5vxAcq`iV19t5`n01fPC}dGe>|MOkh3d=}{-S2V63h1tTO?$-j;c`HMYr@9w+ z1qrTV!Q$$rM}Rwrd@rnpTkJ3SAW%OJIqhh|g)o8B<#1y236M#A0{EFv06nuL)qxA2 z8@wH$z2R`%ywiQX)ow?4w_^qn+~(9iCze;w%d1niy)s@UaZ^vKEGh3I%DbmH#-L85 zebt3<1(D1y7I5DyHE>_)2Dlj2sA`T9IW=)>$s=IZC3QoymskzE(M!w&pqz|)3HGBJ zHUJ+_v=M;MU1W2ZY~OsVOLE&RH4m`&{RH?te98^B+wp4u(=QmYQ%*ePKx#!AK^x1XV zvhwgx5lR*(y>UbAVrq z$HXr(*jg#>J~ag88UE9=GI{*6kc*0+&lYbn^uDk;{>fg*=1(lXn3R49p5vdnBru6l zP_0_^MN)qZYdqI$G(;DSHIOzobXW#`e{7C_iFJ|jAH7_U;6Zgb%fA7qW|rk&97-d^ zVKLHusBBs4o&{0@gdUzX7D>@lo5hm+LGBal=%N}o2y#mfA|xmHx0P;E>9-DB?I_8R zH{|K2lnKWu^xjRtV(HzgWmdINLbYHE0JOW&pxJJ9;wZtb^!&|xP3bqgAGPRv9p@P}74)7?~c8OU-A_MjxZP#@G zn_poG4AOsSq3`N#`=kI6`|*hdR@2x3K}8fw)H5j)f?UDZ{4Z7#D-)tUhXMdOzYA7{ zkXMC(FoKeWFU9RK$uA~u5-UR>ZIXlDnm$(jHUh{=gX9<3ozRM^EBcYRy!M!!Geej$bYTH1vdU4@(=Eio2IxRmyt8Y z1qFHzxy2^*RBkaVE;vp6f02@J#q|W|3ZIrrK45-eOUMV(-4!STTf2p7?2|#!BFG2M z0Rm-(6PeyLI2TgzXV$s^NyZ}29!9(pxT1xRgKiObLvi0ox+m}_rm%*XXFy)ud6-Xw zP)uPyUn2xq_?Xp@=c)kQX(G%&QxQ|3Z7O)mrTnoJQ{qI@-z&)hn|AQmsD-f9CGdC(eKGK!#BBX64hRS^3k(6`Gz3kvC0uclSsWX32NMQ%2Lc$BpV(wnu!*YpW z(ZVgs`qinQAh$eL|6MnLc~zrstadIQE^#E>y(JqN&MoJ%>s7Zr_`~Q<^BWE<^opxy z*!=x%m?dCW8ev}{`D22HN6tiYQ3b3p2#XmB8l(pjmPSK+P{8A3eX#_MMPm4QI82Qy znz%gVW)3O-;z=&W#VADTt7Mi#E-*EJPDy<}gqCc6zPD%TQ~`iGmkI8C#fbLD{az;i zf=Bzq?O(n#-pJ)UV>n`V&gFlxPA;AjQ_CJ$a(^SKg31KJeRE}ud`?a-%jTbk&7UGP zoFdu305~h96_DU*-ZKhRSvj@B0%@Dgf658l@_p@7SZUTm0D>MENuQ4nfYbokIY&vd zksl?wgqbEoq+B`x38kkafhLpkG;Bg2gNDGcm;yGT)5a9WW?~8hEiol@6j+L{GoMFE z*Z7o)lFI=halttd`VKLSjN28P?B9>}5OyvD+uPkex!~^hbulRy?sj=JIx3ttr|9spS^(I2 zWWY*1*d4ccp3?zHch;4a&;ppz`b!w+FC7@?pamHB5lK0>tPp@l{O5OZNoBxfFPEgg zXIjZp?sPsX4pv^|_`~K`L(kAcgcQC-4m)yhg+82EH54mhzAg&j zk}3)2_|lEk67&&EkP8z!8*)ry3hvF~zAw2U*AVQ~Sb|KsbOR1e**IP73)9jL2?55vI!mYkfEXM?4TsU7PyD&g`EOMKxw8S-P?sF0vMPXf zE8e@-?%j`M$|%YN7AME?jU;-|>Kt5W3B#0(u|J4AyREipzo9B1^pASIZZe4XZa%~s zePA~{xvwyUnSzrt@mO9YdB&j7ssP$6q5{Z`n@*$xFngX-LrsGWmK=>w2;S7E;#kDO zv9xGx5#>bV6GCg8&W5%KD9K6y^5Y{b0U&8`K`r#uB>-Lnit7*>5&{xj0#HK|L^tp; zlg}dwz|cG^0U+R11B9{?03;y5JAE?;D-MJf8@{B@N0jp1y;!d zZ@>Qd(2YBson+qlEV*A5o^q8Xnxj^TJrPL9Lbj;@=Fr<0DB$*zIeAqOd=#1KA1+W0sviN z2Gz6Z@EpP7JQo0z6y-bPb}%~^s62^4v64N9i2Pjpqn)hBZ)aqR+L*$=Cp>s#uKmXF*=HXF$&^m6#4?G2iq}F`KmCuS{uz6?b zjc;{{)cxjn-DAP2ETfhQrN-)hzq?fycoFkFThv9IEn@V+-iqf4jk6E&&xNki{616nvIzf>hJwiEjzbnqJsrK6 zB>gT6$Y;6!)s=_ zA1A}W%CM%{ct@APQ)8)pj*d=b`%f#JN8qV)(fzbBnc81WC%x6zG9%ncLsD_M*-|}Kk-q~U) zc9{`jdd0`{YUDa_-cVR7ga5zy-`DVKf2;+p1(wPJZ#}iX{?r#J0l+O*88lCM(5t@* z5w!0I8G?m8-GTXsv>yn`@C+0QhO_8tw2`g=3fEpEz zgFg*pM?L|cdg;`bf)86TrNYFDmxSU17jTccFPy_=HfU2d?xB1TOdHK}0}G z?K|DqTkUp)CmAzf>CK#9601STt3kjmo01@em*JVAIa#>?WEo4xa3)Ohhsi!D|EIuw zPXB%kUdV+z+)~1Rbd$6Kob8{_^#7DlAuIboFLE$i zEI_YVsS=RWFpCsbLo8u5Ry^|mbSTL3|BC<5HOjUih-VAXXmNQ;qh;{_kNl4{{MsLD zfu*y+J5N6Lxc}z&Qjq`4DbX4Gpj;AxJFKx)8Rc5b8R`F<7E1Y&eS4@hq6E8cd8`>Yj15@M@=5Epr9c{o}PI`&+_%RFkfGw=MY~%p{M2Rn^NS=Qq8ry(IAz4PD-I= zSpAesxS9FPAYtE&AI4NiZk*uShW!(ph@93K6No5Cui;}%Fi4JKT0oiRQAP(7kUv&k zpC)O{pjM_lL0o{$R)C}uD}Y5a!=P>0KWuZ7mR~mLwNm`Od4Jo;e@YSnTtfFarhqoY z6kaQQiryJ~itr2E3?UQeH#~a3=*<4PP*x5$-$wouvybaAyU2fb>3CVQK$c3NYg`ss z$)J1)oN*qyHNLzV-D5Kd{htx4vi!f||0j&SaoMW~!>3sMQsaHzv)Fi_@6&>+V)&C1 zZkVYOCR|trsWIp@J3Mq0CT@IeX2ZNZ}I=hQ2e z_mg5n@AHPHkB6zGo>aB^A%IRO$x2=DK0U09vL{n{v8JVdd3J z|6j&!mG{LvJ0SqTQJnzN+z0MYaBWzh)j@oRoJYm7W#YY1z<*>o=}I{UnR~CS&&mew zTOmLRxK<2<+5G<)D{;Mo?*w4lC0FExSYY%2XJ;~-mZK2fgcRhmB7h*2Q3Sr98PH!A zc-i#-4BA3n4GPJM074%lp<3Wj-i2zT29lg*A!v}mLcRiLEdropu^NCu+#V5>SrANF z{@?Qd932@M-<0f~D-ZD8n{a4pu~Y{CfAN>r@N0jp1*`?;%mQzH;a%(Q=aIqx51;+q zOSf*^x>k9voAkN~Qlob}V$ODqq@BBQW&6c1RvJiWUFjy3e(SK+j*_dt&CcgShWuc1 zXF!snOamtmL9JtT1toL^JpKA^J31svKHJB$pF{`neSmltt8h|Uve3Z~yN0 zbuoFy5h8IrK8!m99vspx7a)cCPB*!E3m|ZQ%&Qa>l>1CYuU5{W6}!17ZcLe@*57^ zudh&^0emd#iOHO#bq;Z(zOE=I* zH%4m-%$Kf6YB@Yh%QqlI(AgJi4#hc&eH79XkOLU!f=~qr%$^!9qQ#noIf4xNPM+5AoMFu_Z6y0GGaxU=Ir8nZ9EymytQuk3dE9 z+7{?J6xcGMC(czQR9`X(>W!)`M_6yS+wLYpbezf-kOxwpF*-cU`~h=-y;7@s z<2K|^2&O_2)s86|32Ony71Etx*~T-JpAh zMg$n&P%eZK*eLL;Y$sXR$iIwRi#AXM9I-uSN&@IYLLPwQqIHHP*5-@13@oCpb4r8jyq;aV!Eh604!Km=pS;)2LQGJf}EBnJRz|7t?qNxE`Cvn)=~K zKRm>8$7tbP2+5G)=pTgw(ZF|db80xr^8bqepD>!nl^7NOU!Z5P5j@`~2P=4Cn5dx` z3ttk8t`mZm?-%y*SlK|k)rq471~n5`rfiwLpqc7;94By>@e9`M z?z|fBlGZ_z{~1m=x#$p5!&Irss6SpeFbA%!<(3to0AMLSf#|m?LV`J|a%p~m@Wfda zLEd}>!*wph%P71!$`6*RbtyR5NiqW22aW)R*n>K06sj)r)`E?7u>fimo~Y7=|3)|9 zCsBUHGgc^YC718adcY`&L4H#WXA_vv(+UL&gUGoP3M?|XT14Hz%pubvCYxHv#g;^>FO0X0;E zU`)GDe7dKbBn}V&hLOt{m9zDjI6!JtAQeC_PCU<69l$HUWyk|+>?lX71B$~Ptp~`e z6F}w$s?mTI4cr+D*9_^T)qKJ`l9n%~2WWVZaD-4L1fdh+@jpvBKzaCM`G3p*&ocgh z0urG&p-^WJ=Ta-svzW+2p`Q%?|B>HW!>|3Z7FZ?=yz}9Uk5}LP_+#(9pZ|+Y|DR7~ z#^>^~OCp08Z*5hw<8LatGR|E6t$*=%+@Jhw|0;thpWik%f; zOchm#(36B-H!W*gH%DxS3V? zFbDXJlXjQ}xKPtTaB7+(P^Lg8Vsfyfj`~PiY3|e zlI$e0C+E>*SCYId3PtLxPD{wK=c-L9$kPSm*ts95MsHQ_t(wC?ee>NcnRcVT3(dl7 zT@v2Y3Dk$9Nw}u)aq!6#xEO)@BgWN|f1mElcN|rJCKNru-ZcVz@?s;jN2q3`v%snX z`be{eArcItWDuoXHkoxX^E-RWN-wyi07J7fx{cL}i?KBZNQ+)JeL!-+uo!JZBE$Vw z8`1F1sMG3q2T8ZrqO55oxh|d`U?lB73kaCv?WOd)mvoQ2Uv6~{np_~hiNWCNCIXkk^xa^r$R6F^+QZstz}P@_A5GyMkg zVzP4t_tP^yt26);c>Z+2zT?0!!?R$Z(U_|g>%S~n0NBg1a;vj@8LvPIy>h-W1*Wq6 zzvcfe|If##Y5t$^f5L7K#GZ~e(VR*6Z9%{p$Rdc6z0-BqP&lb5q|I*#&;Zb|gI&Q@eJVnIH1DJ8z^reFG zrX&9xINC)}>Ps4Jz8Kk{?u?cbtu)LxZ)t(_)3Y*F{bjjWTL*B4bpY#xVFlI)spv3a zygwG})6>flKG*eUqJ4vBof+D9J$5LgW&J~ZD&T(VHl9SWYPip)(mDUgaM>FF$rP*f zt`^hy)5*#Ne_|T$F>Zv@1vH+GYRR6(M)iE3%M|@Q z0mc|7JbUPr#{W+M|6j2Be{zG^pXIW^J0E^T=>Pe@R{x*0c8&&dpI+-xyLUg5<=&&t zpcS=S5teY~mZ<<_V?S!OZ|&cSUhO6?#K}G-20;$H=c3~{8YsfH*L@Ht$R4sECoz^H zdGB7l9|8CS&ma*Ra{=_)t$6R6xnoL^7AME?jU;*iF8%t^KKiR|_6OKht1a5n_cNOm zdna8hR{wAH|5E*7Dp-_HtC(}?%3RlrFyYEM_?83{x?mOf=F{7Rim;=-P2&V6wp8Dk zQhjB8PD)2IUST3{rRmg~kUO1^9DBK>C*qK;{(tI)`pj3m)&Gw#KyhmSRFRkOdFcPI z6Xb(;QdW$!JpYyazMCZx<>x=)**mMOD4!JdC&8%Y`9%tL9?{K{mr_1`9w|@^ig^}` zNnele1kl7wn#c(utG7Hqz2`UYbYE|^+YzVdL4R#dH~4RLrTwY#C|CEVXIaj_L@vKv zAS;KyUzF;DmfjVUcqLHWqR8bZCr|48U9Vv||1hZgwIGDKEaxwZ`ke=+vig6e|Ianr zxBNes!sk%NbJ;=}k;OSlr_=nu#s9-uskD7@jr4B>>M35 zp)1&fxFA6lz5T92`WFAQGE3lSF#Cth>REWjNA=8 z2OuehF@^bjjZmat@ERfHp)`+D^$#hdE&d1Ahk40(n1Xr56wJt{a^~{4`2SwTzl{3} zhD0s^DDWkt9MVl&bv?QRGssou+Q`V{Z#Dn(qWLeusapJR@&D-Hb`Iw7RxqO-Tn}Em I<;Va3AENAF2mk;8 diff --git a/pythonlib/pyproject.toml b/pythonlib/pyproject.toml index b8bc72918..fa4a272f4 100644 --- a/pythonlib/pyproject.toml +++ b/pythonlib/pyproject.toml @@ -45,15 +45,14 @@ numpy = "*" ua_parser = "*" typing_extensions = "*" screeninfo = "*" -lxml = "*" language-tags = "*" pysocks = "*" inquirer = "*" -geoip2 = {version = "*", optional = true} +maxminddb = {version = "*", optional = true} PySide6 = {version = "*", optional = true} [tool.poetry.extras] -geoip = ["geoip2"] +geoip = ["maxminddb"] gui = ["PySide6"] [tool.poetry.scripts] diff --git a/pythonlib/tests/data/webgl-gtx980-linux.json b/pythonlib/tests/data/webgl-gtx980-linux.json new file mode 100644 index 000000000..f4ce02899 --- /dev/null +++ b/pythonlib/tests/data/webgl-gtx980-linux.json @@ -0,0 +1,567 @@ +{ + "webGl:renderer": "NVIDIA GeForce GTX 980, or similar", + "webGl:vendor": "NVIDIA Corporation", + "webGl:contextAttributes": { + "alpha": true, + "antialias": true, + "depth": true, + "failIfMajorPerformanceCaveat": false, + "powerPreference": "default", + "premultipliedAlpha": true, + "preserveDrawingBuffer": false, + "stencil": false + }, + "webGl:supportedExtensions": [ + "ANGLE_instanced_arrays", + "EXT_blend_minmax", + "EXT_color_buffer_half_float", + "EXT_depth_clamp", + "EXT_float_blend", + "EXT_frag_depth", + "EXT_shader_texture_lod", + "EXT_sRGB", + "EXT_texture_compression_bptc", + "EXT_texture_compression_rgtc", + "EXT_texture_filter_anisotropic", + "OES_element_index_uint", + "OES_fbo_render_mipmap", + "OES_standard_derivatives", + "OES_texture_float", + "OES_texture_float_linear", + "OES_texture_half_float", + "OES_texture_half_float_linear", + "OES_vertex_array_object", + "WEBGL_color_buffer_float", + "WEBGL_compressed_texture_etc", + "WEBGL_compressed_texture_s3tc", + "WEBGL_compressed_texture_s3tc_srgb", + "WEBGL_debug_renderer_info", + "WEBGL_debug_shaders", + "WEBGL_depth_texture", + "WEBGL_draw_buffers", + "WEBGL_lose_context" + ], + "webGl:parameters": { + "2849": 1, + "2884": false, + "2885": 1029, + "2886": 2305, + "2928": [ + 0, + 1 + ], + "2929": false, + "2930": true, + "2931": 1, + "2932": 513, + "2960": false, + "2961": 0, + "2962": 519, + "2963": 4294967295, + "2964": 7680, + "2965": 7680, + "2966": 7680, + "2967": 0, + "2968": 4294967295, + "2978": [ + 0, + 0, + 300, + 150 + ], + "3024": true, + "3042": false, + "3074": null, + "3088": [ + 0, + 0, + 300, + 150 + ], + "3089": false, + "3106": [ + 0, + 0, + 0, + 0 + ], + "3107": [ + true, + true, + true, + true + ], + "3314": null, + "3315": null, + "3316": null, + "3317": 4, + "3330": null, + "3331": null, + "3332": null, + "3333": 4, + "3379": 32768, + "3386": [ + 32768, + 32768 + ], + "3408": 8, + "3410": 8, + "3411": 8, + "3412": 8, + "3413": 8, + "3414": 24, + "3415": 0, + "7936": "Mozilla", + "7937": "NVIDIA GeForce GTX 980, or similar", + "7938": "WebGL 1.0", + "10752": 0, + "32773": [ + 0, + 0, + 0, + 0 + ], + "32777": 32774, + "32823": false, + "32824": 0, + "32873": null, + "32877": null, + "32878": null, + "32883": null, + "32926": false, + "32928": false, + "32936": 1, + "32937": 4, + "32938": 1, + "32939": false, + "32968": 0, + "32969": 1, + "32970": 0, + "32971": 1, + "33000": null, + "33001": null, + "33170": 4352, + "33901": [ + 1, + 2047 + ], + "33902": [ + 1, + 10 + ], + "34016": 33984, + "34024": 32768, + "34045": null, + "34047": null, + "34068": null, + "34076": 32768, + "34467": null, + "34816": 519, + "34817": 7680, + "34818": 7680, + "34819": 7680, + "34852": null, + "34853": null, + "34854": null, + "34855": null, + "34856": null, + "34857": null, + "34858": null, + "34859": null, + "34860": null, + "34877": 32774, + "34921": 16, + "34930": 32, + "34964": null, + "34965": null, + "35071": null, + "35076": null, + "35077": null, + "35371": null, + "35373": null, + "35374": null, + "35375": null, + "35376": null, + "35377": null, + "35379": null, + "35380": null, + "35657": null, + "35658": null, + "35659": null, + "35660": 32, + "35661": 192, + "35723": null, + "35724": "WebGL GLSL ES 1.0", + "35725": null, + "35738": 5121, + "35739": 6408, + "35968": null, + "35977": null, + "35978": null, + "35979": null, + "36003": 0, + "36004": 4294967295, + "36005": 4294967295, + "36006": null, + "36007": null, + "36063": null, + "36183": null, + "36203": null, + "36345": null, + "36347": 1024, + "36348": 32, + "36349": 1024, + "36387": null, + "36388": null, + "36392": null, + "36795": null, + "37137": null, + "37154": null, + "37157": null, + "37440": false, + "37441": false, + "37443": 37444, + "37444": null, + "37445": "NVIDIA Corporation", + "37446": "NVIDIA GeForce GTX 980, or similar", + "37447": null, + "38449": null + }, + "webGl:shaderPrecisionFormats": { + "35633,36336": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35633,36337": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35633,36338": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35633,36339": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35633,36340": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35633,36341": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35632,36336": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35632,36337": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35632,36338": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35632,36339": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35632,36340": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35632,36341": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + } + }, + "webGl2:contextAttributes": { + "alpha": true, + "antialias": true, + "depth": true, + "failIfMajorPerformanceCaveat": false, + "powerPreference": "default", + "premultipliedAlpha": true, + "preserveDrawingBuffer": false, + "stencil": false + }, + "webGl2:supportedExtensions": [ + "EXT_color_buffer_float", + "EXT_depth_clamp", + "EXT_float_blend", + "EXT_texture_compression_bptc", + "EXT_texture_compression_rgtc", + "EXT_texture_filter_anisotropic", + "OES_draw_buffers_indexed", + "OES_texture_float_linear", + "WEBGL_compressed_texture_etc", + "WEBGL_compressed_texture_s3tc", + "WEBGL_compressed_texture_s3tc_srgb", + "WEBGL_debug_renderer_info", + "WEBGL_debug_shaders", + "WEBGL_lose_context" + ], + "webGl2:parameters": { + "2849": 1, + "2884": false, + "2885": 1029, + "2886": 2305, + "2928": [ + 0, + 1 + ], + "2929": false, + "2930": true, + "2931": 1, + "2932": 513, + "2960": false, + "2961": 0, + "2962": 519, + "2963": 4294967295, + "2964": 7680, + "2965": 7680, + "2966": 7680, + "2967": 0, + "2968": 4294967295, + "2978": [ + 0, + 0, + 300, + 150 + ], + "3024": true, + "3042": false, + "3074": 1029, + "3088": [ + 0, + 0, + 300, + 150 + ], + "3089": false, + "3106": [ + 0, + 0, + 0, + 0 + ], + "3107": [ + true, + true, + true, + true + ], + "3314": 0, + "3315": 0, + "3316": 0, + "3317": 4, + "3330": 0, + "3331": 0, + "3332": 0, + "3333": 4, + "3379": 32768, + "3386": [ + 32768, + 32768 + ], + "3408": 8, + "3410": 8, + "3411": 8, + "3412": 8, + "3413": 8, + "3414": 24, + "3415": 0, + "7936": "Mozilla", + "7937": "NVIDIA GeForce GTX 980, or similar", + "7938": "WebGL 2.0", + "10752": 0, + "32773": [ + 0, + 0, + 0, + 0 + ], + "32777": 32774, + "32823": false, + "32824": 0, + "32873": null, + "32877": 0, + "32878": 0, + "32883": 16384, + "32926": false, + "32928": false, + "32936": 1, + "32937": 4, + "32938": 1, + "32939": false, + "32968": 0, + "32969": 1, + "32970": 0, + "32971": 1, + "33000": 1048576, + "33001": 1048576, + "33170": 4352, + "33901": [ + 1, + 2047 + ], + "33902": [ + 1, + 1 + ], + "34016": 33984, + "34024": 32768, + "34045": 15, + "34047": null, + "34068": null, + "34076": 32768, + "34467": null, + "34816": 519, + "34817": 7680, + "34818": 7680, + "34819": 7680, + "34852": 8, + "34853": 1029, + "34854": 0, + "34855": 0, + "34856": 0, + "34857": 0, + "34858": 0, + "34859": 0, + "34860": 0, + "34877": 32774, + "34921": 16, + "34930": 32, + "34964": null, + "34965": null, + "35071": 2048, + "35076": -8, + "35077": 7, + "35371": 14, + "35373": 14, + "35374": 84, + "35375": 84, + "35376": 65536, + "35377": 233472, + "35379": 233472, + "35380": 256, + "35657": 4096, + "35658": 4096, + "35659": 124, + "35660": 32, + "35661": 192, + "35723": 4352, + "35724": "WebGL GLSL ES 3.00", + "35725": null, + "35738": 5121, + "35739": 6408, + "35968": 4, + "35977": false, + "35978": 128, + "35979": 4, + "36003": 0, + "36004": 4294967295, + "36005": 4294967295, + "36006": null, + "36007": null, + "36063": 8, + "36183": 32, + "36203": 4294967295, + "36345": null, + "36347": 1024, + "36348": 32, + "36349": 1024, + "36387": false, + "36388": false, + "36392": null, + "36795": null, + "37137": 1.8446744073709552e+19, + "37154": 128, + "37157": 128, + "37440": false, + "37441": false, + "37443": 37444, + "37444": null, + "37445": "NVIDIA Corporation", + "37446": "NVIDIA GeForce GTX 980, or similar", + "37447": 1000000000, + "38449": null + }, + "webGl2:shaderPrecisionFormats": { + "35633,36336": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35633,36337": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35633,36338": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35633,36339": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35633,36340": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35633,36341": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35632,36336": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35632,36337": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35632,36338": { + "rangeMin": 127, + "rangeMax": 127, + "precision": 23 + }, + "35632,36339": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35632,36340": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + }, + "35632,36341": { + "rangeMin": 24, + "rangeMax": 24, + "precision": 0 + } + }, + "webGl2Enabled": true +} diff --git a/pythonlib/tests/test_cli_list.py b/pythonlib/tests/test_cli_list.py new file mode 100644 index 000000000..038166365 --- /dev/null +++ b/pythonlib/tests/test_cli_list.py @@ -0,0 +1,25 @@ +"""`camoufox list --path` shows install paths in both listing modes.""" + +from pathlib import Path + +from click.testing import CliRunner + +from camoufox import __main__ as cli +from camoufox.multiversion import InstalledVersion +from camoufox.pkgman import Version + + +def test_list_all_shows_the_path_of_an_installed_build(monkeypatch): + install = Path("/cache/browsers/official/152.0.4-beta.30") + installed = InstalledVersion( + repo_name="official", version=Version(build="beta.30", version="152.0.4"), path=install + ) + cache = {"repos": [{"name": "official", "versions": [{"version": "152.0.4", "build": "beta.30"}]}]} + monkeypatch.setattr(cli, "_ensure_synced", lambda: True) + monkeypatch.setattr(cli, "load_repo_cache", lambda: cache) + monkeypatch.setattr(cli, "list_installed", lambda: [installed]) + + result = CliRunner().invoke(cli.cli, ["list", "all", "--path"]) + + assert result.exit_code == 0, result.output + assert str(install) in result.output diff --git a/pythonlib/tests/test_coherence.py b/pythonlib/tests/test_coherence.py index fe0c774e0..db4d07f88 100644 --- a/pythonlib/tests/test_coherence.py +++ b/pythonlib/tests/test_coherence.py @@ -1,7 +1,7 @@ """Every identity Camoufox can produce has to be a machine that could exist. The pools are sampled independently -- navigator and screen from fpgen, the GPU -from webgl_data.db, fonts and voices from their own catalogues -- so an +from fpgen's WebGL records, fonts and voices from their own catalogues -- so an incoherent identity is assembled rather than inherited, and cleaning the pools cannot prevent it. These tests run the assembled identity, from every source, past camoufox.coherence. @@ -36,7 +36,8 @@ def test_apple_silicon_never_has_fewer_than_eight_cores(self): assert config["navigator.hardwareConcurrency"] == 8 def test_a_mac_cannot_report_a_braswell_atom_igp(self): - # webgl_data.db weights this at 7.4% of the macOS pool. + # The retired WebGL database weighted this at 7.4% of the macOS pool, + # and fpgen records it from macOS too. config = {"webGl:renderer": "Intel(R) HD Graphics 400, or similar"} assert [v.rule for v in coherence.validate(config, "mac")] == ["gpu-matches-os"] @@ -149,3 +150,37 @@ def test_every_bundled_preset(self, os_name): for i, preset in enumerate(fp.load_presets("150")["presets"][os_name]): config = launch(os=os_name, fingerprint_preset=preset) assert coherence.validate(config, get_target_os(config)) == [], (os_name, i) + + +_MIDPOINT_REPAIRS = """ +from camoufox import coherence +out = [] +for os_key, steps in sorted(coherence.PLAUSIBLE_DPR.items()): + steps = sorted(steps) + for low, high in zip(steps, steps[1:]): + config = {"window.devicePixelRatio": (low + high) / 2} + coherence.apply(config, os_key) + out.append(config["window.devicePixelRatio"]) +print(out) +""" + + +def test_a_midpoint_repairs_to_the_lower_step_whether_or_not_bytecode_is_cached(tmp_path): + """The steps were frozensets, and min() keeps the first of equal distances. + A frozenset literal iterates in one order when compiled from source and in + another when loaded back from a .pyc, so the same identity repaired + differently on its first launch than on later ones.""" + import subprocess + import sys + from pathlib import Path + + env = {"PYTHONPYCACHEPREFIX": str(tmp_path), "PYTHONPATH": str(Path(coherence.__file__).parents[1])} + runs = [ + subprocess.run([sys.executable, "-c", _MIDPOINT_REPAIRS], env=env, capture_output=True, + text=True, check=True).stdout + for _ in range(2) # the first compiles and writes the .pyc, the second loads it + ] + assert runs[0] == runs[1] + lower = [low for _, steps in sorted(coherence.PLAUSIBLE_DPR.items()) + for low in sorted(steps)[:-1]] + assert runs[0].strip() == str(lower) diff --git a/pythonlib/tests/test_config_schema.py b/pythonlib/tests/test_config_schema.py index bd00fc9a9..7ef369440 100644 --- a/pythonlib/tests/test_config_schema.py +++ b/pythonlib/tests/test_config_schema.py @@ -38,6 +38,18 @@ # Add here (with a reason) only when the read genuinely cannot name its key. ALLOWED_UNDECLARED: set = set() +# Declared keys the browser never reads, each with the reason it is declared +# anyway. Everything else in properties.json must be read by a patch or by +# Juggler: a key nothing reads does nothing, silently. +NOT_READ_BY_THE_BROWSER = { + "locale:script": "the launcher joins it with locale:language/region into the UI locale", + "navigator.doNotTrack": "the launcher applies it as privacy.donottrackheader.enabled (#760)", + "navigator.buildID": "declared ahead of the patch that reads it (#780)", +} + +# How Juggler and the patches name a key: a quoted string literal. +QUOTED = '"{key}"', "'{key}'" + def _sources(): for pattern in ("patches/**/*.patch", "additions/**/*"): @@ -97,6 +109,34 @@ def test_every_key_the_browser_reads_is_declared(): pytest.fail("\n".join(lines)) +def test_every_declared_key_is_read_by_the_browser(): + """The other direction: canvas:seed (#721) was declared, generated by both + launchers and sent on every launch for three releases after the patch that + read it was removed (#528).""" + text = "".join(path.read_text(errors="ignore") for path in _sources()) + unread = sorted( + key + for key in _declared_keys() + if key not in NOT_READ_BY_THE_BROWSER + and not any(form.format(key=key) in text for form in QUOTED) + ) + assert not unread, ( + "settings/properties.json declares keys that no patch or Juggler file " + f"reads, so setting them does nothing: {unread}. Remove them, or add them " + "to NOT_READ_BY_THE_BROWSER with the reason they are declared." + ) + + +def test_keys_not_read_by_the_browser_are_really_unread(): + """An exemption must lapse once the browser starts reading the key.""" + text = "".join(path.read_text(errors="ignore") for path in _sources()) + now_read = [ + key for key in NOT_READ_BY_THE_BROWSER + if any(form.format(key=key) in text for form in QUOTED) + ] + assert not now_read, f"remove from NOT_READ_BY_THE_BROWSER, the browser reads them now: {now_read}" + + @pytest.mark.parametrize("key", ["media:spoof_codecs"]) def test_known_previously_missing_keys_stay_declared(key): """Pin the specific keys this guard was written for, so a schema edit that diff --git a/pythonlib/tests/test_fallback_warnings.py b/pythonlib/tests/test_fallback_warnings.py new file mode 100644 index 000000000..1d2b2045f --- /dev/null +++ b/pythonlib/tests/test_fallback_warnings.py @@ -0,0 +1,82 @@ +"""Every place an identity falls back to a substitute value says so. + +A substitute is a value the rest of the identity was not drawn to match, which +a page can see. Each site warns with a report block the user can paste into a +GitHub issue, so the failure reaches us instead of shipping silently. +""" + +import pytest +from test_identity_salt import launch + +from camoufox import fingerprints as fp +from camoufox import utils +from camoufox._warnings import FallbackWarning + +REPORT = r"Please report this at https://github\.com/daijro/camoufox/issues/new" + + +def _fail(error): + def raiser(*_args, **_kwargs): + raise error + + return raiser + + +def _preset(): + preset = fp.load_presets("152")["presets"]["windows"][0] + return {**preset, "fonts": ["Arial"]} + + +def _report(record): + (warning,) = [w for w in record if w.category is FallbackWarning] + text = str(warning.message) + assert "camoufox:" in text and "python:" in text and "os:" in text + return text + + +def test_preset_font_draw(monkeypatch): + monkeypatch.setattr(fp, "_generate_random_font_subset", _fail(OSError("fonts.json missing"))) + with pytest.warns(FallbackWarning, match=REPORT) as record: + config = fp.from_preset(_preset(), "152") + assert "OSError: fonts.json missing" in _report(record) + assert "Arial" in config["fonts"] + + +def test_preset_voice_draw(monkeypatch): + monkeypatch.setattr(fp, "_generate_random_voice_subset", _fail(ValueError("bad manifest"))) + with pytest.warns(FallbackWarning, match=REPORT) as record: + fp.from_preset(_preset(), "152") + assert "ValueError: bad manifest" in _report(record) + + +@pytest.mark.parametrize( + "target, error, key", + [ + ("_generate_random_font_subset", OSError("fonts.json missing"), "fonts"), + ("_generate_random_voice_subset", ValueError("bad manifest"), "voices"), + ], +) +def test_context_draws(monkeypatch, target, error, key): + monkeypatch.setattr(fp, target, _fail(error)) + with pytest.warns(FallbackWarning, match=REPORT) as record: + context = fp.generate_context_fingerprint(os="linux") + assert f"{type(error).__name__}: {error}" in _report(record) + assert key not in context["config"] + + +@pytest.mark.parametrize( + "loader, cache", [("_load_font_groups", "_FONT_GROUPS_CACHE"), ("_load_font_bases", "_FONT_BASES_CACHE")] +) +def test_font_data_loaders(monkeypatch, tmp_path, loader, cache): + monkeypatch.setattr(fp, cache, None) + monkeypatch.setattr(fp, "__file__", str(tmp_path / "fingerprints.py")) + with pytest.warns(FallbackWarning, match=REPORT) as record: + assert getattr(fp, loader)() == {} + assert "FileNotFoundError" in _report(record) + + +def test_launch_font_draw(monkeypatch): + monkeypatch.setattr(utils, "_generate_random_font_subset", _fail(OSError("font-bases.json missing"))) + with pytest.warns(FallbackWarning, match=REPORT): + config = launch() + assert config["fonts"] diff --git a/pythonlib/tests/test_identity_salt.py b/pythonlib/tests/test_identity_salt.py index f63ef7a96..2e2d9e493 100644 --- a/pythonlib/tests/test_identity_salt.py +++ b/pythonlib/tests/test_identity_salt.py @@ -42,7 +42,7 @@ def launch(**kwargs): return config_of(utils.launch_options(**kwargs)) -DRAWN = ("canvas:seed", "audio:seed", "fonts", "voices", "webGl:renderer") +DRAWN = ("audio:seed", "fonts", "voices", "webGl:renderer") def drawn(config): @@ -51,7 +51,7 @@ def drawn(config): class TestUnpinnedLaunchesAreDistinct: def test_noise_seeds_do_not_collide(self): - seeds = [launch()["canvas:seed"] for _ in range(40)] + seeds = [launch()["audio:seed"] for _ in range(40)] # 40 draws from 2**32: any collision means the seed space collapsed. assert len(set(seeds)) == len(seeds) @@ -77,33 +77,31 @@ def test_fixed_preset_reproduces_noise_seeds(self): pytest.skip("no presets bundled") first = launch(os="windows", fingerprint_preset=preset) second = launch(os="windows", fingerprint_preset=preset) - assert (first["canvas:seed"], first["audio:seed"]) == (second["canvas:seed"], second["audio:seed"]) + assert first["audio:seed"] == second["audio:seed"] assert first["fonts"] == second["fonts"] @pytest.mark.parametrize("os_name", ["windows", "macos", "linux"]) def test_every_bundled_preset_launches(self, os_name): # The test above draws ONE preset at random, so a preset that cannot - # launch shows up as a 1-in-11 flake rather than a failure -- which is - # how it reached CI. 39 of the 435 bundled presets name a GPU that is - # not among the 33 in webgl_data.db, and sample_webgl raises for those. - # Every preset has to produce launch options; see the fallback in - # utils.launch_options. - from camoufox.webgl import sample_webgl - + # launch would show up as a flake rather than a failure. Every preset + # must launch with its own GPU and that GPU's recorded parameters. presets = fp.load_presets("150")["presets"][os_name] - key = {"windows": "win", "macos": "mac", "linux": "lin"}[os_name] for i, preset in enumerate(presets): config = launch(os=os_name, fingerprint_preset=preset) - # Whatever GPU survives, the renderer the page reads and the - # parameters behind it must come from the SAME recorded device -- - # merge_into does not overwrite, so a fallback that forgets to drop - # the preset's pair leaves one device's name on another's data. + assert config["webGl:renderer"] == preset["webgl"]["unmaskedRenderer"], (os_name, i) assert config.get("webGl:parameters"), (os_name, i) - sample_webgl(key, config["webGl:vendor"], config["webGl:renderer"]) - def test_caller_seeds_are_kept(self): - config = launch(config={"canvas:seed": 7, "audio:seed": 9}) - assert (config["canvas:seed"], config["audio:seed"]) == (7, 9) + def test_caller_seed_is_kept(self): + assert launch(config={"audio:seed": 9})["audio:seed"] == 9 + + +def test_no_canvas_seed_is_generated(): + """The browser adds no canvas noise (#528), and no patch reads canvas:seed + (#721). Generating one only sent the browser a value it ignored.""" + assert "canvas:seed" not in launch() + context = fp.generate_context_fingerprint(os="linux") + assert "canvas:seed" not in context["config"] + assert "setCanvasSeed" not in context["init_script"] def test_salt_of_equal_objects_is_equal(self): a = fp.generate_fingerprint(os="windows") @@ -162,3 +160,34 @@ def test_non_ascii_pref_round_trips(self): joined = "".join(env[f"CAMOU_PREFS_{i}"] for i in range(1, len(env) + 1)) assert joined.isascii() assert orjson.loads(joined) == prefs + + +@pytest.mark.parametrize("off", [None, False]) +def test_fingerprint_preset_off_never_draws_a_preset(off): + """`fingerprint_preset=False` means off, the same as None. It used to be + checked with `is not None`, so False drew a random bundled preset.""" + with mock.patch.object(utils, "get_random_preset", side_effect=AssertionError("preset drawn")): + launch(fingerprint_preset=off) + + +def test_no_glyph_spacing_seed_is_generated(): + """Glyph-spacing noise moved every measured text width off what the same + font gives on a real machine, so it was itself a fingerprint; the feature + is gone from the browser, and the launcher sends nothing for it.""" + assert "fonts:spacing_seed" not in launch() + context = fp.generate_context_fingerprint(os="linux") + assert "fonts:spacing_seed" not in context["config"] + assert "setFontSpacingSeed" not in context["init_script"] + + +def test_config_overrides_reach_the_config_and_the_init_script(): + context = fp.generate_context_fingerprint(os="linux", config_overrides={"audio:seed": 7}) + assert context["config"]["audio:seed"] == 7 + assert "setAudioFingerprintSeed(7)" in context["init_script"] + + +def test_instant_animations_warn_that_they_are_detectable(): + from camoufox._warnings import LeakWarning + + with pytest.warns(LeakWarning, match="getComputedTiming"): + launch(config={"instantAnimations": True}, i_know_what_im_doing=False) diff --git a/pythonlib/tests/test_launch_environment.py b/pythonlib/tests/test_launch_environment.py index 2505395a0..02cceef93 100644 --- a/pythonlib/tests/test_launch_environment.py +++ b/pythonlib/tests/test_launch_environment.py @@ -14,8 +14,8 @@ def isolated_launch_dependencies(monkeypatch): monkeypatch.setattr(utils, "generate_fingerprint", lambda *args, **kwargs: object()) monkeypatch.setattr(utils, "from_fpgen", lambda *args, **kwargs: {}) monkeypatch.setattr(utils, "get_screen_cons", lambda *args, **kwargs: None) - monkeypatch.setattr(utils, "_generate_random_font_subset", lambda *args: []) - monkeypatch.setattr(utils, "_generate_random_voice_subset", lambda *args: []) + monkeypatch.setattr(utils, "_generate_random_font_subset", lambda *args, **kwargs: []) + monkeypatch.setattr(utils, "_generate_random_voice_subset", lambda *args, **kwargs: []) monkeypatch.setattr(utils, "fix_navigator_arch", lambda *args: None) monkeypatch.setattr(utils, "fix_screen_no_taskbar", lambda *args: None) monkeypatch.setattr(utils, "clamp_window_dimensions", lambda *args: None) diff --git a/pythonlib/tests/test_new_context_version.py b/pythonlib/tests/test_new_context_version.py new file mode 100644 index 000000000..031d60b20 --- /dev/null +++ b/pythonlib/tests/test_new_context_version.py @@ -0,0 +1,36 @@ +"""A context's user agent carries the Firefox version of the browser it runs in. + +Without an explicit ff_version, NewContext kept whatever version fpgen drew +(e.g. Firefox/146) on a 160 browser, so the UA disagreed with every +version-dependent API the page could probe. +""" + +import asyncio +import re +from unittest import mock + +import pytest + +from camoufox import async_api, sync_api + + +def _user_agent(init_script): + return re.search(r'setNavigatorUserAgent\("([^"]+)"\)', init_script).group(1) + + +@pytest.mark.parametrize("api", ["sync", "async"]) +def test_user_agent_matches_the_browser_version(api): + context = mock.MagicMock() + browser = mock.MagicMock() + # Playwright's Browser.version for Firefox: MOZ_APP_VERSION_DISPLAY. + browser.version = "160.0.1" + if api == "sync": + browser.new_context.return_value = context + sync_api.NewContext(browser, os="linux") + else: + context.add_init_script = mock.AsyncMock() + browser.new_context = mock.AsyncMock(return_value=context) + asyncio.run(async_api.AsyncNewContext(browser, os="linux")) + + user_agent = _user_agent(context.add_init_script.call_args.args[0]) + assert "Firefox/160.0" in user_agent and "rv:160.0" in user_agent, user_agent diff --git a/pythonlib/tests/test_proxy_geo.py b/pythonlib/tests/test_proxy_geo.py new file mode 100644 index 000000000..0b8dbfb89 --- /dev/null +++ b/pythonlib/tests/test_proxy_geo.py @@ -0,0 +1,96 @@ +"""NewContext derives the WebRTC IP and timezone from the proxy's exit IP. + +Two defects, both of which left a context with the host's WebRTC IP and +timezone while its traffic went through the proxy: + +- The lookup built its own proxy URL with urlparse, which reads a scheme-less + server such as "1.2.3.4:8080" (a form Playwright accepts) as scheme "1.2.3.4" + and drops the host. +- A failed lookup was swallowed, and the context launched without the values. +""" + +import asyncio +from unittest import mock + +import pytest + +from camoufox import async_api, ip, sync_api +from camoufox.exceptions import InvalidIP + + +class _Response: + def __init__(self, payload): + self._payload = payload + + def raise_for_status(self): + pass + + def json(self): + return self._payload + + +EXIT = {"status": "success", "query": "203.0.113.7", "timezone": "Europe/Paris"} + + +def _sync_browser(): + browser = mock.MagicMock() + browser.version = "152.0.4" + return browser + + +def _async_browser(): + context = mock.MagicMock() + context.add_init_script = mock.AsyncMock() + browser = mock.MagicMock() + browser.version = "152.0.4" + browser.new_context = mock.AsyncMock(return_value=context) + return browser + + +def _new_context(api, proxy, **kwargs): + if api == "sync": + browser = _sync_browser() + sync_api.NewContext(browser, os="linux", proxy=proxy, **kwargs) + return browser.new_context.call_args.kwargs, browser.new_context.return_value + browser = _async_browser() + asyncio.run(async_api.AsyncNewContext(browser, os="linux", proxy=proxy, **kwargs)) + return browser.new_context.call_args.kwargs, browser.new_context.return_value + + +@pytest.mark.parametrize("api", ["sync", "async"]) +@pytest.mark.parametrize( + "server, expected", + [ + ("1.2.3.4:8080", "http://u:p@1.2.3.4:8080"), + ("proxy.example.com:8080", "http://u:p@proxy.example.com:8080"), + ("http://proxy.example.com:8080", "http://u:p@proxy.example.com:8080"), + ("socks5://proxy.example.com:1080", "socks5://u:p@proxy.example.com:1080"), + ], +) +def test_lookup_goes_through_the_proxy_with_its_credentials(api, server, expected): + with mock.patch.object(ip.requests, "get", return_value=_Response(EXIT)) as get: + options, context = _new_context(api, {"server": server, "username": "u", "password": "p"}) + assert get.call_args.kwargs["proxies"] == {"http": expected, "https": expected} + assert options["timezone_id"] == "Europe/Paris" + assert "203.0.113.7" in context.add_init_script.call_args.args[0] + + +@pytest.mark.parametrize("api", ["sync", "async"]) +@pytest.mark.parametrize( + "failure", + [ + ip.requests.ConnectionError("proxy refused"), + _Response({"status": "fail", "message": "private range"}), + ], +) +def test_a_failed_lookup_raises_instead_of_launching_without_the_values(api, failure): + get = mock.Mock(side_effect=failure) if isinstance(failure, Exception) else mock.Mock(return_value=failure) + with mock.patch.object(ip.requests, "get", get), pytest.raises(InvalidIP, match="webrtc_ip"): + _new_context(api, {"server": "1.2.3.4:8080"}) + + +@pytest.mark.parametrize("api", ["sync", "async"]) +def test_no_lookup_when_both_values_are_given(api): + with mock.patch.object(ip.requests, "get") as get: + _new_context(api, {"server": "1.2.3.4:8080"}, webrtc_ip="198.51.100.1", timezone_id="UTC") + get.assert_not_called() diff --git a/pythonlib/tests/test_scroll_offset_unmapped.py b/pythonlib/tests/test_scroll_offset_unmapped.py new file mode 100644 index 000000000..0d7847c51 --- /dev/null +++ b/pythonlib/tests/test_scroll_offset_unmapped.py @@ -0,0 +1,16 @@ +"""No identity pins the page's scroll offset. + +The browser returns a configured screen.pageYOffset from scrollY on every read, +so a drawn value froze the page at one scroll position whatever the user did. +""" + +from camoufox.fingerprints import from_fpgen, generate_fingerprint + +_SCROLL_KEYS = ("screen.pageXOffset", "screen.pageYOffset") + + +def test_a_drawn_scroll_offset_is_not_carried_into_the_config(): + fingerprint = generate_fingerprint(os="windows") + fingerprint["window"] = {**fingerprint["window"], "pageXOffset": 17, "pageYOffset": 528} + config = from_fpgen(fingerprint, "152") + assert not set(_SCROLL_KEYS) & config.keys() diff --git a/pythonlib/tests/test_server.py b/pythonlib/tests/test_server.py index 507ae17ef..6e0fb84fc 100644 --- a/pythonlib/tests/test_server.py +++ b/pythonlib/tests/test_server.py @@ -275,3 +275,79 @@ def test_launch_server_surfaces_child_exit_instead_of_pipe_error(monkeypatch, tm server.launch_server() assert "3" in str(excinfo.value), str(excinfo.value) + + +class _FakeVirtualDisplay: + instances = [] + + def __init__(self, debug=None): + self.debug = debug + self.killed = False + _FakeVirtualDisplay.instances.append(self) + + def get(self): + return ":99" + + def kill(self): + self.killed = True + + +class _ExitedProcess: + def __init__(self): + self.stdin = open(os.devnull, "w") + self.returncode = 0 + + def poll(self): + return self.returncode + + def wait(self, timeout=None): + return self.returncode + + +@pytest.fixture +def fake_virtual_display(monkeypatch): + _FakeVirtualDisplay.instances = [] + monkeypatch.setattr(server, "VirtualDisplay", _FakeVirtualDisplay, raising=False) + monkeypatch.setattr(server, "get_nodejs", lambda: "/node") + return _FakeVirtualDisplay.instances + + +def test_launch_server_runs_virtual_headless_on_a_virtual_display( + monkeypatch, fake_virtual_display +): + # headless='virtual' is a Camoufox() option, not a Playwright one: the + # server must start Xvfb, launch headful on it, and kill it when the + # server process exits. + launched = {} + + def fake_launch_options(**kwargs): + launched.update(kwargs) + return {} + + monkeypatch.setattr(server, "launch_options", fake_launch_options) + monkeypatch.setattr( + server.subprocess, "Popen", lambda *args, **kwargs: _ExitedProcess() + ) + + with pytest.raises(RuntimeError): + server.launch_server(headless="virtual") + + assert len(fake_virtual_display) == 1 + assert launched["headless"] is False + assert launched["virtual_display"] == ":99" + assert fake_virtual_display[0].killed + + +def test_launch_server_kills_virtual_display_when_launch_fails( + monkeypatch, fake_virtual_display +): + def failing_launch_options(**kwargs): + raise ValueError("invalid options") + + monkeypatch.setattr(server, "launch_options", failing_launch_options) + + with pytest.raises(ValueError, match="invalid options"): + server.launch_server(headless="virtual") + + assert len(fake_virtual_display) == 1 + assert fake_virtual_display[0].killed diff --git a/pythonlib/tests/test_shipped_data.py b/pythonlib/tests/test_shipped_data.py index 4efef5db2..5ee580692 100644 --- a/pythonlib/tests/test_shipped_data.py +++ b/pythonlib/tests/test_shipped_data.py @@ -9,11 +9,10 @@ Dropped on 2026-09-17: 38 of 435 presets (26 with a GPU their OS cannot report, 7 pairing Apple Silicon with a core count Apple never shipped, 4 with a colour depth their GPU contradicts, 3 with a phone viewport, 1 claiming 40 touch -points), and 2 impossible macOS weights in webgl_data.db. +points). """ import json -import sqlite3 import sys from os.path import dirname, join from pathlib import Path @@ -24,7 +23,6 @@ from camoufox import coherence # noqa: E402 from camoufox.fingerprints import from_preset # noqa: E402 -from camoufox.webgl.sample import DB_PATH # noqa: E402 DATA = Path(__file__).parent.parent / "camoufox" PRESET_FILES = ("fingerprint-presets.json", "fingerprint-presets-v150.json") @@ -46,31 +44,19 @@ def test_every_bundled_preset_is_coherent_as_stored(filename): ) -def test_no_gpu_is_offered_to_an_os_that_cannot_report_it(): - connection = sqlite3.connect(DB_PATH) - try: - rows = connection.execute( - "SELECT vendor, renderer, win, mac, lin FROM webgl_fingerprints" - ).fetchall() - finally: - connection.close() - assert rows, "webgl_data.db is empty" - for vendor, renderer, *weights in rows: - for os_key, weight in zip(("win", "mac", "lin"), weights): - if weight and weight > 0: - assert coherence.gpu_fits_os(renderer, os_key), ( - f"{renderer!r} is offered to {os_key} at {weight}" - ) - +@pytest.mark.parametrize("filename", PRESET_FILES) +def test_every_preset_gpu_has_webgl_data(filename): + """A preset records only its GPU's name; the WebGL parameters behind it come + from fpgen. A GPU fpgen has never seen Firefox report on that OS has none, so + launching it would pair the name with another device's parameters.""" + from camoufox.webgl import firefox_gpus -def test_each_os_still_has_gpus_to_draw_from(): - """The filter must not empty a pool -- a single GPU per OS is its own tell.""" - connection = sqlite3.connect(DB_PATH) - try: - for os_key in ("win", "mac", "lin"): - count = connection.execute( - f"SELECT COUNT(*) FROM webgl_fingerprints WHERE {os_key} > 0" # nosec - ).fetchone()[0] - assert count >= 2, f"{os_key} has {count} GPU(s) left" - finally: - connection.close() + presets = json.loads((DATA / filename).read_text())["presets"] + for os_name, entries in presets.items(): + known = firefox_gpus(os_name) + for index, preset in enumerate(entries): + gpu = (preset["webgl"]["unmaskedVendor"], preset["webgl"]["unmaskedRenderer"]) + assert gpu in known, ( + f"{filename} {os_name}[{index}]: {gpu[1]!r} has no WebGL data" + " -- run scripts/clean-fingerprint-data.py --write" + ) diff --git a/pythonlib/tests/test_viewport_default.py b/pythonlib/tests/test_viewport_default.py index 70e9eb51d..a14757b28 100644 --- a/pythonlib/tests/test_viewport_default.py +++ b/pythonlib/tests/test_viewport_default.py @@ -22,7 +22,6 @@ def _opts(config_blob: str): [ ('{"window.outerWidth": 360}', True), ('{"window.innerHeight": 740}', True), - ('{"document.body.clientWidth": 360}', True), ('{"screen.width": 360}', False), ('{"navigator.userAgent": "x"}', False), ("{}", False), diff --git a/pythonlib/tests/test_voices.py b/pythonlib/tests/test_voices.py index a5b7e5248..529101d38 100644 --- a/pythonlib/tests/test_voices.py +++ b/pythonlib/tests/test_voices.py @@ -152,11 +152,14 @@ def test_caller_can_override_the_block_flag(): def test_voice_generation_failure_fails_closed(monkeypatch): import camoufox.utils as utils + from camoufox._warnings import FallbackWarning + def boom(*_args, **_kwargs): - raise RuntimeError("voices.json unreadable") + raise OSError("voice-manifests.json unreadable") monkeypatch.setattr(utils, "_generate_random_voice_subset", boom) - cfg = _launch_config(os="macos") + with pytest.warns(FallbackWarning, match="github.com/daijro/camoufox/issues/new"): + cfg = _launch_config(os="macos") # An empty list plus the block flag means "no voices" -- never "all of the # host's". assert cfg["voices"] == [] diff --git a/pythonlib/tests/test_webgl.py b/pythonlib/tests/test_webgl.py new file mode 100644 index 000000000..d794a82b5 --- /dev/null +++ b/pythonlib/tests/test_webgl.py @@ -0,0 +1,186 @@ +"""WebGL identities drawn from fpgen's recorded Firefox devices (camoufox.webgl).""" + +import json +from pathlib import Path + +import pytest +from test_identity_salt import host, launch + +from camoufox import coherence, utils +from camoufox import fingerprints as fp +from camoufox import webgl +from camoufox.fingerprints import gpu_screen_is_plausible, is_software_renderer +from camoufox.webgl import sample_webgl_for_screen, webgl_for_gpu + +SEEDS = range(300) +OSES = ("win", "mac", "lin") + +_GTX_980_LINUX = ("NVIDIA Corporation", "NVIDIA GeForce GTX 980, or similar") +_BASIC_RENDER_DRIVER = ( + "Google Inc. (Microsoft)", + "ANGLE (Microsoft, Microsoft Basic Render Driver Direct3D11 vs_5_0 ps_5_0), or similar", +) + +# Limits WebGL1 and WebGL2 read from the same device: MAX_TEXTURE_SIZE, +# MAX_VIEWPORT_DIMS, MAX_RENDERBUFFER_SIZE, MAX_CUBE_MAP_TEXTURE_SIZE, +# MAX_VERTEX_ATTRIBS, MAX_TEXTURE_IMAGE_UNITS, MAX_VERTEX_TEXTURE_IMAGE_UNITS, +# MAX_COMBINED_TEXTURE_IMAGE_UNITS and the three uniform/varying vector limits. +_SHARED_LIMITS = ("3379", "3386", "34024", "34076", "34921", "34930", "35660", "35661", "36347", "36348", "36349") + + +def _as_json(value): + # JSON has one number type: 2**64 stored as an int and as a float is the + # same value to the browser's parser. + return json.loads(json.dumps(value), parse_int=float) + + +def test_converter_reproduces_the_recorded_device(): + """The fixture is what the retired webgl_data.db gave launch_options for + this GPU. fpgen records the same device, so everything the browser reads + must come out identical: parameters are compared where the old row had a + value, less the UNMASKED_* strings, which the browser takes from + webGl:vendor/renderer rather than the table.""" + old = json.loads((Path(__file__).parent / "data" / "webgl-gtx980-linux.json").read_text()) + new = webgl_for_gpu("lin", *_GTX_980_LINUX, seed=0) + + assert new.keys() == old.keys() + for key in old: + if key.endswith(":parameters"): + recorded = { + pname: value + for pname, value in old[key].items() + if value is not None and pname not in ("37445", "37446") + } + assert _as_json({pname: new[key].get(pname) for pname in recorded}) == _as_json(recorded), key + else: + assert new[key] == old[key], key + + +def test_same_seed_same_device(): + for target_os in OSES: + for seed in (0, 1, 12345): + assert sample_webgl_for_screen(target_os, 1920, 1080, seed) == sample_webgl_for_screen( + target_os, 1920, 1080, seed + ) + gpu = ("AMD", "Radeon R9 200 Series, or similar") + assert webgl_for_gpu("lin", *gpu, seed=7) == webgl_for_gpu("lin", *gpu, seed=7) + + +def test_seed_chooses_among_a_gpus_recorded_devices(): + # fpgen records several Linux R9 200 devices; one seed must not pin them all. + drawn = {json.dumps(webgl_for_gpu("lin", "AMD", "Radeon R9 200 Series, or similar", seed=s)) for s in range(40)} + assert len(drawn) > 1 + + +@pytest.mark.parametrize("target_os", OSES) +def test_synthetic_draw_is_a_hardware_gpu_the_os_reports(target_os): + renderers = {sample_webgl_for_screen(target_os, 1920, 1080, s)["webGl:renderer"] for s in SEEDS} + for renderer in renderers: + assert not is_software_renderer(renderer), renderer + assert renderer != "Mozilla" + assert coherence.gpu_fits_os(renderer, target_os), renderer + # A single GPU per OS is its own tell. + assert len(renderers) >= 2 + + +@pytest.mark.parametrize("target_os", OSES) +def test_netbook_screen_never_draws_a_discrete_gpu(target_os): + for seed in SEEDS: + renderer = sample_webgl_for_screen(target_os, 1024, 600, seed)["webGl:renderer"] + assert gpu_screen_is_plausible(renderer, 1024, 600), renderer + + +def test_no_coherent_gpu_raises(monkeypatch): + # A pool with nothing that fits is a data defect; substituting a GPU the + # filters rejected would present exactly what they exist to prevent. + only_software = tuple(r for r in webgl._trace("gpu", "lin") if is_software_renderer(r.value["renderer"])) + assert only_software + monkeypatch.setattr(webgl, "_trace", lambda *a, **kw: only_software) + with pytest.raises(ValueError, match="No recorded lin GPU"): + sample_webgl_for_screen("lin", 1920, 1080, seed=0) + + +@pytest.mark.parametrize("target_os", OSES) +def test_webgl2_comes_from_the_same_device_as_webgl1(target_os): + # webgl2 is pinned to the drawn webgl; drawn on the GPU alone, a Linux + # Intel identity paired MAX_TEXTURE_SIZE 8192 with 16384. + for seed in SEEDS: + config = sample_webgl_for_screen(target_os, 1920, 1080, seed) + for pname in _SHARED_LIMITS: + assert config["webGl:parameters"][pname] == config["webGl2:parameters"][pname], (seed, pname) + + +def test_gpu_is_pinned_by_vendor_and_renderer(): + # "Mesa" and "AMD" both report this renderer on Linux. A dict condition on + # fpgen matches the renderer alone and mixed their devices. + for seed in range(40): + assert webgl_for_gpu("lin", "Mesa", "Radeon HD 3200 Graphics, or similar", seed)["webGl:vendor"] == "Mesa" + + +def test_device_without_webgl2(): + config = webgl_for_gpu("win", *_BASIC_RENDER_DRIVER, seed=0) + assert config["webGl2Enabled"] is False + assert not any(key.startswith("webGl2:") for key in config) + + with host(): + options = utils.launch_options( + os="windows", webgl_config=_BASIC_RENDER_DRIVER, headless=True, i_know_what_im_doing=True + ) + assert options["firefox_user_prefs"]["webgl.enable-webgl2"] is False + + +def test_unknown_webgl_config_raises(): + with pytest.raises(ValueError, match="No recorded WebGL data"): + launch(os="windows", webgl_config=("Apple", "Apple M1, or similar")) + + +def test_preset_keeps_its_own_gpu(): + preset = fp.load_presets("152")["presets"]["linux"][0] + gpu = (preset["webgl"]["unmaskedVendor"], preset["webgl"]["unmaskedRenderer"]) + config = launch(os="linux", fingerprint_preset=preset) + assert (config["webGl:vendor"], config["webGl:renderer"]) == gpu + pinned = (webgl._pin("gpu", {"vendor": gpu[0], "renderer": gpu[1]}),) + recorded = [webgl.to_config(r.value, [], "lin")["webGl:parameters"] for r in webgl._trace("webgl", "lin", pinned)] + assert config["webGl:parameters"] in recorded + + +def test_preset_gpu_fpgen_has_never_seen_raises(): + preset = fp.load_presets("152")["presets"]["windows"][0] + gpu = "ANGLE (Acme, Acme GPU 9000 Direct3D11 vs_5_0 ps_5_0)" + preset = {**preset, "webgl": {"unmaskedVendor": "Google Inc. (Acme)", "unmaskedRenderer": gpu}} + with pytest.raises(ValueError, match="Acme GPU 9000"): + launch(os="windows", fingerprint_preset=preset) + + +# -- extensions --------------------------------------------------------------- + + +def _extensions(target_os, key): + return [ + set(sample_webgl_for_screen(target_os, 1920, 1080, s).get(key) or ()) for s in range(100) + ] + + +def test_windows_keeps_ovr_multiview2_on_webgl2(): + assert any("OVR_multiview2" in exts for exts in _extensions("win", "webGl2:supportedExtensions")) + + +def test_linux_filters_ovr_multiview2(): + # fpgen records it on ~20% of Linux WebGL2 devices. + assert not any("OVR_multiview2" in exts for exts in _extensions("lin", "webGl2:supportedExtensions")) + + +def test_draft_extensions_filtered_on_every_os(): + recorded = { + "vendor": "v", + "renderer": "r", + "contextAttributes": {}, + "params": {}, + "shaderPrecisionFormats": [], + "supportedExtensions": ["ANGLE_instanced_arrays", "WEBGL_multi_draw", "WEBGL_compressed_texture_etc1"], + } + webgl2 = {**recorded, "supportedExtensions": ["EXT_texture_norm16", "WEBGL_clip_cull_distance", "OVR_multiview2"]} + for target_os in OSES: + config = webgl.to_config(recorded, webgl2, target_os) + assert config["webGl:supportedExtensions"] == ["ANGLE_instanced_arrays"] + assert config["webGl2:supportedExtensions"] == (["OVR_multiview2"] if target_os == "win" else []) diff --git a/pythonlib/tests/test_webgl_extension_filter.py b/pythonlib/tests/test_webgl_extension_filter.py deleted file mode 100644 index 057748c6a..000000000 --- a/pythonlib/tests/test_webgl_extension_filter.py +++ /dev/null @@ -1,60 +0,0 @@ -"""Which sampled WebGL extensions reach the page, per OS.""" - -import sqlite3 - -import orjson - -from camoufox.webgl import sample -from camoufox.webgl.sample import DB_PATH, _load_webgl_data - - -def _row_with(ext, key="webGl2:supportedExtensions", os="win"): - con = sqlite3.connect(DB_PATH) - try: - for (data,) in con.execute(f"SELECT data FROM webgl_fingerprints WHERE {os} > 0"): # nosec - if ext in (orjson.loads(data).get(key) or []): - return data - finally: - con.close() - raise AssertionError(f"no {os} row carries {ext}") - - -def test_windows_keeps_ovr_multiview2_on_webgl2(): - data = _load_webgl_data(_row_with("OVR_multiview2"), "win") - assert "OVR_multiview2" in data["webGl2:supportedExtensions"] - - -def test_linux_filters_ovr_multiview2(): - data = _load_webgl_data(_row_with("OVR_multiview2", os="lin"), "lin") - assert "OVR_multiview2" not in data["webGl2:supportedExtensions"] - - -def test_ovr_multiview2_never_on_webgl1_in_corpus(): - con = sqlite3.connect(DB_PATH) - try: - for (data,) in con.execute("SELECT data FROM webgl_fingerprints"): - assert "OVR_multiview2" not in (orjson.loads(data).get("webGl:supportedExtensions") or []) - finally: - con.close() - - -def test_draft_extensions_filtered_on_every_os(): - blob = orjson.dumps( - { - "webGl:supportedExtensions": ["ANGLE_instanced_arrays", "WEBGL_multi_draw"], - "webGl2:supportedExtensions": ["EXT_texture_norm16", "WEBGL_clip_cull_distance", "OVR_multiview2"], - } - ) - for os in ("win", "mac", "lin"): - data = _load_webgl_data(blob, os) - assert data["webGl:supportedExtensions"] == ["ANGLE_instanced_arrays"] - assert "EXT_texture_norm16" not in data["webGl2:supportedExtensions"] - assert "WEBGL_clip_cull_distance" not in data["webGl2:supportedExtensions"] - - -def test_sampled_windows_identities_can_carry_it(): - hits = sum( - "OVR_multiview2" in (sample.sample_webgl("win", seed=s).get("webGl2:supportedExtensions") or []) - for s in range(200) - ) - assert hits > 0 diff --git a/pythonlib/tests/test_webgl_screen_consistency.py b/pythonlib/tests/test_webgl_screen_consistency.py index 424b163cd..82607a4e6 100644 --- a/pythonlib/tests/test_webgl_screen_consistency.py +++ b/pythonlib/tests/test_webgl_screen_consistency.py @@ -4,8 +4,8 @@ Run with: cd pythonlib && python -m pytest tests/test_webgl_screen_consistency.py -v -The regression these guard (daijro/camoufox#729): BrowserForge picks the -screen, webgl_data.db picks the GPU, and nothing ties them together -- so the +The regression these guard (daijro/camoufox#729): the generator picks the +screen, camoufox.webgl picks the GPU, and nothing ties them together -- so the synthetic path can emit pairs no real machine ships (a discrete GPU behind a 1024x600 netbook panel). @@ -24,15 +24,14 @@ sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) -from camoufox import fingerprints # noqa: E402 from camoufox.fingerprints import ( # noqa: E402 MODERN_SCREEN_FLOOR, _renderer_bucket, gpu_screen_is_plausible, is_software_renderer, raise_screen_to_modern_floor, - sample_webgl_for_screen, ) +from camoufox.webgl import sample_webgl_for_screen # noqa: E402 # The three spellings Gecko emits for one discrete-NVIDIA bucket. _NV_ANGLE = "ANGLE (NVIDIA, NVIDIA GeForce GTX 980 Direct3D11 vs_5_0 ps_5_0), or similar" @@ -153,64 +152,10 @@ def test_hardware_is_not_mistaken_for_software(monkeypatch): assert not is_software_renderer(renderer) -def test_software_first_draw_is_resampled_to_hardware(monkeypatch): - """A presented llvmpipe / WARP / SwiftShader is the first thing every - consumer-hardware check flags (measured 2026-09-14 with sundial), so a - software first draw is retried until a hardware renderer that fits the - screen comes up, and only kept when the pool offers nothing else.""" - draws = iter([{"webGl:renderer": _LLVMPIPE}, {"webGl:renderer": _INTEL}]) - monkeypatch.setattr(fingerprints, "sample_webgl", lambda *a, **kw: next(draws)) - - assert sample_webgl_for_screen("lin", 1024, 600)["webGl:renderer"] == _INTEL - - only_software = iter([{"webGl:renderer": _LLVMPIPE}] * 40) - monkeypatch.setattr(fingerprints, "sample_webgl", lambda *a, **kw: next(only_software)) - assert sample_webgl_for_screen("lin", 1920, 1080)["webGl:renderer"] == _LLVMPIPE - - -def test_software_draws_are_skipped_when_resampling(monkeypatch): - # A hardware first draw settles the class as hardware, so a software - # candidate mid-loop is skipped instead of accepted -- otherwise the - # rejection loop still leaks probability mass onto the rasterizers. - draws = iter( - [ - {"webGl:renderer": _NV_ANGLE}, # implausible at 1024x600 - {"webGl:renderer": _LLVMPIPE}, # plausible, but wrong class - {"webGl:renderer": _INTEL}, # the coherent hardware answer - ] - ) - monkeypatch.setattr(fingerprints, "sample_webgl", lambda *a, **kw: next(draws)) - - assert sample_webgl_for_screen("lin", 1024, 600)["webGl:renderer"] == _INTEL - - -def test_falls_back_to_the_first_draw_when_nothing_is_coherent(monkeypatch): - monkeypatch.setattr( - fingerprints, "sample_webgl", lambda *a, **kw: {"webGl:renderer": _NV_ANGLE} - ) - fp = sample_webgl_for_screen("win", 800, 600, attempts=4) - assert fp["webGl:renderer"] == _NV_ANGLE - - -def test_a_plausible_first_draw_costs_one_query(monkeypatch): - # sample_webgl opens a fresh sqlite connection per call, and the common - # case (any screen at or above the floor) must not pay for 32 of them. - calls = [] - - def _counted(*args, **kwargs): - calls.append(args) - return {"webGl:renderer": _NV_ANGLE} - - monkeypatch.setattr(fingerprints, "sample_webgl", _counted) - sample_webgl_for_screen("win", 1920, 1080) - assert len(calls) == 1 - - @pytest.mark.parametrize("target_os", ["win", "mac", "lin"]) def test_sampled_gpu_is_coherent_with_the_screen(target_os): - # Against the real webgl_data.db pool. - for _ in range(25): - fp = sample_webgl_for_screen(target_os, 1280, 800) + for seed in range(25): + fp = sample_webgl_for_screen(target_os, 1280, 800, seed=seed) assert gpu_screen_is_plausible(fp.get("webGl:renderer"), 1280, 800) @@ -257,8 +202,8 @@ def test_screen_floor_is_a_no_op_without_screen_values(): @pytest.mark.parametrize("target_os", ["windows", "macos", "linux"]) def test_context_fingerprints_get_the_same_treatment(target_os): """generate_context_fingerprint() is the per-context API #729 names, and - build-tester drives the browser through it. It sampled the GPU with a bare - sample_webgl() and never applied the floor, so the coherence fix reached + build-tester drives the browser through it. It drew the GPU without the + screen and never applied the floor, so the coherence fix reached launch_options() only.""" from camoufox.fingerprints import generate_context_fingerprint diff --git a/pythonlib/tests/test_webrtc_ip_setter.py b/pythonlib/tests/test_webrtc_ip_setter.py new file mode 100644 index 000000000..45dd2249a --- /dev/null +++ b/pythonlib/tests/test_webrtc_ip_setter.py @@ -0,0 +1,23 @@ +"""The per-context init script hands a WebRTC IP to the setter for its family.""" + +import pytest + +from camoufox.exceptions import InvalidIP +from camoufox.fingerprints import _build_init_script + + +def test_ipv4_goes_to_the_ipv4_setter(): + script = _build_init_script({"webrtcIP": "203.0.113.7"}) + assert 'w.setWebRTCIPv4("203.0.113.7")' in script + assert "setWebRTCIPv6(" not in script + + +def test_ipv6_goes_to_the_ipv6_setter(): + script = _build_init_script({"webrtcIP": "2001:db8::7"}) + assert 'w.setWebRTCIPv6("2001:db8::7")' in script + assert "2001:db8::7" not in script.split("setWebRTCIPv6")[0] + + +def test_an_invalid_address_is_refused(): + with pytest.raises(InvalidIP): + _build_init_script({"webrtcIP": "not-an-ip"}) diff --git a/scripts/_mixin.py b/scripts/_mixin.py index f2efea4cc..d7cebbce6 100644 --- a/scripts/_mixin.py +++ b/scripts/_mixin.py @@ -9,7 +9,6 @@ import fnmatch import optparse import os -import re import sys import time @@ -34,9 +33,6 @@ def get_options(): """Get options""" parser = optparse.OptionParser() parser.add_option('--mozconfig-only', dest='mozconfig_only', default=False, action="store_true") - parser.add_option( - '-P', '--no-settings-pane', dest='settings_pane', default=True, action="store_false" - ) return parser.parse_args() @@ -79,9 +75,6 @@ def list_patches(root_dir='../patches', suffix='*.patch'): """List all patch files""" return sorted(list_files(root_dir, suffix), key=os.path.basename) -def is_bootstrap_patch(name): - return bool(re.match(r'\d+\-.*', os.path.basename(name))) - def script_exit(statuscode): """Exit the script""" @@ -109,24 +102,9 @@ def run(cmd, exit_on_fail=True, do_print=True): return retval -def patch(patchfile, reverse=False, silent=False): - """Run a patch file""" - if reverse: - cmd = f"patch -p1 -R -i {patchfile}" - else: - cmd = f"patch -p1 -i {patchfile}" - if silent: - cmd += ' > /dev/null' - else: - print(f"\n*** -> {cmd}") - sys.stdout.flush() - run(cmd) - - __all__ = [ 'get_moz_target', 'list_patches', - 'patch', 'run', 'script_exit', 'temp_cd', diff --git a/scripts/bootstrap.py b/scripts/bootstrap.py deleted file mode 100644 index c2f1eea64..000000000 --- a/scripts/bootstrap.py +++ /dev/null @@ -1,430 +0,0 @@ -#!/usr/bin/env python3 -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this file, -# You can obtain one at http://mozilla.org/MPL/2.0/. - -# This script provides one-line bootstrap support to configure systems to build -# the tree. It does so by cloning the repo before calling directly into `mach -# bootstrap`. - -# Note that this script can't assume anything in particular about the host -# Python environment (except that it's run with a sufficiently recent version of -# Python 3), so we are restricted to stdlib modules. - -import sys - -major, minor = sys.version_info[:2] -if (major < 3) or (major == 3 and minor < 8): - print("Bootstrap currently only runs on Python 3.8+." "Please try re-running with python3.8+.") - sys.exit(1) - -import ctypes -import os -import shutil -import subprocess -import tempfile -from optparse import OptionParser -from pathlib import Path - -CLONE_MERCURIAL_PULL_FAIL = """ -Failed to pull from hg.mozilla.org. - -This is most likely because of unstable network connection. -Try running `cd %s && hg pull https://hg.mozilla.org/mozilla-unified` manually, -or download a mercurial bundle and use it: -https://firefox-source-docs.mozilla.org/contributing/vcs/mercurial_bundles.html""" - -WINDOWS = sys.platform.startswith("win32") or sys.platform.startswith("msys") -VCS_HUMAN_READABLE = { - "hg": "Mercurial", - "git": "Git", -} - - -def which(name): - """Python implementation of which. - - It returns the path of an executable or None if it couldn't be found. - """ - search_dirs = os.environ["PATH"].split(os.pathsep) - potential_names = [name] - if WINDOWS: - potential_names.insert(0, name + ".exe") - - for path in search_dirs: - for executable_name in potential_names: - test = Path(path) / executable_name - if test.is_file() and os.access(test, os.X_OK): - return test - - return None - - -def validate_clone_dest(dest: Path): - dest = dest.resolve() - - if not dest.exists(): - return dest - - if not dest.is_dir(): - print(f"ERROR! Destination {dest} exists but is not a directory.") - return None - - if not any(dest.iterdir()): - return dest - else: - print(f"ERROR! Destination directory {dest} exists but is nonempty.") - print( - f"To re-bootstrap the existing checkout, go into '{dest}' and run './mach bootstrap'." - ) - return None - - -def input_clone_dest(vcs, no_interactive): - repo_name = "mozilla-unified" - print(f"Cloning into {repo_name} using {VCS_HUMAN_READABLE[vcs]}...") - while True: - dest = None - if not no_interactive: - dest = input( - f"Destination directory for clone (leave empty to use " - f"default destination of {repo_name}): " - ).strip() - if not dest: - dest = repo_name - dest = validate_clone_dest(Path(dest).expanduser()) - if dest: - return dest - if no_interactive: - return None - - -def hg_clone_firefox(hg: Path, dest: Path, head_repo, head_rev): - # We create an empty repo then modify the config before adding data. - # This is necessary to ensure storage settings are optimally - # configured. - args = [ - str(hg), - # The unified repo is generaldelta, so ensure the client is as - # well. - "--config", - "format.generaldelta=true", - "init", - str(dest), - ] - res = subprocess.call(args) - if res: - print("unable to create destination repo; please try cloning manually") - return None - - # Strictly speaking, this could overwrite a config based on a template - # the user has installed. Let's pretend this problem doesn't exist - # unless someone complains about it. - with open(dest / ".hg" / "hgrc", "a") as fh: - fh.write("[paths]\n") - fh.write("default = https://hg.mozilla.org/mozilla-unified\n") - fh.write("\n") - - # The server uses aggressivemergedeltas which can blow up delta chain - # length. This can cause performance to tank due to delta chains being - # too long. Limit the delta chain length to something reasonable - # to bound revlog read time. - fh.write("[format]\n") - fh.write("# This is necessary to keep performance in check\n") - fh.write("maxchainlen = 10000\n") - - # Pulling a specific revision into an empty repository induces a lot of - # load on the Mercurial server, so we always pull from mozilla-unified (which, - # when done from an empty repository, is equivalent to a clone), and then pull - # the specific revision we want (if we want a specific one, otherwise we just - # use the "central" bookmark), at which point it will be an incremental pull, - # that the server can process more easily. - # This is the same thing that robustcheckout does on automation. - res = subprocess.call( - [str(hg), "pull", "https://hg.mozilla.org/mozilla-unified"], cwd=str(dest) - ) - if not res and head_repo: - res = subprocess.call([str(hg), "pull", head_repo, "-r", head_rev], cwd=str(dest)) - print("") - if res: - print(CLONE_MERCURIAL_PULL_FAIL % dest) - return None - - head_rev = head_rev or "central" - print(f'updating to "{head_rev}" - the development head of Gecko and Firefox') - res = subprocess.call([str(hg), "update", "-r", head_rev], cwd=str(dest)) - if res: - print(f"error updating; you will need to `cd {dest} && hg update -r central` " "manually") - return dest - - -def git_clone_firefox(git: Path, dest: Path, watchman: Path, head_repo, head_rev): - tempdir = None - cinnabar = None - env = dict(os.environ) - try: - cinnabar = which("git-cinnabar") - if not cinnabar: - from urllib.request import urlopen - - cinnabar_url = "https://github.com/glandium/git-cinnabar/" - # If git-cinnabar isn't installed already, that's fine; we can - # download a temporary copy. `mach bootstrap` will install a copy - # in the state dir; we don't want to copy all that logic to this - # tiny bootstrapping script. - tempdir = Path(tempfile.mkdtemp()) - with open(tempdir / "download.py", "wb") as fh: - shutil.copyfileobj(urlopen(f"{cinnabar_url}/raw/master/download.py"), fh) - - subprocess.check_call( - [sys.executable, str(tempdir / "download.py")], - cwd=str(tempdir), - ) - env["PATH"] = str(tempdir) + os.pathsep + env["PATH"] - print( - "WARNING! git-cinnabar is required for Firefox development " - "with git. After the clone is complete, the bootstrapper " - "will ask if you would like to configure git; answer yes, " - "and be sure to add git-cinnabar to your PATH according to " - "the bootstrapper output." - ) - - # We're guaranteed to have `git-cinnabar` installed now. - # Configure git per the git-cinnabar requirements. - subprocess.check_call( - [ - str(git), - "-c", - "fetch.prune=true", - "clone", - "--no-checkout", - "hg::https://hg.mozilla.org/mozilla-unified", - str(dest), - ], - env=env, - ) - subprocess.check_call([str(git), "config", "fetch.prune", "true"], cwd=str(dest), env=env) - subprocess.check_call([str(git), "config", "pull.ff", "only"], cwd=str(dest), env=env) - - if head_repo: - subprocess.check_call( - [str(git), "cinnabar", "fetch", f"hg::{head_repo}", head_rev], - cwd=str(dest), - env=env, - ) - - subprocess.check_call( - [ - str(git), - "checkout", - "FETCH_HEAD" if head_rev else "bookmarks/central", - "--", - ], - cwd=str(dest), - env=env, - ) - - watchman_sample = dest / ".git/hooks/fsmonitor-watchman.sample" - # Older versions of git didn't include fsmonitor-watchman.sample. - if watchman and watchman_sample.exists(): - print("Configuring watchman") - watchman_config = dest / ".git/hooks/query-watchman" - if not watchman_config.exists(): - print(f"Copying {watchman_sample} to {watchman_config}") - copy_args = [ - "cp", - ".git/hooks/fsmonitor-watchman.sample", - ".git/hooks/query-watchman", - ] - subprocess.check_call(copy_args, cwd=str(dest)) - - config_args = [ - str(git), - "config", - "core.fsmonitor", - ".git/hooks/query-watchman", - ] - subprocess.check_call(config_args, cwd=str(dest), env=env) - return dest - finally: - if tempdir: - shutil.rmtree(str(tempdir)) - - -def add_microsoft_defender_antivirus_exclusions(dest, no_system_changes): - if no_system_changes: - return - - if not WINDOWS: - return - - powershell_exe = which("powershell") - - if not powershell_exe: - return - - def print_attempt_exclusion(path): - print(f"Attempting to add exclusion path to Microsoft Defender Antivirus for: {path}") - - powershell_exe = str(powershell_exe) - paths = [] - - # mozilla-unified / clone dest - repo_dir = Path.cwd() / dest - paths.append(repo_dir) - print_attempt_exclusion(repo_dir) - - # MOZILLABUILD - mozillabuild_dir = os.getenv("MOZILLABUILD") - if mozillabuild_dir: - paths.append(mozillabuild_dir) - print_attempt_exclusion(mozillabuild_dir) - - # .mozbuild - mozbuild_dir = Path.home() / ".mozbuild" - paths.append(mozbuild_dir) - print_attempt_exclusion(mozbuild_dir) - - args = ";".join(f"Add-MpPreference -ExclusionPath '{path}'" for path in paths) - command = f'-Command "{args}"' - - # This will attempt to run as administrator by triggering a UAC prompt - # for admin credentials. If "No" is selected, no exclusions are added. - ctypes.windll.shell32.ShellExecuteW(None, "runas", powershell_exe, command, None, 0) - - -def clone(options): - vcs = options.vcs - no_interactive = options.no_interactive - no_system_changes = options.no_system_changes - - if vcs == "hg": - hg = which("hg") - if not hg: - print("Mercurial is not installed. Mercurial is required to clone Firefox.") - try: - # We're going to recommend people install the Mercurial package with - # pip3. That will work if `pip3` installs binaries to a location - # that's in the PATH, but it might not be. To help out, if we CAN - # import "mercurial" (in which case it's already been installed), - # offer that as a solution. - import mercurial # noqa: F401 - - print( - "Hint: have you made sure that Mercurial is installed to a " - "location in your PATH?" - ) - except ImportError: - print("Try installing hg with `pip3 install Mercurial`.") - return None - binary = hg - else: - binary = which(vcs) - if not binary: - print("Git is not installed.") - print("Try installing git using your system package manager.") - return None - - dest = input_clone_dest(vcs, no_interactive) - if not dest: - return None - - add_microsoft_defender_antivirus_exclusions(dest, no_system_changes) - - print(f"Cloning Firefox {VCS_HUMAN_READABLE[vcs]} repository to {dest}") - - head_repo = os.environ.get("GECKO_HEAD_REPOSITORY") - head_rev = os.environ.get("GECKO_HEAD_REV") - - if vcs == "hg": - return hg_clone_firefox(binary, dest, head_repo, head_rev) - else: - watchman = which("watchman") - return git_clone_firefox(binary, dest, watchman, head_repo, head_rev) - - -def bootstrap(srcdir: Path, application_choice, no_interactive, no_system_changes): - args = [sys.executable, "mach"] - - if no_interactive: - # --no-interactive is a global argument, not a command argument, - # so it needs to be specified before "bootstrap" is appended. - args += ["--no-interactive"] - - args += ["bootstrap"] - - if application_choice: - args += ["--application-choice", application_choice] - if no_system_changes: - args += ["--no-system-changes"] - - print("Running `%s`" % " ".join(args)) - return subprocess.call(args, cwd=str(srcdir)) - - -def main(args): - parser = OptionParser() - parser.add_option( - "--application-choice", - dest="application_choice", - help='Pass in an application choice (see "APPLICATIONS" in ' - "python/mozboot/mozboot/bootstrap.py) instead of using the " - "default interactive prompt.", - ) - parser.add_option( - "--vcs", - dest="vcs", - default="hg", - choices=["git", "hg"], - help="VCS (hg or git) to use for downloading the source code, " - "instead of using the default interactive prompt.", - ) - parser.add_option( - "--no-interactive", - dest="no_interactive", - action="store_true", - help="Answer yes to any (Y/n) interactive prompts.", - ) - parser.add_option( - "--no-system-changes", - dest="no_system_changes", - action="store_true", - help="Only executes actions that leave the system " "configuration alone.", - ) - - options, leftover = parser.parse_args(args) - try: - srcdir = clone(options) - if not srcdir: - return 1 - print("Clone complete.") - print( - "If you need to run the tooling bootstrapping again, " - "then consider running './mach bootstrap' instead." - ) - if not options.no_interactive: - remove_bootstrap_file = input( - "Unless you are going to have more local copies of Firefox source code, " - "this 'bootstrap.py' file is no longer needed and can be deleted. " - "Clean up the bootstrap.py file? (Y/n)" - ) - if not remove_bootstrap_file: - remove_bootstrap_file = "y" - if options.no_interactive or remove_bootstrap_file == "y": - try: - Path(sys.argv[0]).unlink() - except FileNotFoundError: - print("File could not be found !") - return bootstrap( - srcdir, - options.application_choice, - options.no_interactive, - options.no_system_changes, - ) - except Exception: - print("Could not bootstrap Firefox! Consider filing a bug.") - raise - - -if __name__ == "__main__": - sys.exit(main(sys.argv)) diff --git a/scripts/clean-fingerprint-data.py b/scripts/clean-fingerprint-data.py index 6b2c3939f..3e65df718 100644 --- a/scripts/clean-fingerprint-data.py +++ b/scripts/clean-fingerprint-data.py @@ -11,17 +11,13 @@ What it covers: fingerprint-presets.json, fingerprint-presets-v150.json - Each preset is converted the way a launch converts it and checked. A row + Each preset is converted the way a launch converts it and checked, and + its GPU must be one fpgen has seen Firefox report on that OS: WebGL + parameters come from fpgen, and a GPU it has never seen has none. A row that fails is dropped rather than repaired: repairing would write an invented value ("what core count does a 2-core Apple M1 really have?") into a file whose entire purpose is being real. - webgl/webgl_data.db - Each (vendor, renderer) pair carries a probability per OS. A pair the OS - cannot report has that probability zeroed, which keeps the row for the - platforms where it IS real -- "Radeon R9 200 Series" is a genuine Linux - and Windows GPU, it simply never shipped in a Mac. - Usage: python3 scripts/clean-fingerprint-data.py # report only python3 scripts/clean-fingerprint-data.py --write # rewrite the files @@ -33,7 +29,6 @@ import argparse import json -import sqlite3 import sys from collections import Counter from pathlib import Path @@ -43,12 +38,12 @@ from camoufox import coherence # noqa: E402 from camoufox.fingerprints import from_preset # noqa: E402 +from camoufox.webgl import firefox_gpus # noqa: E402 PRESET_FILES = ( REPO / 'pythonlib' / 'camoufox' / 'fingerprint-presets.json', REPO / 'pythonlib' / 'camoufox' / 'fingerprint-presets-v150.json', ) -WEBGL_DB = REPO / 'pythonlib' / 'camoufox' / 'webgl' / 'webgl_data.db' OS_KEY = {'macos': 'mac', 'windows': 'win', 'linux': 'lin'} # The Firefox version only decides the UA rewrite, which no rule reads. FF_VERSION = '152' @@ -60,7 +55,12 @@ def preset_violations(preset, os_name): config = from_preset(preset, FF_VERSION) except Exception as exc: # a row too malformed to convert is itself a defect return [coherence.Violation('unconvertible', f'{type(exc).__name__}: {exc}')] - return coherence.validate(config, OS_KEY[os_name]) + violations = coherence.validate(config, OS_KEY[os_name]) + gpu = (preset.get('webgl', {}).get('unmaskedVendor'), preset.get('webgl', {}).get('unmaskedRenderer')) + if gpu not in firefox_gpus(os_name): + violations.append(coherence.Violation( + 'gpu-without-webgl-data', f'fpgen has never seen Firefox on {os_name} report {gpu[1]!r}')) + return violations def clean_presets(path, write): @@ -97,30 +97,6 @@ def clean_presets(path, write): return dropped_total -def clean_webgl_db(write): - connection = sqlite3.connect(WEBGL_DB) - cursor = connection.cursor() - cursor.execute('SELECT rowid, vendor, renderer, win, mac, lin FROM webgl_fingerprints') - rows = cursor.fetchall() - zeroed = 0 - for rowid, vendor, renderer, *weights in rows: - for column, weight in zip(('win', 'mac', 'lin'), weights): - if weight and weight > 0 and not coherence.gpu_fits_os(renderer, column): - zeroed += 1 - print(f' zero webgl_data.db {column}={weight:.3f} for {renderer[:58]!r}') - if write: - cursor.execute( - f'UPDATE webgl_fingerprints SET {column} = 0 WHERE rowid = ?', # nosec - (rowid,), - ) - if write and zeroed: - connection.commit() - connection.close() - print(f'webgl_data.db: {zeroed} impossible OS weight(s)' - + (' zeroed' if write and zeroed else '')) - return zeroed - - def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument('--write', action='store_true', help='rewrite the data files') @@ -128,7 +104,6 @@ def main(): args = parser.parse_args() total = sum(clean_presets(path, args.write) for path in PRESET_FILES) - total += clean_webgl_db(args.write) if not total: print('\nEvery shipped identity is coherent.') diff --git a/scripts/copy-additions.sh b/scripts/copy-additions.sh index 4c6d2c285..1861ecc14 100644 --- a/scripts/copy-additions.sh +++ b/scripts/copy-additions.sh @@ -58,7 +58,6 @@ fi run 'cp -r ../additions/* .' # Provide a script that fetches and bootstraps Nightly and some mozconfigs -run 'cp -v ../scripts/mozfetch.sh lw/' # Override the firefox version for file in "browser/config/version.txt" "browser/config/version_display.txt"; do diff --git a/scripts/cursor-demo.py b/scripts/cursor-demo.py new file mode 100644 index 000000000..c77d7dec4 --- /dev/null +++ b/scripts/cursor-demo.py @@ -0,0 +1,89 @@ +""" +Record humanize=True cursor movements from a real build and draw them as an +animated SVG: assets/humanize-cursor.svg, the figure in the README's "Page +Interactions" section. + +Every point is a mousemove event the page received, at the time it received +it, so the figure shows what a site sees: the shape of each path, the spacing +of its events (close together = slow), and the pauses, replayed in real time. + + python3 scripts/cursor-demo.py /path/to/camoufox-bin +""" + +import asyncio +import sys +from pathlib import Path + +from camoufox.async_api import AsyncCamoufox + +W, H = 900, 420 +# A tour of short and long moves in several directions, like a user working +# through a form. +STOPS = [(80, 360), (820, 70), (460, 330), (170, 110), (640, 380), (840, 250), (80, 360)] +RECORDER = """ + window.moves = []; + addEventListener("mousemove", e => moves.push([e.clientX, e.clientY, performance.now()])); +""" +OUT = Path(__file__).resolve().parent.parent / "assets" / "humanize-cursor.svg" +LABEL = 36 # caption strip under the page area +PAUSE_MS = 600 # between moves in the replay; the recorded gap is Playwright overhead + + +async def record(executable): + moves = [] + async with AsyncCamoufox( + headless=True, os="linux", humanize=True, executable_path=executable, + window=(W + 100, H + 200), + ) as browser: + page = await browser.new_page() + await page.set_content(f'') + await page.mouse.move(*STOPS[0]) + for stop in STOPS[1:]: + await page.evaluate(RECORDER) + await page.mouse.move(*stop) + moves.append(await page.evaluate("moves")) + return moves + + +def render(moves): + paths, frames, t0 = [], [], 0.0 + for events in moves: + start = events[0][2] + paths.append(events) + for x, y, t in events: + frames.append((x, y, t0 + t - start)) + t0 += events[-1][2] - start + PAUSE_MS + total = t0 + keyTimes = ";".join(f"{t / total:.5f}" for *_, t in frames) + ";1" + values = ";".join(f"{x},{y}" for x, y, _ in frames) + f";{frames[-1][0]},{frames[-1][1]}" + + out = [ + f'', + f'', + ] + for events in paths: + pts = " ".join(f"{x},{y}" for x, y, _ in events) + out.append(f'') + out += [f'' for x, y, _ in events] + for x, y in STOPS[:-1]: + out.append(f'') + out.append( + '' + f'' + "" + ) + n = sum(len(e) for e in moves) + out.append( + f'{n} mousemove events from' + f" {len(moves)} page.mouse.move() calls, humanize=True, replayed at recorded speed" + ) + out.append("") + return "\n".join(out) + "\n" + + +if __name__ == "__main__": + OUT.write_text(render(asyncio.run(record(sys.argv[1])))) + print(OUT) diff --git a/scripts/developer.py b/scripts/developer.py deleted file mode 100644 index f01baf1e1..000000000 --- a/scripts/developer.py +++ /dev/null @@ -1,362 +0,0 @@ -#!/usr/bin/env python3 - -""" -GUI for managing Camoufox patches. -""" -import os -import re -import sys -import easygui - -from _mixin import find_src_dir, is_bootstrap_patch, list_patches, patch, run, temp_cd - - -def into_camoufox_dir(): - """Cd to the camoufox-* folder""" - this_script = os.path.dirname(os.path.abspath(__file__)) - # Go one directory up from the current script path - os.chdir(os.path.dirname(this_script)) - os.chdir(find_src_dir('.', version=sys.argv[1], release=sys.argv[2])) - - -def reset_camoufox(): - """Reset the Camoufox source""" - with temp_cd('..'): - run('make revert') - run('touch _READY') - - -def run_patches(reverse=False): - """Apply patches""" - patch_files = list_patches() - - # Create a display list with status labels and a reverse lookup mapping - display_choices = [] - mapping = {} - for patch_file in patch_files: - # If the patch is a bootstrap patch, mark it with the appropriate label - if is_bootstrap_patch(patch_file): - status = "BOOTSTRAP" - else: - can_apply, can_reverse, broken = check_patch(patch_file) - if broken: - status = "BROKEN" - elif can_reverse: - status = "APPLIED" - elif can_apply: - status = "NOT APPLIED" - else: - status = "UNKNOWN" - # Format the display string (remove the '../patches/' prefix) - display_name = f"[{status}] {patch_file[len('../patches/'):].strip()}" - display_choices.append(display_name) - mapping[display_name] = patch_file - - title = "Unpatch files" if reverse else "Patch files" - selected_display = easygui.multchoicebox(title, "Patches", display_choices, preselect=[]) - if not selected_display: - return False - - # Convert the selected items back to filenames - for display_name in selected_display: - patch_file = mapping[display_name] - patch(patch_file, reverse=reverse) - return True - -def open_patch_workspace(selected_patch, stop_at_patch=False): - """ - Resets a workspace for editing a patch. - - Process: - 1. Resets Camoufox - 2. Patches all except the selected patch - 3. Sets checkpoint - 4. Reruns the selected patch, but reads rejects similar to "Find broken patches" - """ - # Prepare UI - patch_files = list_patches() - - # Reset workspace - reset_camoufox() - - skipped_patches = [] - applied_patches = [] - # Patch all except the selected patch - for patch_file in patch_files: - if patch_file == selected_patch: - if stop_at_patch: - break - continue - if is_broken(patch_file): - print(f'Skipping broken patch: {patch_file}') - skipped_patches.append(patch_file) - continue - patch(patch_file, silent=True) - applied_patches.append(patch_file) - - # Set checkpoint - if applied_patches: - with temp_cd('..'): - run('make first-checkpoint') - - # Set message for patch result - patch_broken = is_broken(selected_patch) - if patch_broken: - message = "Broken patch has been applied to the workspace.\n\nPLEASE FIX THE FOLLOWING:\n" - else: - message = "Successfully applied patch to the workspace.\n" - - # Run the selected patch - patch_result = os.popen(f'patch -p1 -i "{selected_patch}"').read() - - # Find any line containing a file .rej - if patch_broken: - for line in patch_result.splitlines(): - if file := re.search(r'[^\s]+\.rej', line): - message += f'> {file[0]}' + '\n' - - def msg_format_paths(file_list): - message = '' - for patch_file in file_list: - message += '> ' + patch_file[len('../patches/') :] + '\n' - return message - - # Show which patches were applied if not all patches were allowed - if stop_at_patch and applied_patches: - message += f'\n{"-" * 22} Applied patches {"-" * 22}\n' - message += msg_format_paths(applied_patches) - - if skipped_patches: - message += f'\n{"-" * 17} Skipped patches (broken!) {"-" * 17}\n' - message += msg_format_paths(skipped_patches) - - message += f'\n{"-" * 24} Full output {"-" * 24}\n{patch_result}' - easygui.textbox("Patch Result", "Patch Result", message) - - -def check_patch(patch_file): - """ - Checks if the patch can be applied or can be reversed - Returns (can_apply, can_reverse, is_broken) - """ - can_apply = not bool( - os.system(f'patch -p1 --dry-run --force -i "{patch_file}" > /dev/null 2>&1') - ) - can_reverse = not bool( - os.system(f'patch -p1 -R --dry-run --force -i "{patch_file}" > /dev/null 2>&1') - ) - return can_apply, can_reverse, not (can_apply or can_reverse) - - -def is_broken(patch_file): - """Check if a patch file is broken""" - _, _, is_broken = check_patch(patch_file) - return is_broken - - -def get_rejects(patch_file): - """Get rejects from a patch file""" - cmd = f'patch -p1 -i "{patch_file}" | tee /dev/stderr | sed -n -E \'s/^.*saving rejects to file (.*\\.rej)$/\\1/p\'' - result = os.popen(cmd).read().strip() - return result.split('\n') if result else [] - - -# GUI Choicebox with options -choices = [ - "Reset workspace", - "Edit a patch", - "Create new patch", - "\u2014" * 44, - "List patches currently applied", - "Select patches", - "Reverse patches", - "Find broken patches (resets workspace)", - "\u2014" * 44, - "See current workspace", - "Write workspace to patch", - "Set checkpoint", -] - -""" -GUI Choicebox -""" - - -def handle_choice(choice): - """Handle UI choice""" - match choice: - case "Reset workspace": - reset_camoufox() - easygui.msgbox( - "Reset. All patches & changes have been removed.", - "Reset Complete", - ) - - case "Create new patch": - # Reset camoufox, apply all patches, then create a checkpoint - reset_camoufox() - with temp_cd('..'): - run('make dir') - run('make first-checkpoint') - easygui.msgbox( - "Created new patch workspace. You can test Camoufox with 'make run'.\n\n" - "When you are finished, write your workspace back to a new patch.", - "New Patch Workspace", - ) - - case "List patches currently applied": - # Produces a list of patches that are applied - apply_dict = {} - for patch_file in list_patches(): - print(f'FILE: {patch_file}') - # Ignore bootstrap files, these will always break. - if is_bootstrap_patch(patch_file): - apply_dict[patch_file] = 'IGNORED' - continue - # Check if the patch can be applied or reversed - can_apply, can_reverse, broken = check_patch(patch_file) - if broken: - apply_dict[patch_file] = 'BROKEN' - elif can_reverse: - apply_dict[patch_file] = 'APPLIED' - elif can_apply: - apply_dict[patch_file] = 'NOT APPLIED' - else: - apply_dict[patch_file] = 'UNKNOWN (broken .patch?)' - easygui.textbox( - "Patching Result", - "Patching Result", - '\n'.join( - sorted( - ( - f'{v}\t{k[len("../patches/"):-len(".patch")]}' - for k, v in apply_dict.items() - ), - reverse=True, - key=lambda x: x[0], - ) - ), - ) - - case "Set checkpoint": - with temp_cd('..'): - run('make checkpoint') - easygui.msgbox("Checkpoint set.", "Checkpoint Set") - - case "Select patches": - result = run_patches(reverse=False) - if result: - easygui.msgbox("Patching completed.", "Patching Complete") - - case "Reverse patches": - result = run_patches(reverse=True) - if result: - easygui.msgbox("Unpatching completed.", "Unpatching Complete") - - case "Find broken patches (resets workspace)": - reset_camoufox() - - get_all = None - - broken_patches = [] - for patch_file in list_patches(): - print(f'Testing: {patch_file}') - if reject_files := get_rejects(patch_file): - # Add the patch to the list - broken_patches.append((patch_file, reject_files)) - # If get_all is None, ask the user if they want to get all the rejects - if get_all is None: - get_all = easygui.ynbox( - f"Reject was found: {patch_file}.\nGet the rest of them?", - "Get All Rejects", - choices=["Yes", "No"], - ) - # If the user closed the dialog, return - if get_all is None: - return - # If the user said no, break the patch loop - if not get_all: - break - - if not broken_patches: - easygui.msgbox("All patches applied successfully.", "Patching Result") - return - - # Display message - message = "Some patches failed to apply:\n\n" - for patch_file, rejects in broken_patches: - message += '> ' + patch_file[len('../patches/') :] + '\n' - message += '\n\n\n' - - # Show file contents - for patch_file, rejects in broken_patches: - message += f"Patch: {patch_file[len('../patches/'):]}\nRejects:\n" - for reject in rejects: - with open(reject, 'r') as f: - count = len(re.findall('^@@.*', f.read(), re.MULTILINE)) - message += f"{count} reject(s) > {reject}\n" - message += '\n' - easygui.textbox("Patching Result", "Failed Patches", message) - - case "Edit a patch": - patch_files = list_patches() - ui_choices = [] - for n, file_name in enumerate(patch_files): - # If the patch is bootstrap, label it - if is_bootstrap_patch(file_name): - status = "BOOTSTRAP" - else: - can_apply, can_reverse, broken = check_patch(file_name) - if broken: - status = "BROKEN" - elif can_reverse: - status = "APPLIED" - elif can_apply: - status = "NOT APPLIED" - else: - status = "UNKNOWN" - display_name = f"{n+1}. [{status}] {file_name[len('../patches/'):]}".strip() - ui_choices.append(display_name) - - selected_patch = easygui.choicebox( - "Select patch to open in workspace", - "Patches", - ui_choices, - ) - # Return if user cancelled - if not selected_patch: - return - # Get file path of selected patch - selected_patch = patch_files[ui_choices.index(selected_patch)] - open_patch_workspace( - selected_patch, - # Patches starting with 0- rely on being ran first. - stop_at_patch=is_bootstrap_patch(selected_patch), - ) - - case "See current workspace": - result = os.popen('git diff').read() - easygui.textbox("Diff", "Diff", result) - - case "Write workspace to patch": - # Open a file dialog to select a file to write the diff to - with temp_cd('../patches'): - file_path = easygui.filesavebox( - "Select a file to write the patch to", - "Write Patch", - filetypes="*.patch", - ) - if not file_path: - exit() - run(f'git diff first-checkpoint > {file_path}') - easygui.msgbox(f"Patch has been written to {file_path}.", "Patch Written") - - case _: - print('No choice selected') - - -if __name__ == "__main__": - into_camoufox_dir() - - while choice := easygui.choicebox("Select an option:", "Camoufox Dev Tools", choices): - handle_choice(choice) diff --git a/scripts/examples/buttonclick.html b/scripts/examples/buttonclick.html deleted file mode 100644 index d12c27758..000000000 --- a/scripts/examples/buttonclick.html +++ /dev/null @@ -1,63 +0,0 @@ - - - - - - Random Button Clicker - - - - - - diff --git a/scripts/examples/serve.sh b/scripts/examples/serve.sh deleted file mode 100644 index 9fa39bbdc..000000000 --- a/scripts/examples/serve.sh +++ /dev/null @@ -1 +0,0 @@ -python -m http.server diff --git a/scripts/examples/webgl.html b/scripts/examples/webgl.html deleted file mode 100644 index e6dcc3a09..000000000 --- a/scripts/examples/webgl.html +++ /dev/null @@ -1,341 +0,0 @@ - - - - - - WebGL Fingerprinting - Camoufox - - - -
-

Your WebGL Information (in Camoufox format)

-
- Warning: This browser is not Firefox. The fingerprint below will not run - properly on Camoufox. -
-
- 

-      

-        
-        
-      

-    

-
-    
-  
-
diff --git a/scripts/install-deps.sh b/scripts/install-deps.sh
index 6765421bf..eba1e0d3d 100755
--- a/scripts/install-deps.sh
+++ b/scripts/install-deps.sh
@@ -28,11 +28,10 @@
 #   rust + cargo       Required by `./mach bootstrap` / the build. Installed
 #                      via rustup (not available as a Homebrew keg we control
 #                      the toolchain version of).
-#   p7zip (`7z`)       scripts/package.py + package-helper.sh use `7z`.
+#   p7zip (`7z`)       scripts/package.py uses `7z`.
 #   aria2 (`aria2c`)   `make fetch` downloads the Firefox source tarball.
-#   go / golang        Building the launcher (legacy/launcher).
 #   msitools           `msiextract` — Windows font/redist extraction.
-#   wget               scripts/mozfetch.sh + setup-wasi.
+#   wget               Downloads during `make setup` and `mach bootstrap`.
 #   sqlite             libsqlite3 headers for the Linux build target.
 #   git, curl, make,   Core build tooling. Present by default on macOS via
 #   clang, unzip,      the Xcode Command Line Tools; installed explicitly on
@@ -136,7 +135,7 @@ install_macos() {
 
   # Note: `p7zip` provides the `7z` binary the scripts call; the newer
   # `sevenzip` formula only ships `7zz`.
-  local formulae=(python@3.14 aria2 p7zip go msitools wget sqlite)
+  local formulae=(python@3.14 aria2 p7zip msitools wget sqlite)
   log "Installing Homebrew formulae: ${formulae[*]}"
   brew install "${formulae[@]}"
 
@@ -149,9 +148,9 @@ install_macos() {
 install_linux() {
   log "Detected Linux."
 
-  local debs="python3 python3-dev python3-pip p7zip-full golang-go msitools wget aria2 libsqlite3-dev build-essential make git curl unzip rsync ca-certificates"
-  local rpms="python3 python3-devel p7zip golang msitools wget aria2 sqlite-devel gcc gcc-c++ make git curl unzip rsync ca-certificates"
-  local pacman_pkgs="python python-pip p7zip go msitools wget aria2 sqlite base-devel git curl unzip rsync ca-certificates"
+  local debs="python3 python3-dev python3-pip p7zip-full msitools wget aria2 libsqlite3-dev build-essential make git curl unzip rsync ca-certificates"
+  local rpms="python3 python3-devel p7zip msitools wget aria2 sqlite-devel gcc gcc-c++ make git curl unzip rsync ca-certificates"
+  local pacman_pkgs="python python-pip p7zip msitools wget aria2 sqlite base-devel git curl unzip rsync ca-certificates"
 
   if have apt-get; then
     log "Using apt-get..."
diff --git a/scripts/install-local-build.sh b/scripts/install-local-build.sh
deleted file mode 100755
index ad3ec6e22..000000000
--- a/scripts/install-local-build.sh
+++ /dev/null
@@ -1,141 +0,0 @@
-#!/usr/bin/env bash
-# Install a custom Camoufox build into the local channel.
-#
-# Usage:
-#   ./install-local-build.sh [artifact.zip] [version-build]
-#
-# If no artifact is given, uses the latest zip in dist/.
-# If no version-build is given, extracts it from the zip filename.
-#
-# Installs to ~/.cache/camoufox/browsers/local//
-# and sets active_version in config.json.
-
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
-REPO_ROOT="$(dirname "$SCRIPT_DIR")"
-
-CACHE_DIR="${HOME}/Library/Caches/camoufox"
-# Fall back to XDG if not on macOS
-if [[ ! -d "${HOME}/Library/Caches" ]]; then
-    CACHE_DIR="${XDG_CACHE_HOME:-${HOME}/.cache}/camoufox"
-fi
-
-BROWSERS_DIR="${CACHE_DIR}/browsers"
-CONFIG_FILE="${CACHE_DIR}/config.json"
-
-# --- Resolve artifact zip ---
-
-ARTIFACT="${1:-}"
-if [[ -z "$ARTIFACT" ]]; then
-    ARTIFACT="$(ls -t "$REPO_ROOT"/dist/camoufox-*-mac.arm64.zip 2>/dev/null | head -1)"
-    if [[ -z "$ARTIFACT" ]]; then
-        echo "No artifact found in dist/. Pass the zip path as an argument."
-        exit 1
-    fi
-    echo "Using latest artifact: $ARTIFACT"
-fi
-
-if [[ ! -f "$ARTIFACT" ]]; then
-    echo "Artifact not found: $ARTIFACT"
-    exit 1
-fi
-
-# --- Resolve version-build string ---
-
-VERSION_BUILD="${2:-}"
-if [[ -z "$VERSION_BUILD" ]]; then
-    # Extract from filename: camoufox---mac.arm64.zip
-    BASENAME="$(basename "$ARTIFACT")"
-    # Strip prefix "camoufox-" and suffix "-mac.arm64.zip" (or similar)
-    VERSION_BUILD="${BASENAME#camoufox-}"
-    VERSION_BUILD="${VERSION_BUILD%-mac.*}"
-    VERSION_BUILD="${VERSION_BUILD%-linux.*}"
-    VERSION_BUILD="${VERSION_BUILD%-win.*}"
-fi
-
-echo "Version: $VERSION_BUILD"
-
-# --- Extract version and build parts ---
-
-# version-build format: "146.0.1-ruben.brotli-fix.1"
-# version = everything up to the first hyphen-followed-by-non-digit
-# For simplicity, split on first hyphen after the semver
-VERSION="$(echo "$VERSION_BUILD" | grep -oE '^[0-9]+\.[0-9]+\.[0-9]+')"
-BUILD="${VERSION_BUILD#${VERSION}-}"
-
-INSTALL_DIR="${BROWSERS_DIR}/local/${VERSION_BUILD}"
-
-echo "Installing to: $INSTALL_DIR"
-
-# --- Install ---
-
-if [[ -d "$INSTALL_DIR" ]]; then
-    echo "Removing existing installation..."
-    rm -rf "$INSTALL_DIR"
-fi
-
-mkdir -p "$INSTALL_DIR"
-
-# Unzip to temp dir first to handle nested structure
-TMP_DIR="$(mktemp -d)"
-trap 'rm -rf "$TMP_DIR"' EXIT
-
-unzip -q "$ARTIFACT" -d "$TMP_DIR"
-
-# Handle macOS structure: the zip may contain Camoufox.app directly or nested
-if [[ -d "$TMP_DIR/Camoufox.app" ]]; then
-    mv "$TMP_DIR/Camoufox.app" "$INSTALL_DIR/Camoufox.app"
-elif [[ -d "$TMP_DIR/Camoufox/Camoufox.app" ]]; then
-    mv "$TMP_DIR/Camoufox/Camoufox.app" "$INSTALL_DIR/Camoufox.app"
-else
-    # Linux/Windows: move everything
-    mv "$TMP_DIR"/* "$INSTALL_DIR/"
-fi
-
-# Fix permissions (cp/unzip can strip executable bits)
-chmod -R 755 "$INSTALL_DIR"
-
-# Write version.json
-cat > "$INSTALL_DIR/version.json" < "$CONFIG_FILE"
-fi
-
-echo ""
-echo "Installed: $INSTALL_DIR"
-echo "Active:    $RELATIVE_PATH"
-
-# Verify
-PLIST="$INSTALL_DIR/Camoufox.app/Contents/Info.plist"
-if [[ -f "$PLIST" ]]; then
-    BUNDLE_VERSION="$(/usr/libexec/PlistBuddy -c "Print :CFBundleShortVersionString" "$PLIST" 2>/dev/null || echo "unknown")"
-    echo "Bundle:    $BUNDLE_VERSION"
-fi
-
-echo ""
-echo "Done. Run 'camoufox list' to see installed versions."
diff --git a/scripts/mozfetch.sh b/scripts/mozfetch.sh
deleted file mode 100644
index c6a8eb8f4..000000000
--- a/scripts/mozfetch.sh
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/usr/bin/env bash
-set -e 
-
-rm -f bootstrap.py
-wget -q https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py
-python3 bootstrap.py --no-interactive --application-choice=browser
-rm -f bootstrap.py
diff --git a/scripts/moztree b/scripts/moztree
deleted file mode 100644
index 3b77323a8..000000000
--- a/scripts/moztree
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/usr/bin/env sh
-
-du  | grep -v 'browser/locales/l10n' | grep -v 'js/src/tests' | grep -v './mobile' | grep -v './third_party' | grep -v './testing' | grep -v './js/src/jit-test' | grep -v './devtools/client/debugger/test' | grep -v './dom/canvas/test' | grep -v './browser/components/migration/tests' | grep -v './devtools' | grep -v './layout/reftests' | grep -v './xpcom/tests' | grep -v './gfx/wr/wrench/reftests' | grep -v './netwerk/test' | grep -v './taskcluster' | grep -v './security/nss/doc' | grep -v './security/nss/tests' | grep -v './security/nss/gtests' | grep -v './other-licenses' | grep -v './python/mozperftest/mozperftest' | grep -v './python/mozbuild' | less -S
diff --git a/scripts/package-helper.sh b/scripts/package-helper.sh
deleted file mode 100644
index d42a22b1a..000000000
--- a/scripts/package-helper.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/bash
-
-# package_helper.sh
-add_includes_to_package() {
-    echo "Adding includes to package: $1"
-    temp_dir=$(mktemp -d)
-    7z x "$1" "-o$temp_dir"
-    if [ -d "$temp_dir/camoufox" ]; then
-        mv "$temp_dir/camoufox"/* "$temp_dir/"
-        rmdir "$temp_dir/camoufox"
-    fi
-    for include in "${@:2}"; do
-        if [ -e "$include" ]; then
-            cp -r "$include" "$temp_dir/"
-        fi
-    done
-    (cd "$temp_dir" && 7z u "../$1" ./* -r -tzip -mx=9)
-    rm -rf "$temp_dir"
-}
-
-# Execute the function with all arguments passed to the script
-add_includes_to_package "$@"
\ No newline at end of file
diff --git a/scripts/patch.py b/scripts/patch.py
index c26f91ac9..7523e8a49 100644
--- a/scripts/patch.py
+++ b/scripts/patch.py
@@ -6,12 +6,11 @@
 https://gitlab.com/librewolf-community/browser/source/-/blob/main/scripts/librewolf-patches.py
 
 Run:
-    python3 scripts/init-patch.py  
+    python3 scripts/patch.py  
 """
 
 import hashlib
 import os
-import re
 import shutil
 import subprocess
 import sys
@@ -22,7 +21,6 @@
     get_moz_target,
     get_options,
     list_patches,
-    patch,
     run,
     temp_cd,
 )
@@ -70,30 +68,8 @@ def camoufox_patches(self):
             self._update_mozconfig()
 
             if not options.mozconfig_only:
-                # Apply patches with roverfox patches at the very end
-                all_patches = list_patches()
-                # Normalize paths and partition into non-roverfox and roverfox
-                non_roverfox = []
-                roverfox = []
-                for p in all_patches:
-                    norm = os.path.normpath(p)
-                    parts = norm.split(os.sep)
-                    if 'roverfox' in parts:
-                        roverfox.append(p)
-                    else:
-                        non_roverfox.append(p)
-
-                # Track patch failures
                 failed_patches = []
-
-                # Apply non-roverfox patches first
-                for patch_file in non_roverfox:
-                    rejects = self._apply_and_check(patch_file)
-                    if rejects:
-                        failed_patches.append((patch_file, rejects))
-
-                # Apply roverfox patches last
-                for patch_file in roverfox:
+                for patch_file in list_patches():
                     rejects = self._apply_and_check(patch_file)
                     if rejects:
                         failed_patches.append((patch_file, rejects))
diff --git a/scripts/run-pw.py b/scripts/run-pw.py
deleted file mode 100644
index 29a5e7f69..000000000
--- a/scripts/run-pw.py
+++ /dev/null
@@ -1,82 +0,0 @@
-import argparse
-import json
-import os
-import shutil
-import time
-from pathlib import Path
-
-from _mixin import find_src_dir, get_moz_target, temp_cd
-from playwright.sync_api import sync_playwright
-
-LOCAL_PATH = Path(os.path.abspath(__file__)).parent
-
-CONFIG = {
-    'window.outerWidth': 1920,
-    'window.outerHeight': 1080,
-}
-
-
-def launch_playwright(executable_path):
-    """Launch playwright Firefox with unlimited time"""
-    with sync_playwright() as p:
-        print('Launching browser.', executable_path)
-        browser = p.firefox.launch(
-            executable_path=executable_path,
-            headless=False,
-            args=[
-                '--stderr',
-                str(LOCAL_PATH / 'debug.log'),
-                '--config',
-                json.dumps(CONFIG),
-            ],
-        )
-        page = browser.new_page()
-        url = os.getenv('URL') or 'https://google.com'
-        page.goto(url)
-        try:
-            time.sleep(1e9)
-        except:
-            print('Closing...')
-        finally:
-            browser.close()
-
-
-def get_args():
-    """Get CLI parameters"""
-    parser = argparse.ArgumentParser(
-        description='Package Camoufox for different operating systems.'
-    )
-    parser.add_argument('--version', required=True, help='Camoufox version')
-    parser.add_argument('--release', required=True, help='Camoufox release number')
-    return parser.parse_args()
-
-
-def main():
-    """Run the browser with Playwright"""
-    args = get_args()
-
-    src_dir = find_src_dir('.', args.version, args.release)
-    moz_target = get_moz_target(target='linux', arch='x86_64')
-
-    launcher_path = os.path.abspath(os.path.join('.', 'legacy', 'launcher', 'dist', 'launch'))
-
-    with temp_cd(src_dir):
-        print(f'Looking for file: obj-{moz_target}/dist/bin/camoufox-bin')
-        with temp_cd(f'obj-{moz_target}/dist/bin'):
-            if os.path.exists('camoufox-bin'):
-                # Copy launcher_path to . if we are using camoufox-bin
-                shutil.copy(launcher_path, '.')
-                file_name = 'launch'
-            elif os.path.exists('firefox-bin'):
-                # Or else just use firefox-bin
-                file_name = 'firefox-bin'
-            else:
-                raise FileNotFoundError(f'Binary not found: obj-{moz_target}/dist/bin')
-        file_path = os.path.abspath(f'obj-{moz_target}/dist/bin/{file_name}')
-
-    with temp_cd(os.path.dirname(file_path)):
-        launch_playwright(file_path)
-
-
-if __name__ == '__main__':
-    main()
diff --git a/scripts/setup-wasi-linux.sh b/scripts/setup-wasi-linux.sh
deleted file mode 100644
index 3a1a1c32a..000000000
--- a/scripts/setup-wasi-linux.sh
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/usr/bin/env bash
-
-# resources:
-# https://www.talospace.com/2021/12/firefox-95-on-power.html
-
-distro=linux
-#distro=macos
-
-
-# ac_add_options --with-wasi-sysroot=$HOME/.mozbuild/wrlb/wasi-sysroot
-target_wasi_location=$HOME/.mozbuild/wrlb/
-
-set -e
-
-# taken from: https://github.com/WebAssembly/wasi-sdk/
-export WASI_VERSION=14
-export WASI_VERSION_FULL=${WASI_VERSION}.0
-
-# cleanup first..
-rm -f wasi-sdk-${WASI_VERSION_FULL}-$distro.tar.gz*
-rm -rf wasi-sdk-${WASI_VERSION_FULL}
-
-wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-${WASI_VERSION}/wasi-sdk-${WASI_VERSION_FULL}-$distro.tar.gz
-tar xvf wasi-sdk-${WASI_VERSION_FULL}-$distro.tar.gz
-
-
-# taken from macos: https://gitlab.com/librewolf-community/browser/macos/-/blob/master/build.sh#L109
-if [[ "$distro" == "macos" ]]; then
-    wasi_path=/usr/local/Cellar/llvm/13.0.0_2/lib/clang/13.0.0/lib
-    mkdir $HOME/.mozbuild/wrlb
-    mkdir $wasi_path/wasi
-    cp -r wasi-sdk-14.0/share/wasi-sysroot $HOME/.mozbuild/wrlb/wasi-sysroot
-    cp -v wasi-sdk-14.0/lib/clang/13.0.0/lib/wasi/libclang_rt.builtins-wasm32.a $wasi_path/wasi/
-elif [[ "$distro" == "linux" ]]; then
-     mkdir -p $target_wasi_location
-     rm -rf $target_wasi_location/wasi-sysroot
-     cp -vr wasi-sdk-14.0/share/wasi-sysroot $target_wasi_location
-
-     rm -f wasi-sdk-${WASI_VERSION_FULL}-$distro.tar.gz*
-     rm -rf wasi-sdk-${WASI_VERSION_FULL}
-fi
diff --git a/service-tester/README.md b/service-tester/README.md
index abe8afeb6..253a7652b 100644
--- a/service-tester/README.md
+++ b/service-tester/README.md
@@ -4,7 +4,7 @@ End-to-end antibot-detection tests that verify a pip-installed camoufox release
 
 ## Prerequisites
 
-- Python 3.9+
+- Python 3.10+ (what `pythonlib/` requires)
 - Node.js (for building the TypeScript checks bundle via `esbuild`)
 - At least one proxy in `proxies.txt`
 
@@ -30,7 +30,12 @@ End-to-end antibot-detection tests that verify a pip-installed camoufox release
 
 Use `--binary local` or `--binary fetched` to run only one phase.
 
-> **Heads-up on Phase 2:** if the latest `official/stable` is much older than the local pythonlib (e.g. v135 binary vs pythonlib targeting v149+), the binary may not understand newer fingerprint patches and the test page can fail to produce results. Pin a newer binary with `--browser-version` to avoid this.
+On Windows, `.\run_tests.ps1` does the same in PowerShell, with the local build
+looked up at `..\camoufox-*\obj-*-windows-msvc\dist\bin\camoufox.exe`. Its
+options are the ones below in PowerShell form (`-BrowserVersion`,
+`-ProfileCount`, `-Proxies`, `-Headful`, `-NoCert`, `-SaveCert`, `-Binary`).
+
+> **Heads-up on Phase 2:** if the latest `official/stable` is older than the Firefox version the local pythonlib targets, the binary may not understand newer fingerprint patches and the test page can fail to produce results. Pin a newer binary with `--browser-version` to avoid this.
 
 ## Proxies
 
@@ -85,13 +90,14 @@ python run_tests.py
 python run_tests.py [options]
 
   --browser-version VER   Camoufox version specifier (default: official/stable)
-                          e.g. official/prerelease/146.0.1-beta.50
+                          e.g. official/prerelease/-
   --profile-count N       Number of profiles to test (1-6, default: 6)
   --proxies PATH          Path to proxies file (default: proxies.txt)
   --headful               Run with visible browser window
   --no-cert               Skip certificate generation
   --save-cert PATH        Save certificate text to a file
   --secret KEY            HMAC signing key for the certificate
+                          (run_tests.py only)
   --binary MODE           Which binary to test: local | fetched | both (default: both)
                           (run_tests.sh only — orchestrates the two phases)
   --executable-path PATH  Run against a specific binary path
@@ -102,7 +108,7 @@ python run_tests.py [options]
 
 6 browser contexts run simultaneously — 3 macOS profiles and 3 Linux profiles — each with:
 
-- A unique fingerprint generated by camoufox via fpgen (navigator, screen, WebGL, fonts, voices, audio/canvas seeds)
+- A unique fingerprint generated by camoufox via fpgen (navigator, screen, WebGL, fonts, voices, audio seed)
 - A distinct timezone
 - Its own proxy, with WebRTC ICE candidates spoofed to the proxy's IP
 
@@ -116,7 +122,7 @@ Each context is scored across these categories:
 | Firefox APIs | Firefox-specific API presence |
 | Cross-Signal | Consistency across navigator, screen, etc. |
 | CSS Fingerprint | CSS rendering fingerprint |
-| Canvas Noise | Canvas hash uniqueness and stability |
+| Canvas Noise | Canvas output is identical across renders (no random noise) |
 | WebGL Render | WebGL rendering hash |
 | Audio Integrity | AudioContext fingerprint |
 | Font Platform | OS-consistent font availability |
diff --git a/service-tester/run_tests.ps1 b/service-tester/run_tests.ps1
index 92b1fca21..1087e0080 100644
--- a/service-tester/run_tests.ps1
+++ b/service-tester/run_tests.ps1
@@ -8,7 +8,7 @@
 
 .PARAMETER BrowserVersion
     Camoufox version specifier (default: official/stable)
-    e.g. official/prerelease/146.0.1-beta.50
+    e.g. official/stable/152.0.4-beta.31
 
 .PARAMETER ProfileCount
     Number of profiles to test (1-6, default: 6)
@@ -30,7 +30,7 @@
 
 .EXAMPLE
     .\run_tests.ps1
-    .\run_tests.ps1 -BrowserVersion official/prerelease/146.0.1-beta.50 -Headful
+    .\run_tests.ps1 -BrowserVersion official/stable/152.0.4-beta.31 -Headful
     .\run_tests.ps1 -Binary local
     .\run_tests.ps1 -Binary fetched -ProfileCount 3
 #>
diff --git a/service-tester/run_tests.py b/service-tester/run_tests.py
index da0175ca8..8216eb8d4 100644
--- a/service-tester/run_tests.py
+++ b/service-tester/run_tests.py
@@ -11,7 +11,7 @@
 
 Options:
   --browser-version VER   Camoufox version specifier (default: official/stable)
-                          e.g. official/prerelease/146.0.1-beta.50
+                          e.g. official/stable/152.0.4-beta.31
   --profile-count N       Number of profiles to test (1-6, default: 6)
   --headful               Run with visible browser window
   --proxies PATH          Path to proxies file (default: proxies.txt next to this script)
diff --git a/settings/camoucfg.jvv b/settings/camoucfg.jvv
deleted file mode 100644
index 5dfa791dd..000000000
--- a/settings/camoucfg.jvv
+++ /dev/null
@@ -1,319 +0,0 @@
-{
-    "navigator.userAgent$__UA": "str",
-    "navigator.appVersion$__UA": "str",
-    "navigator.platform$__UA": "str",
-    "navigator.oscpu$__UA": "str",
-    
-    "navigator.appCodeName$__PROD_CODE": "str",
-    "navigator.appName$__PROD_CODE": "str",
-    "navigator.product$__PROD_CODE": "str",
-    "navigator.productSub": "str[/^\\d+$/]",
-    "navigator.buildID": "str[/^\\d+$/]",
-    
-    "screen.height$__SC": "int[>0]",
-    "screen.width$__SC": "int[>0]",
-    "screen.availHeight$__SC": "int[>=0]",
-    "screen.availWidth$__SC": "int[>=0]",
-    "screen.availTop": "int[>=0]",
-    "screen.availLeft": "int[>=0]",
-    "locale:language$__LOCALE": "str",
-    "locale:region$__LOCALE": "str",
-    "locale:script": "str",
-    "geolocation:latitude$__GEO": "double[-90 - 90]",
-    "geolocation:longitude$__GEO": "double[-180 - 180]",
-    "geolocation:accuracy": "double[>=0]",
-    "timezone": "str[/^[\\w_]+/[\\w_]+$/]",
-
-    "locale:all": "str",
-    "headers.Accept-Language": "str",
-    "navigator.language": "str",
-    "navigator.languages": "array[str]",
-    
-    "headers.User-Agent": "str",
-    "headers.Accept-Encoding": "str",
-    "navigator.doNotTrack": "str[0, 1, unspecified]",
-    "navigator.hardwareConcurrency": "int[>0]",
-    "navigator.maxTouchPoints": "int[>=0]",
-    "navigator.cookieEnabled": "bool",
-    "navigator.globalPrivacyControl": "bool",
-    "navigator.onLine": "bool",
-    "window.history.length": "int[>=0]",
-    "pdfViewerEnabled": "bool",
-    
-    "window.outerHeight$__W_OUTER": "int[>0]",
-    "window.outerWidth$__W_OUTER": "int[>0]",
-    "window.innerHeight$__W_INNER": "int[>0]",
-    "window.innerWidth$__W_INNER": "int[>0]",
-
-    "screen.colorDepth": "int[>0]",
-    "screen.pixelDepth": "int[>0]",
-    "screen.pageXOffset": "double",
-    "screen.pageYOffset": "double",
-    "window.scrollMinX": "int",
-    "window.scrollMinY": "int",
-    "window.scrollMaxX": "int",
-    "window.scrollMaxY": "int",
-    "window.screenX": "int",
-    "window.screenY": "int",
-    "window.devicePixelRatio": "double[>0]",
-    
-    "document.body.clientWidth$__DOC_BODY": "int[>=0]",
-    "document.body.clientHeight$__DOC_BODY": "int[>=0]",
-    "document.body.clientTop": "int",
-    "document.body.clientLeft": "int",
-
-    "webrtc:ipv4": "@IPV4",
-    "webrtc:ipv6": "@IPV6",
-    "webrtc:localipv4": "@IPV4",
-    "webrtc:localipv6": "@IPV6",
-
-    "@IPV4": "str[/^(?:[0-9]{1,3}\\.){3}[0-9]{1,3}$/]",
-    "@IPV6": "str[/^(([0-9a-fA-F]{0,4}:){1,7}[0-9a-fA-F]{0,4})$/]",
-
-    "battery:charging$__BATTERY": "bool",
-    "battery:chargingTime$__BATTERY": "double[>=0]",
-    "battery:dischargingTime$__BATTERY": "double[>=0]",
-    "battery:level$__BATTERY": "double[>0]",
-
-    "fonts": "array[str]",
-    "fonts:spacing_seed": "int[>=0]",
-    "audio:seed": "int[>=0]",
-    "canvas:seed": "int[>=0]",
-
-    "AudioContext:sampleRate": "int[>=0]",
-    "AudioContext:outputLatency": "double[>=0]",
-    "AudioContext:maxChannelCount": "int[>=0]",
-
-    "mediaDevices:micros": "int[>=0]",
-    "mediaDevices:webcams": "int[>=0]",
-    "mediaDevices:speakers": "int[>=0]",
-    "mediaDevices:enabled": "bool",
-    "mediaDevices:microphoneLabels": "array[str]",
-    "mediaDevices:microphoneGroups": "array[str]",
-    "mediaDevices:webcamLabels": "array[str]",
-    "mediaDevices:webcamGroups": "array[str]",
-    "mediaDevices:speakerLabels": "array[str]",
-    "mediaDevices:speakerGroups": "array[str]",
-    "media:spoof_codecs": "bool",
-
-    "webGl:renderer$__WEBGL": "str",
-    "webGl:vendor$__WEBGL": "str",
-
-    "webGl:supportedExtensions": "array[str[/^[\\w_]+$/]]",
-    "webGl2:supportedExtensions": "array[str[/^[\\w_]+$/]]",
-
-    "webGl:parameters": "@WEBGL_PARAMS",
-    "webGl2:parameters": "@WEBGL_PARAMS",
-    "webGl:parameters:blockIfNotDefined": "bool",
-    "webGl2:parameters:blockIfNotDefined": "bool",
-
-    "webGl:shaderPrecisionFormats": "@WEBGL_SHADER_PRECISION_FORMATS",
-    "webGl2:shaderPrecisionFormats": "@WEBGL_SHADER_PRECISION_FORMATS",
-    "webGl:shaderPrecisionFormats:blockIfNotDefined": "bool",
-    "webGl2:shaderPrecisionFormats:blockIfNotDefined": "bool",
-
-    "webGl:contextAttributes": "@WEBGL_CONTEXT_ATTRIBUTES",
-    "webGl2:contextAttributes": "@WEBGL_CONTEXT_ATTRIBUTES",
-
-    "@WEBGL_PARAMS": {
-        "2849": "int",
-        "2884": "bool",
-        "2885": "int",
-        "2886": "int",
-        "2928": "array[int, 2]",
-        "2929": "bool",
-        "2930": "bool",
-        "2931": "int",
-        "2932": "int",
-        "2960": "bool",
-        "2961": "int",
-        "2962": "int",
-        "2963": "int",
-        "2964": "int",
-        "2965": "int",
-        "2966": "int",
-        "2967": "int",
-        "2968": "int",
-        "2978": "array[int, 4]",
-        "3024": "bool",
-        "3042": "bool",
-        "3074": "int | nil",
-        "3088": "array[int, 4]",
-        "3089": "bool",
-        "3106": "array[int, 4]",
-        "3107": "array[bool, 4]",
-        "3314": "int | nil",
-        "3315": "int | nil",
-        "3316": "int | nil",
-        "3317": "int",
-        "3330": "int | nil",
-        "3331": "int | nil",
-        "3332": "int | nil",
-        "3333": "int",
-        "3379": "int",
-        "3386": "array[int, 2]",
-        "3408": "int",
-        "3410": "int",
-        "3411": "int",
-        "3412": "int",
-        "3413": "int",
-        "3414": "int",
-        "3415": "int",
-        "7936": "str",
-        "7937": "str",
-        "7938": "str",
-        "10752": "int",
-        "32773": "array[int, 4]",
-        "32777": "int",
-        "32823": "bool",
-        "32824": "int",
-        "32873": "nil",
-        "32877": "int | nil",
-        "32878": "int | nil",
-        "32883": "int | nil",
-        "32926": "bool",
-        "32928": "bool",
-        "32936": "int",
-        "32937": "int",
-        "32938": "int",
-        "32939": "bool",
-        "32968": "int",
-        "32969": "int",
-        "32970": "int",
-        "32971": "int",
-        "33000": "int | nil",
-        "33001": "int | nil",
-        "33170": "int",
-        "33901": "array[double, 2]",
-        "33902": "array[double, 2]",
-        "34016": "int",
-        "34024": "int",
-        "34045": "int | nil",
-        "34047": "nil",
-        "34068": "nil",
-        "34076": "int",
-        "34467": "nil",
-        "34816": "int",
-        "34817": "int",
-        "34818": "int",
-        "34819": "int",
-        "34852": "int | nil",
-        "34853": "int | nil",
-        "34854": "int | nil",
-        "34855": "int | nil",
-        "34856": "int | nil",
-        "34857": "int | nil",
-        "34858": "int | nil",
-        "34859": "int | nil",
-        "34860": "int | nil",
-        "34877": "int",
-        "34921": "int",
-        "34930": "int",
-        "34964": "nil",
-        "34965": "nil",
-        "35071": "int | nil",
-        "35076": "int | nil",
-        "35077": "int | nil",
-        "35371": "int | nil",
-        "35373": "int | nil",
-        "35374": "int | nil",
-        "35375": "int | nil",
-        "35376": "int | nil",
-        "35377": "int | nil",
-        "35379": "int | nil",
-        "35380": "int | nil",
-        "35657": "int | nil",
-        "35658": "int | nil",
-        "35659": "int | nil",
-        "35660": "int",
-        "35661": "int",
-        "35723": "int | nil",
-        "35724": "str",
-        "35725": "nil",
-        "35738": "int",
-        "35739": "int",
-        "35968": "int | nil",
-        "35977": "bool | nil",
-        "35978": "int | nil",
-        "35979": "int | nil",
-        "36003": "int",
-        "36004": "int",
-        "36005": "int",
-        "36006": "nil",
-        "36007": "nil",
-        "36063": "int | nil",
-        "36183": "int | nil",
-        "36203": "int | nil",
-        "36345": "int | nil",
-        "36347": "int",
-        "36348": "int",
-        "36349": "int",
-        "36387": "bool | nil",
-        "36388": "bool | nil",
-        "36392": "nil",
-        "36795": "nil",
-        "37137": "int | double | nil",
-        "37154": "int | nil",
-        "37157": "int | nil",
-        "37440": "bool",
-        "37441": "bool",
-        "37443": "int",
-        "37444": "nil",
-        "37445": "str",
-        "37446": "str",
-        "37447": "int | nil",
-        "38449": "nil"
-    },
-
-    "@WEBGL_SHADER_PRECISION_FORMATS": {
-        "/^\\d+,\\d+$/": {
-            "*rangeMin": "int[>=0]",
-            "*rangeMax": "int[>=0]",
-            "*precision": "int[>=0]"
-        }
-    },
-
-    "@WEBGL_CONTEXT_ATTRIBUTES": {
-        "alpha": "bool",
-        "antialias": "bool",
-        "depth": "bool",
-        "failIfMajorPerformanceCaveat": "bool",
-        "powerPreference": "str[low, high, default]",
-        "premultipliedAlpha": "bool",
-        "preserveDrawingBuffer": "bool",
-        "stencil": "bool"
-    },
-
-    "canvas:aaOffset": "int",
-    "canvas:aaCapOffset": "bool",
-
-    "voices": "array[@VOICE_TYPE]",
-    "voices:blockIfNotDefined": "bool",
-    "voices:fakeCompletion": "bool",
-    "voices:fakeCompletion:charsPerSecond": "double[>0]",
-
-    "@VOICE_TYPE": {
-        "*isLocalService": "bool",
-        "*isDefault": "bool",
-        "*voiceURI": "str",
-        "*name": "str",
-        "*lang": "str"
-    },
-
-    "humanize": "bool",
-    "humanize:maxTime": "double[>=0]",
-    "humanize:minTime": "double[>=0]",
-    "showcursor": "bool",
-
-    "allowMainWorld": "bool",
-    "disableWorldIsolation": "bool",
-    "allowAddonNewtab": "bool",
-    "forceScopeAccess": "bool",
-
-    "disableTheming": "bool",
-    "memorysaver": "bool",
-    "addons": "array[str]",
-    "certificatePaths": "array[str]",
-    "certificates": "array[str]",
-    "debug": "bool"
-}
diff --git a/settings/properties.json b/settings/properties.json
index f51b27a64..078cdab73 100644
--- a/settings/properties.json
+++ b/settings/properties.json
@@ -1,21 +1,14 @@
 [
   { "property": "navigator.userAgent", "type": "str" },
   { "property": "navigator.doNotTrack", "type": "str" },
-  { "property": "navigator.appCodeName", "type": "str" },
-  { "property": "navigator.appName", "type": "str" },
   { "property": "navigator.appVersion", "type": "str" },
   { "property": "navigator.oscpu", "type": "str" },
   { "property": "navigator.language", "type": "str" },
-  { "property": "navigator.languages", "type": "array" },
   { "property": "navigator.platform", "type": "str" },
   { "property": "navigator.hardwareConcurrency", "type": "uint" },
-  { "property": "navigator.product", "type": "str" },
-  { "property": "navigator.productSub", "type": "str" },
   { "property": "navigator.maxTouchPoints", "type": "uint" },
-  { "property": "navigator.cookieEnabled", "type": "bool" },
   { "property": "navigator.globalPrivacyControl", "type": "bool" },
   { "property": "navigator.buildID", "type": "str" },
-  { "property": "navigator.onLine", "type": "bool" },
   { "property": "screen.availHeight", "type": "uint" },
   { "property": "screen.availWidth", "type": "uint" },
   { "property": "screen.availTop", "type": "uint" },
@@ -24,40 +17,20 @@
   { "property": "screen.width", "type": "uint" },
   { "property": "screen.colorDepth", "type": "uint" },
   { "property": "screen.pixelDepth", "type": "uint" },
-  { "property": "screen.pageXOffset", "type": "double" },
-  { "property": "screen.pageYOffset", "type": "double" },
-  { "property": "window.scrollMinX", "type": "int" },
-  { "property": "window.scrollMinY", "type": "int" },
-  { "property": "window.scrollMaxX", "type": "int" },
-  { "property": "window.scrollMaxY", "type": "int" },
   { "property": "window.outerHeight", "type": "uint" },
   { "property": "window.outerWidth", "type": "uint" },
   { "property": "window.innerHeight", "type": "uint" },
   { "property": "window.innerWidth", "type": "uint" },
   { "property": "window.screenX", "type": "int" },
   { "property": "window.screenY", "type": "int" },
-  { "property": "window.history.length", "type": "uint" },
   { "property": "window.devicePixelRatio", "type": "double" },
-  { "property": "document.body.clientWidth", "type": "uint" },
-  { "property": "document.body.clientHeight", "type": "uint" },
-  { "property": "document.body.clientTop", "type": "uint" },
-  { "property": "document.body.clientLeft", "type": "uint" },
   { "property": "headers.User-Agent", "type": "str" },
   { "property": "headers.Accept-Language", "type": "str" },
   { "property": "headers.Accept-Encoding", "type": "str" },
   { "property": "webrtc:ipv4", "type": "str" },
   { "property": "webrtc:ipv6", "type": "str" },
-  { "property": "webrtc:localipv4", "type": "str" },
-  { "property": "webrtc:localipv6", "type": "str" },
-  { "property": "pdfViewerEnabled", "type": "bool" },
-  { "property": "battery:charging", "type": "bool" },
-  { "property": "battery:chargingTime", "type": "double" },
-  { "property": "battery:dischargingTime", "type": "double" },
-  { "property": "battery:level", "type": "double" },
   { "property": "fonts", "type": "array" },
-  { "property": "fonts:spacing_seed", "type": "uint" },
   { "property": "audio:seed", "type": "uint" },
-  { "property": "canvas:seed", "type": "uint" },
   { "property": "geolocation:latitude", "type": "double" },
   { "property": "geolocation:longitude", "type": "double" },
   { "property": "geolocation:accuracy", "type": "double" },
@@ -78,21 +51,15 @@
   { "property": "webGl:supportedExtensions", "type": "array" },
   { "property": "webGl2:supportedExtensions", "type": "array" },
   { "property": "webGl:parameters", "type": "dict" },
-  { "property": "webGl:parameters:blockIfNotDefined", "type": "bool" },
   { "property": "webGl2:parameters", "type": "dict" },
-  { "property": "webGl2:parameters:blockIfNotDefined", "type": "bool" },
   { "property": "webGl:shaderPrecisionFormats", "type": "dict" },
   { "property": "webGl:shaderPrecisionFormats:blockIfNotDefined", "type": "bool" },
   { "property": "webGl2:shaderPrecisionFormats", "type": "dict" },
   { "property": "webGl2:shaderPrecisionFormats:blockIfNotDefined", "type": "bool" },
   { "property": "webGl:contextAttributes", "type": "dict" },
   { "property": "webGl2:contextAttributes", "type": "dict" },
-  { "property": "canvas:aaOffset", "type": "int" },
-  { "property": "canvas:aaCapOffset", "type": "bool" },
   { "property": "voices", "type": "array" },
   { "property": "voices:blockIfNotDefined", "type": "bool" },
-  { "property": "voices:fakeCompletion", "type": "bool" },
-  { "property": "voices:fakeCompletion:charsPerSecond", "type": "double" },
   { "property": "mediaDevices:micros", "type": "uint" },
   { "property": "mediaDevices:webcams", "type": "uint" },
   { "property": "mediaDevices:speakers", "type": "uint" },
@@ -110,8 +77,7 @@
   { "property": "forceScopeAccess", "type": "bool" },
 
   { "property": "disableTheming", "type": "bool" },
-  { "property": "disableInstantAnimations", "type": "bool" },
-  { "property": "memorysaver", "type": "bool" },
+  { "property": "instantAnimations", "type": "bool" },
   { "property": "addons", "type": "array" },
   { "property": "certificatePaths", "type": "array" },
   { "property": "certificates", "type": "array" },
diff --git a/tests/camoufox/README.md b/tests/camoufox/README.md
index 2ed3e3c10..0740c597c 100644
--- a/tests/camoufox/README.md
+++ b/tests/camoufox/README.md
@@ -23,7 +23,11 @@ Write a test here when, and only when, one of these is true:
   to *ignore* the context locale (microsoft/playwright#38919); Camoufox sets the
   locale below that layer, so its workers agree with the main thread. The
   upstream test is skiplisted in `ci/skiplist.yml` and the version here takes
-  over guarding the behaviour.
+  over guarding the behaviour. `test_user_agent_token.py` is the other one:
+  upstream expects `Firefox` in the User-Agent, while the bare binary
+  advertises `Camoufox/` until the Python package injects a
+  fingerprint, so it asserts a well-formed Gecko UA that matches between the
+  request header and `navigator.userAgent`.
 
 In the second case the `ci/skiplist.yml` entry must name the test that replaces
 it, so a skip can never quietly mean "nothing checks this any more".
diff --git a/tests/patches/animation-timing.py b/tests/patches/animation-timing.py
new file mode 100644
index 000000000..eff5b35dc
--- /dev/null
+++ b/tests/patches/animation-timing.py
@@ -0,0 +1,66 @@
+"""
+Verify animations run on stock timing unless `instantAnimations` is set.
+
+no-css-animations.patch can finish every finite animation at once, so Playwright
+never waits on one. That used to be the default, and a page could read it back in
+one line: `el.animate(frames, 1000).effect.getComputedTiming().duration` was 0
+where stock Firefox reports 1000, and a CSS transition reported 0 as well. It is
+now an opt-in.
+
+What PASS means:
+    * by default, a 1000ms Web Animation and a 500ms CSS transition report
+      their real durations;
+    * with config {"instantAnimations": True}, both report 0.
+
+    python tests/patches/animation-timing.py
+"""
+
+import asyncio
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from helpers import resolve_binary  # noqa: E402
+
+PROBE = """() => {
+  const animation = document.body.animate([{opacity: 0}, {opacity: 1}], 1000);
+  const el = document.createElement('div');
+  document.body.append(el);
+  el.style.transition = 'opacity 500ms';
+  el.style.opacity = '0';
+  el.offsetWidth;
+  el.style.opacity = '1';
+  return {
+    animation: animation.effect.getComputedTiming().duration,
+    transition: el.getAnimations().map(a => a.effect.getComputedTiming().duration),
+  };
+}"""
+
+
+async def probe(config):
+    from camoufox.async_api import AsyncCamoufox
+
+    async with AsyncCamoufox(headless=True, os="linux", config=config,
+                             i_know_what_im_doing=True,
+                             executable_path=str(resolve_binary())) as browser:
+        page = await browser.new_page()
+        await page.set_content("")
+        return await page.evaluate(PROBE)
+
+
+async def main() -> int:
+    passed = True
+    for config, expected in (({}, {"animation": 1000, "transition": [500]}),
+                             ({"instantAnimations": True}, {"animation": 0, "transition": [0]})):
+        label = "instantAnimations" if config else "default"
+        got = await probe(dict(config))  # the launcher fills in the dict it is given
+        if got == expected:
+            print(f"  PASS {label}: {got}")
+        else:
+            passed = False
+            print(f"  FAIL {label}: got {got}, expected {expected}")
+    return 0 if passed else 1
+
+
+if __name__ == "__main__":
+    sys.exit(asyncio.run(main()))
diff --git a/tests/patches/config-overrides.py b/tests/patches/config-overrides.py
deleted file mode 100644
index 2c51b62a4..000000000
--- a/tests/patches/config-overrides.py
+++ /dev/null
@@ -1,152 +0,0 @@
-"""
-Verify that config_overrides={'fonts:spacing_seed': 0} disables font spacing perturbation.
-
-The bug: there was no way to disable font spacing perturbation through the Python API.
-generate_context_fingerprint() always generated a random non-zero seed, and the caller
-couldn't override it because init_script was already rendered by the time config was
-returned. config_overrides applies after config is built but before init_script is
-rendered, giving callers a clean override point.
-
-Run:
-    cd ~/20tech/drivingtest/dvsa-bot
-    uv run python ~/20tech/oss/camoufox/tests/patches/2026-04-30-font-spacing-seed-override.py
-"""
-
-import asyncio
-import sys
-
-from helpers import MAX_PRESET_ATTEMPTS
-
-
-async def test():
-    from camoufox.async_api import AsyncCamoufox
-    from camoufox.fingerprints import generate_context_fingerprint, get_random_preset
-
-    test_string = "The quick brown fox jumps over the lazy dog"
-    failures = []
-
-    # --- Test 1: config_overrides disables font spacing perturbation ---
-    print("=== Test 1: config_overrides={'fonts:spacing_seed': 0} ===")
-
-    last_error = None
-    for attempt in range(MAX_PRESET_ATTEMPTS):
-        preset = get_random_preset(os="macos")
-        fp = generate_context_fingerprint(
-            preset=preset,
-            config_overrides={"fonts:spacing_seed": 0},
-        )
-
-        if fp["config"]["fonts:spacing_seed"] != 0:
-            failures.append(
-                f"Config seed is {fp['config']['fonts:spacing_seed']}, expected 0"
-            )
-            break
-
-        try:
-            async with AsyncCamoufox(
-                fingerprint_preset=fp["preset"],
-                headless=True,
-                os="macos",
-            ) as browser:
-                context = await browser.new_context(**fp["context_options"])
-                await context.add_init_script(fp["init_script"])
-                page = await context.new_page()
-                await page.goto("about:blank")
-
-                widths = await page.evaluate(
-                    """(testStr) => {
-                    const canvas = document.createElement('canvas');
-                    const ctx = canvas.getContext('2d');
-                    const results = [];
-                    for (let i = 0; i < 5; i++) {
-                        ctx.font = '16px Arial';
-                        results.push(ctx.measureText(testStr).width);
-                    }
-                    return results;
-                }""",
-                    test_string,
-                )
-
-                unique = set(widths)
-                if len(unique) == 1:
-                    print(f"  Measurements stable (all {widths[0]}): PASS")
-                else:
-                    failures.append(f"Measurements unstable with seed=0: {widths}")
-                    print(f"  Measurements vary: {widths}: FAIL")
-                break
-        except ValueError as e:
-            if "WebGL" in str(e):
-                last_error = e
-                continue
-            raise
-    else:
-        raise RuntimeError("Could not find a valid preset") from last_error
-
-    # --- Test 2: without config_overrides, the perturbation is OFF (seed 0) ---
-    # Glyph-advance perturbation moves every measured text width off the value
-    # the same font gives on a real machine, so the default is 0; an explicit
-    # non-zero seed must still be honoured (opt-in).
-    print("\n=== Test 2: default (no overrides) seed is 0, explicit seed honoured ===")
-    preset2 = get_random_preset(os="macos")
-    fp2 = generate_context_fingerprint(preset=preset2)
-    seed2 = fp2["config"]["fonts:spacing_seed"]
-    if seed2 == 0:
-        print("  Default seed is 0 (perturbation off): PASS")
-    else:
-        failures.append(f"Default seed is {seed2} — should be 0 (perturbation off by default)")
-        print(f"  Default seed is {seed2}: FAIL")
-    fp2b = generate_context_fingerprint(preset=preset2, config_overrides={"fonts:spacing_seed": 12345})
-    if fp2b["config"]["fonts:spacing_seed"] == 12345:
-        print("  Explicit seed 12345 honoured: PASS")
-    else:
-        failures.append("Explicit fonts:spacing_seed override was not honoured")
-        print("  Explicit seed override: FAIL")
-
-    # --- Test 3: init_script contains setFontSpacingSeed(0) when overridden ---
-    print("\n=== Test 3: init_script emits setFontSpacingSeed(0) ===")
-    preset3 = get_random_preset(os="macos")
-    fp3 = generate_context_fingerprint(
-        preset=preset3,
-        config_overrides={"fonts:spacing_seed": 0},
-    )
-    if "setFontSpacingSeed(0)" in fp3["init_script"]:
-        print("  init_script contains setFontSpacingSeed(0): PASS")
-    elif "setFontSpacingSeed" not in fp3["init_script"]:
-        print("  init_script omits setFontSpacingSeed entirely: PASS (acceptable)")
-    else:
-        import re
-
-        match = re.search(r"setFontSpacingSeed\((\d+)\)", fp3["init_script"])
-        val = match.group(1) if match else "?"
-        failures.append(f"init_script has setFontSpacingSeed({val}), expected 0")
-        print(f"  init_script has setFontSpacingSeed({val}): FAIL")
-
-    # --- Test 4: other seeds are NOT affected by a font-only override ---
-    print("\n=== Test 4: audio/canvas seeds unaffected by font override ===")
-    preset4 = get_random_preset(os="macos")
-    fp4 = generate_context_fingerprint(
-        preset=preset4,
-        config_overrides={"fonts:spacing_seed": 0},
-    )
-    audio = fp4["config"]["audio:seed"]
-    canvas = fp4["config"]["canvas:seed"]
-    if audio != 0 and canvas != 0:
-        print(f"  audio:seed={audio}, canvas:seed={canvas} (both non-zero): PASS")
-    else:
-        failures.append(f"Other seeds affected: audio={audio}, canvas={canvas}")
-        print(f"  audio={audio}, canvas={canvas}: FAIL")
-
-    # --- Summary ---
-    print("\n" + "=" * 50)
-    if failures:
-        print(f"FAILED ({len(failures)} issues):")
-        for f in failures:
-            print(f"  - {f}")
-        return 1
-    else:
-        print("ALL TESTS PASSED")
-        return 0
-
-
-if __name__ == "__main__":
-    sys.exit(asyncio.run(test()))
diff --git a/tests/patches/fingerprint-setter-seal.py b/tests/patches/fingerprint-setter-seal.py
index bed8d0079..7f4ae4c55 100644
--- a/tests/patches/fingerprint-setter-seal.py
+++ b/tests/patches/fingerprint-setter-seal.py
@@ -65,7 +65,6 @@
     "setWebRTCIPv4",
     "setWebRTCIPv6",
     "setFontList",
-    "setFontSpacingSeed",
     "setAudioFingerprintSeed",
     "setSpeechVoices",
     "setTimezone",
diff --git a/tests/patches/spoofed-voice-speaks.py b/tests/patches/spoofed-voice-speaks.py
new file mode 100644
index 000000000..56ce0b295
--- /dev/null
+++ b/tests/patches/spoofed-voice-speaks.py
@@ -0,0 +1,87 @@
+"""
+Verify speechSynthesis.speak() on a spoofed voice behaves like a real voice.
+
+A spoofed voice has no speech engine behind it. voice-spoofing.patch used to fire
+an `error` event for it unless `voices:fakeCompletion` was set, and even then it
+fired `start` and `end` in the same instant. A real voice never errors on a plain
+utterance, and its `end` arrives after the text has been spoken. Both were tells.
+
+The host's own speech backend must not matter either. On a Linux host where
+speech-dispatcher cannot start, Firefox broadcasts a voices error, and every
+page used to error its queued utterances. A spoofed Windows identity's voices
+do not depend on the host's daemon, so the guard runs with it unreachable.
+
+What PASS means:
+    * the page sees spoofed voices at all (so the check is not vacuous);
+    * speaking 25 characters on one fires `start`, then `end`, and no `error`;
+    * `end` arrives about as long after `start` as the text takes to say at
+      ~150 words per minute (2s here), not in the same tick.
+
+    python tests/patches/spoofed-voice-speaks.py
+"""
+
+import asyncio
+import os
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from helpers import resolve_binary  # noqa: E402
+
+TEXT = "a" * 25  # 25 chars at 12.5 chars/s = 2.0s
+PROBE = """async (text) => {
+  let voices = speechSynthesis.getVoices();
+  if (!voices.length) {
+    await new Promise(r => { speechSynthesis.onvoiceschanged = r; setTimeout(r, 3000); });
+    voices = speechSynthesis.getVoices();
+  }
+  if (!voices.length) return {voices: 0};
+  const u = new SpeechSynthesisUtterance(text);
+  u.voice = voices[0];
+  const events = [];
+  const t0 = performance.now();
+  const done = new Promise(resolve => {
+    for (const type of ['start', 'end', 'error']) {
+      u.addEventListener(type, () => {
+        events.push([type, Math.round(performance.now() - t0)]);
+        if (type !== 'start') resolve();
+      });
+    }
+    setTimeout(resolve, 10000);
+  });
+  speechSynthesis.speak(u);
+  await done;
+  return {voices: voices.length, events};
+}"""
+
+
+async def main() -> int:
+    from camoufox.async_api import AsyncCamoufox
+
+    # An unreachable speech-dispatcher socket: the host backend fails to start,
+    # as it does on a machine without the daemon.
+    env = {**os.environ, "SPEECHD_ADDRESS": "unix_socket:/nonexistent/speechd.sock"}
+    async with AsyncCamoufox(headless=True, os="windows", env=env,
+                             executable_path=str(resolve_binary())) as browser:
+        page = await browser.new_page()
+        await page.goto("about:blank")
+        got = await page.evaluate(PROBE, TEXT)
+    print(f"  {got}")
+
+    if not got["voices"]:
+        print("  FAIL: the page sees no voices, so nothing was tested")
+        return 1
+    kinds = [kind for kind, _ in got["events"]]
+    if kinds != ["start", "end"]:
+        print(f"  FAIL: expected start then end, got {kinds}")
+        return 1
+    spoken = got["events"][1][1] - got["events"][0][1]
+    if not 1500 <= spoken <= 4000:
+        print(f"  FAIL: end came {spoken}ms after start, expected about 2000ms")
+        return 1
+    print(f"  PASS: start, then end {spoken}ms later, no error")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(asyncio.run(main()))
diff --git a/tests/patches/stock-parity-probes.py b/tests/patches/stock-parity-probes.py
index 6d96ffeb9..a6fddf570 100644
--- a/tests/patches/stock-parity-probes.py
+++ b/tests/patches/stock-parity-probes.py
@@ -143,15 +143,31 @@ def log_message(self, *args):
     for (let i = 0; i < 20000; i++) sink += fn();
     return performance.now() - t;
   };
-  out.costColor = time(() => matchMedia('(color: 8)').matches ? 1 : 0);
-  out.costMinWidth = time(() => matchMedia('(min-width: 1px)').matches ? 1 : 0);
-  out.costHwc = time(() => navigator.hardwareConcurrency);
-  out.costUA = time(() => navigator.userAgent.length);
-  out.sink = sink;
+  // Each pair is timed ROUNDS times, interleaved, and compared by median. One
+  // sample per getter let a single GC pause or CPU-steal spike on a shared
+  // runner decide the verdict (hardwareConcurrency once took 77 ms against a
+  // 50 ms allowance on a healthy build that passed the run before). The
+  // regressions these catch cost extra on EVERY read, so they move every
+  // sample and the median with them; a one-off spike moves one sample.
+  const ROUNDS = 5;
+  const median = (xs) => xs.slice().sort((a, b) => a - b)[xs.length >> 1];
+  const pair = (a, b) => {
+    const ta = [], tb = [];
+    for (let r = 0; r < ROUNDS; r++) { ta.push(time(a)); tb.push(time(b)); }
+    return [median(ta), median(tb)];
+  };
+  [out.costColor, out.costMinWidth] = pair(
+    () => matchMedia('(color: 8)').matches ? 1 : 0,
+    () => matchMedia('(min-width: 1px)').matches ? 1 : 0);
+  [out.costHwc, out.costUA] = pair(
+    () => navigator.hardwareConcurrency,
+    () => navigator.userAgent.length);
   // Fresh Date objects: a Date caches its local-time fields after one read.
   let n = 0;
-  out.costLocalDate = time(() => new Date(1.6e12 + (n++) * 3.6e6).getHours());
-  out.costUTCDate = time(() => new Date(1.6e12 + (n++) * 3.6e6).getUTCHours());
+  [out.costLocalDate, out.costUTCDate] = pair(
+    () => new Date(1.6e12 + (n++) * 3.6e6).getHours(),
+    () => new Date(1.6e12 + (n++) * 3.6e6).getUTCHours());
+  out.sink = sink;
   out.timeZone = Intl.DateTimeFormat().resolvedOptions().timeZone;
   return out;
 }"""
diff --git a/upstream.sh b/upstream.sh
index 1d3c67470..c3382e74b 100644
--- a/upstream.sh
+++ b/upstream.sh
@@ -1,3 +1,2 @@
 version=152.0.4
 release=beta.31
-closedsrc_rev=1.0.0