Repository navigation
Refacto/perf reliability - #11
Merged
Merged
Conversation
…ction bugs Phase 0 (safety net): - add analyze/analyze_golden_test.go: detection golden tests that run without Chrome, pinning the detected-technology set across the refactor. Phase 1 (quick wins): - analyze/regexcache.go: compile each Wappalyzer pattern once (process-wide cache) and skip RE2-incompatible patterns instead of panicking; bounds-checked version extraction. Compilation drops from O(pages*patterns) to O(patterns) and the per-match double-compile is removed. - convert all matchers (headers/html/meta/scriptSrc/dom/url/xhr/dns) to the cache. - analyze_javascript.go: fix missing paren in the typeof eval (dead JS branch) and read confidence from regex[2]. - cmd/root.go: fix leaked WithTimeout cancel; cdncheck/goquery errors no longer abort the whole run; reuse the goquery document via Analyze.Doc; surface the previously discarded InitDialer error. - main.go: buffer the results channel (Threads*ChromeThreads). - technologies.go: close fingerprint files per iteration, verify HTTP 200 before writing a shard, and report unmarshal/merge errors. Verified: go build ./..., go vet ./..., go test -race ./analyze/
…l mutation
Build the HTTP client once in Start() via buildHTTPClient() and share it
read-only across all goroutines. Removes three data races present when
threads>1:
- per-target reassignment of the shared c.HttpClient field in startPortScan
- two per-request mutations of the process-global http.DefaultTransport
(in startPortScan and getWrapper)
Also deletes the dead getClientCtx branch (c.HttpClient == &http.Client{} was
a pointer comparison against a fresh allocation, never true). Proxy/TLS config
is now applied once from Options instead of being rebuilt identically per
target.
Verified: go build ./..., go vet ./..., go test -race ./analyze/
- portOpen: scanner goroutines now report open ports on a buffered channel drained after swg1.Wait(), instead of appending to a shared slice (race). - PortOpenByIP: reads (CheckIpAlreadyScan) and the append are guarded by a new Cmd.portMu mutex, removing the slice data race. A benign TOCTOU remains (two targets sharing an IP may both scan it once) but it can no longer corrupt the cache. cmd.Cmd now holds a sync.Mutex, so wrapper.configureOptions returns *cmd.Cmd instead of copying it by value. Verified: go build ./..., go vet ./..., go test -race ./analyze/
The chromedp ListenTarget callback appends XHR response URLs from a separate goroutine while Analyze.Run() reads the slice in the page ActionFunc, a data race. XHRUrl is now guarded by a per-Analyze mutex: writes go through AddXHRUrl(), and analyze_xhr_main() ranges over a snapshot taken under the lock. Verified: go build ./..., go vet ./..., go test -race ./analyze/
…ingerprints A single malformed target or odd fingerprint entry no longer aborts the whole run: - per-target and per-port goroutines now recover() and log the offending host instead of crashing the process. - Amass input parsing checks the json.Unmarshal error and uses comma-ok for name/addresses/ip; a malformed line is skipped, a missing address leaves ip empty (the host is still scanned by name) rather than panicking. - NewTechno, AddTechno and CheckRequired guard every map/type assertion, so an implies/requires/cpe entry that is missing or non-string is tolerated. Adds TestImpliesMissingTechnoNoPanic covering an implies edge to a technology absent from the DB. Verified: go build ./..., go vet ./..., go test -race ./analyze/
DownloadTechnologies created its working directory in the current working directory via a random name, so the 27 downloaded Wappalyzer shards could be left in the repo (and swept into a commit by `git add -A`). Use os.MkdirTemp so the shards live under the OS temp location instead; callers already os.RemoveAll the returned folder. Also ignore wappaGo_report.html. Verified: go build ./..., go vet ./..., go test -race ./analyze/
The Chrome ExecAllocator now derives from signal.NotifyContext, so the first Ctrl+C cancels the browser context and lets the deferred cleanup run (Chrome processes, dialer, temp fingerprint dir) instead of orphaning them. A Chrome startup failure now reports the error, closes the results channel and returns (so the stdout consumer exits) instead of panicking and skipping cleanup. Verified: go build ./..., go vet ./..., go test -race ./analyze/
- card() now renders through html/template, which contextually escapes the page title, scanned URL, screenshot path and detected technology names. A malicious scanned site can no longer inject script into the operator's report (stored XSS). Adds report/report_test.go covering the escaping. - Report_main is now called once, after swg.Wait(), from the full result set instead of being regenerated after every host (O(n^2) full-file rewrites by concurrent writers). The ResultArray append is guarded by Cmd.reportMu. Verified: go build ./..., go vet ./..., go test -race ./analyze/ ./report/
…ions The previous filename was the URL with ':' '/' '.' stripped, so distinct URLs (e.g. host:443 vs host) could map to the same name and silently overwrite each other's screenshot. Use the hex SHA-1 of the URL instead; the report reference (data.Infos.Screenshot) uses the same value, so the gallery stays consistent. Also guards the previously ignored os.OpenFile error. Verified: go build ./..., go vet ./...
Previously CheckRequired fed "requires" through AddTechno exactly like "implies", so a required technology was added unconditionally whenever the dependent was detected - a systematic false positive (e.g. a plugin's required platform reported even when absent, and the plugin kept even when its platform wasn't there). Now: - CheckRequired only applies "implies" (unchanged: X implies Y => add Y). - FilterRequired runs after dedup and drops any detected technology whose "requires" set isn't satisfied by the rest of the detected technologies, iterating to a fixpoint for requires chains. "requiresCategory" is not enforced (categories aren't loaded). Adds TestRequiresGate. Existing implies tests are unchanged, confirming "implies" behaviour is preserved. Verified: go build ./..., go vet ./..., go test -race ./analyze/ ./report/
Replace the deprecated io/ioutil.ReadAll with io.ReadAll in cmd/root.go and technologies/technologies.go, dropping the io/ioutil import. No behaviour change; clears the deprecation warnings. Verified: go build ./..., go vet ./..., go test -race ./...
Full visual + information redesign of the report, self-contained (no external Bootstrap/jQuery/CDN, so it works offline and leaks nothing). Information: each card now surfaces IP, CDN, content-type, humanised size, response time, open ports, redirect Location, technology chips with version (confidence/cpe on hover), plus collapsible cert SANs and CNAME lists - instead of just title/url/status/techs. A header strip summarises total hosts, a status-distribution bar (2xx/3xx/4xx/5xx) with a legend, the distinct technology count and the generation time. A live client-side filter narrows cards by host/ip/tech, and screenshots open in a lightbox. Design: dark "recon console / contact sheet" theme, monospace-forward identity, single cyan accent, status shown as a coloured stamp with a band accent. Responsive to mobile, keyboard-focusable, honours prefers-reduced-motion. Security: still rendered through html/template, so all target-controlled fields (title, host, cert SANs, technology names) stay escaped. Only integers/labels are assembled outside the template. Adds TestReportRendersFullPage covering the varied-host layout and the escaping end to end. Verified: go build ./..., go vet ./..., go test ./... (+ visual check via a headless-Chrome screenshot of a sample report).
WappaGo's raw HTTP probe sent Go's default "Go-http-client/1.1" User-Agent with no other headers, and Chrome ran with a "HeadlessChrome" UA and the automation flag on — both are instant WAF/bot-detection tells, and the two phases looked like two different clients from one IP. Now a single, consistent browser identity is used across both phases: - the probe sets a realistic User-Agent plus Accept, Accept-Language, Sec-Fetch-*, Upgrade-Insecure-Requests and Sec-CH-UA (derived from the Chrome major version). Accept-Encoding is left unset so Go keeps decompressing gzip. - Chrome gets chromedp.UserAgent(...) (drops "HeadlessChrome") and --disable-blink-features=AutomationControlled (navigator.webdriver = false). - new -user-agent flag / WrapperOptions.UserAgent; defaults to a current Chrome UA (structure.DefaultUserAgent). Both phases use the same value. Verified: TestSetBrowserHeadersOnWire asserts the real headers over the wire (no Go UA), TestChromeMajor covers the version parse, and a headless-Chrome check confirmed UA without "Headless" and navigator.webdriver=false. go build ./..., go vet ./..., go test ./...
… WAFs
Adds an opt-in throttle so a scan stays under a target's rate limits:
- -rps: max HTTP requests per second per host. Each host gets its own schedule
(slot reserved under a short lock, the wait happens outside it), so distinct
hosts never block each other and bursts to one host are evenly spaced.
- -jitter: max random delay (ms) added before each request, so the cadence
isn't metronomic.
Applied at every outbound HTTP point: the IP-resolution GET, both probe
requests in getWrapper, and the Chrome navigation. Defaults (0/0) make the
throttle a no-op, so existing behaviour and speed are unchanged unless opted in.
WrapperOptions gains Rps/Jitter too.
Verified: go build ./..., go vet ./..., go test ./... including
TestHostThrottle{Disabled,PacesPerHost,IndependentHosts}.
Adds a single-line progress bar showing how many hosts have been processed out of the total. It is drawn on stderr only, so it never corrupts the JSON results on stdout, and it is shown only when stderr is a real terminal (so piped/ redirected runs stay clean). Each target increments it on completion (including skipped Amass lines and recovered panics, via defer), and the line is closed after swg.Wait(). A -no-progress flag disables it; library mode (wrapper) keeps it off. Adds TestProgressLine and TestProgressDisabledNoPanic. Verified: go build ./..., go vet ./..., go test ./...
…ton) The zoom worked in current builds, but the per-element click handlers and click-anywhere-to-close were fragile. Switch to a single document-level delegated handler so any screenshot (clicked on the image itself) opens the lightbox, add an explicit × close button, make the bare `<button>` type="button", and guard the filter against a missing count element. Verified with a headless-Chrome check: clicking the screenshot image opens the lightbox with the full-resolution image, and the close button dismisses it. Note: a report produced by a pre-redesign build has no working zoom — rebuild and regenerate the report to get this. go build ./..., go vet ./..., go test ./...
…own debt Fingerprints now come from https://github.com/Serizao/tech (shards at the repo root, not under technologies/). structure.TechnologiesRoot replaces the misspelled WappazlyerRoot, which stays as a deprecated alias. New technologies.Load() downloads, parses, and deletes the on-disk copy before returning: the shards were only ever read once, so holding the temp directory for the whole scan served no purpose and leaked it outright when the download failed early. Both main.go and wrapper go through it, which also removes the drifted second copy of the bootstrap, and a failure is now fatal instead of a logged warning followed by a scan reporting zero technologies everywhere. Detection fixes: - headers: the pattern was looked up with the *response* header name while the comparison used the *fingerprint* key. Resp.Headers is an http.Header clone, so its keys are Go-canonical; 301 of the database's 690 header fingerprints (43%, over 211 technologies) missed, returned a nil interface, and since nil != "" went on to compile the literal "<nil>" as a regex. Also inspect every value of a repeated header, not just values[0]. - headers are now taken from the union of the raw probe and what Chrome saw on the final document, Chrome winning on conflicts. Neither alone is enough: the probe stops at the 30x so its headers describe the redirector (a http->https host reported no Server at all), but Chrome consumes Alt-Svc, the only HTTP/3 signal. CDP newline-joins repeated headers, so they are split back apart. - url: matched against the Location header and skipped entirely unless the host redirected, so all 78 url fingerprints were unreachable on a normal response. Now matched against the scanned URL, redirect target included. - meta: only meta[name=] was searched, so every og:/twitter:/fb: fingerprint (published as meta[property=]) could never match. - text vs html: both ran against the raw markup, so a text pattern could fire on an attribute value or a URL invisible to a visitor. - dom: a text/value pattern re-scanned the entire page body once per matched element; it now reads the selected element, as Wappalyzer intends. - js: `.match(/re/gm)[0]` cannot work with \;version:\N because the g flag discards capture groups, threw on a null match, and broke on any pattern containing "/". Rebuilt via new RegExp from a quoted literal, reading the group the marker names. - implies markers are stripped: the database ships PHP\;confidence:75, which was used verbatim as a technology name and also broke the requires gate. - excludes is honoured (43 edges, previously ignored). Mutually exclusive pairs are kept, since deleting both discards evidence rather than resolving it. - DedupTechno no longer depends on map iteration order, which made the reported version change between runs on identical input for 86 technologies. - every fingerprint value is type-asserted with comma-ok, including in Run() itself, and a nil DnsData no longer nil-derefs the dns matcher. Technologie gains an icon field, copied from the fingerprint database and omitted when it declares none. Debt: - Response is trimmed to the four fields anything reads. Raw, RawHeaders, Data, Words, Lines, TLSData, HTTP2 and Pipeline were write-only, and populating Raw meant buffering every probed body whole, twice, with no size cap. - Response.Duration is actually assigned, so response_time is no longer always 0. - Options.ApplyDefaults() runs at the top of cmd.Start, removing the nil-deref surface across fourteen pointer fields, giving library callers the same defaults as the CLI flags, and closing the -threads 0 hole (sizedwaitgroup treats limit<=0 as unbounded). - the per-IP port cache is a map under an RWMutex instead of a slice linearly scanned while holding an exclusive lock, and is skipped when the IP is empty rather than letting every unresolved host share one bogus entry. - InitDialer returns an error instead of a nil *Dialer the caller uses anyway. - the warm-up GET's body is drained and closed. - SIGINT stops the scan instead of only killing Chrome, and a second one aborts. - the JavaScript-dialog handler no longer marshals the shared result struct onto stdout while another goroutine writes it. - dead code removed: lib.RandStringBytes, lib.GetKey, lib.CheckIpAlreadyScan, Analyze.CSSContent, structure.PortOpenByIp, structure.LetterBytes. go.mod moves 1.19 (EOL) to 1.23, which also gives per-iteration loop variables; go mod tidy drops cryptoutil to indirect and httputil entirely. CI now runs on every branch with a gofmt check, vet and go test -race, and reads the Go version from go.mod instead of a second copy of it. wrapper.StartReconSync/StartReconAsync now return an error. Breaking, but ignoring a failed fingerprint download meant silently scanning with an empty database. README updated. 43 tests, including regression guards for each detection fix above. gofmt -l . clean, go vet ./... clean, go test -race ./... green
…es it
The browser gave itself away in three measurable ways. Each fix below was kept
or dropped on the evidence of the new harness, not on reputation.
structure.DefaultUserAgent was pinned to Chrome 131 while the installed browser
was 150, and --user-agent does not touch the Client Hints Chrome derives from its
own build. Measured against a local echo server, every request advertised two
different versions at once:
User-Agent: ...Chrome/131.0.0.0 Sec-CH-UA: "Google Chrome";v="150"
resolveIdentity now reads the real build with Browser.getVersion, rewrites the
HeadlessChrome token (new headless still emits it, so the override cannot simply
be dropped), and applyIdentity installs it through
Emulation.setUserAgentOverride with matching userAgentMetadata. navigator.userAgent,
navigator.userAgentData and the Sec-CH-UA-* headers now all come from one source.
A hardcoded UA survives only as a fallback when the browser cannot be queried,
and -user-agent still wins while the metadata keeps following the real build.
The flag list no longer starts from chromedp.DefaultExecAllocatorOptions, whose
last entry is Flag("enable-automation", true) — the marker itself. The useful
flags are copied in explicitly so each one is a decision. --enable-automation is
the only one actually dropped: removing the throughput flags alongside it was
measured to cost 47 detections and to time out two hosts, because site isolation
gives every third-party iframe its own process, multiplied by the tab count.
--disable-webgl is gone. An absent WebGL context is a strong bot signal, and
Camoufox spoofs WebGL rather than removing it. Forcing SwiftShader instead was
worse still: software rendering pushed 7 of 20 targets past the 60 s budget,
returning zero technologies. With no graphics flags at all Chrome reports the
real adapter, which is both the most plausible fingerprint and free.
screen was 800x600 while the window was 1400x900 — a window larger than its own
display. Emulation.setDeviceMetricsOverride now declares a 1920x1080 screen
around the viewport. Browser.setWindowBounds was tried for
window.outerWidth/outerHeight and removed again: no measurable effect, and it
made Page.captureScreenshot fail with -32000, which aborted the action chain and
took every matcher down with it.
That last failure exposed a standing fragility: the screenshot sat unconditionally
ahead of the detection ActionFunc, so any capture error cost the whole host, and
it ran even without -screenshot, encoding a PNG per page only to discard it. It
is now conditional and non-fatal.
Accept-Language was en-US on the probe while Chrome used the host locale (fr-FR
here), so the two clients contradicted each other on every host. Both are pinned,
and Accept now matches Chrome byte for byte.
cmd/browser_stealth_test.go is the measurement, opt-in because it starts a real
Chrome (WAPPAGO_STEALTH_CHECK=1). It serves its own page, so verdicts are
reproducible, and it reports rather than hides two residuals: canvas, WebGL and
audio fingerprints are asserted present and *stable* — deliberately not
randomised, since a fingerprint that changes between reads is itself a marker of
an anti-fingerprinting tool — and window.outerWidth/outerHeight stay 0, which no
CDP call fixes and which the usual injected-script workaround would trade for a
broader tell.
Detection unchanged: 149/150/150 technologies over the 20-target reference set
across three runs, no host at zero.
gofmt -l . clean, go vet ./... clean, go test -race ./... green
The previous commit gave the browser a coherent identity, but the raw HTTP probe
is what a scanned host actually sees first: it runs before Chrome on every
target, twice or three times per host. Its handshake was Go's, and no User-Agent
can cover that up — JA3/JA4 key on the ClientHello, which is decided before a
single header is sent.
Measured against a local server, the difference is not subtle:
cipher suites first suite ALPN offered
crypto/tls 13 0xc02b (none)
uTLS Chrome 16 0x5a5a (GREASE) h2, http/1.1
Go was not even offering ALPN, so nothing could negotiate HTTP/2 in the first
place. stealthTransport dials https through uTLS with HelloChrome_Auto, then
speaks whatever ALPN settled on: h2 goes to x/net/http2 so the
SETTINGS/WINDOW_UPDATE fingerprint is a real h2 client's, rather than silently
downgrading to HTTP/1.1 and moving the inconsistency instead of removing it.
HelloChrome_Auto rather than a pinned profile, since a stale Chrome hello would
itself contradict the User-Agent read from the installed browser.
Dispatch happens per connection, not per host: ALPN is only known once the
handshake completes, and probing a host first to learn its protocol would double
the handshake count, working against the pacing -rps already does. The probe runs
with DisableKeepAlives, so a connection serves exactly one request anyway and the
per-request transports cost an allocation, not a round trip.
Two paths deliberately keep the standard transport: plain http, which has no
handshake to disguise, and -proxy, where the origin handshake happens inside a
CONNECT tunnel we do not drive. The TLS 1.2 MaxVersion pin on the proxy path is
dropped while here — capping the tunnel hid every TLS-1.3-only origin behind a
probe failure. MinVersion stays.
Detection over the 20-target reference set: 148 technologies against 150, no host
at zero, no probe errors — within the run-to-run band measured at 149/150/150.
gofmt -l . clean, go vet ./... clean, go test -race ./... green
Bringing the tree up to date, most usefully chromedp v0.8.5 (2022) to v0.16.0
and cdproto from August 2022 to July 2026. Those two pins were why a live scan
flooded stderr with "could not unmarshal event: unknown ClientNavigationReason
value" and "partitionKey" parse errors — Chrome 13x/150 sends CDP events the
2022 schema cannot decode. Over the 20-target reference set stderr goes from
noisy to zero lines, and neither update required a single code change.
chromedp v0.8.5 -> v0.16.0
cdproto 2022-08 -> 2026-07
goquery v1.8.1 -> v1.12.0
goccy/go-json v0.9.11 -> v0.10.6
cdncheck v1.0.2 -> v1.2.46
fastdialer v0.0.16 -> v0.5.13
retryabledns v1.0.24 -> v1.0.115
miekg/dns v1.1.62 -> v1.1.72
x/net v0.38.0 -> v0.57.0
github.com/imdario/mergo is deprecated in favour of dario.cat/mergo; migrated to
the canonical path at v1.0.2. Nothing covered that merge, so it was verified by
measurement instead: the loaded database is byte-for-byte the same shape before
and after — 7245 technologies, 7097 icons, 664 cpes.
The go directive moves to 1.26 because chromedp v0.16.0 declares it. That raises
the floor for anyone building this; CI follows automatically via go-version-file.
The update also silently killed the dns fingerprint family, which is why this
commit is not only a version bump. OVHcloud stopped being detected on a host that
is plainly hosted there, deterministically, 0 of 4 runs. Newer fastdialer only
retains address records:
GetDNSData("bzhunt.fr") -> A=[149.202.40.56] NS=[] SOA=0
All 80 dns patterns (43 SOA, 27 TXT, 18 NS, 5 CNAME, 4 MX) had gone quiet, and
Host.Cname with them. Reverting fastdialer would have papered over the real
problem: fastdialer is a dialer, and leaning on its connection cache for
detection data was a coupling that only ever worked by accident. cmd/dns.go now
asks a resolver directly through retryabledns.QueryMultiple — already a
dependency — naming the record types the fingerprints actually match.
That turned out to be a gain rather than a repair. The old cache only partly
populated those records, so explicit queries take the reference set from 150 to
170 technologies (+20): Amazon SES, Google Workspace, MailChimp, Apple iCloud
Mail and Atlassian Statuspage now fire for the first time, and the host that
exposed the regression goes from 6 detections to 9.
One API break in the whole set: retryabledns.DNSData.SOA changed from []string to
a parsed struct. soaStrings maps NS and Mbox, chosen after checking that all 43
SOA patterns in the live database target a hostname (\.cloudflare\.com,
\.domaincontrol\.com, \.vercel-dns\.com, tech\.sakura\.ad\.jp). The numeric
serial/refresh/TTL fields are left out rather than concatenated into noise a
pattern could accidentally hit, and a test fails if one matches the serial.
Also folds the resolver list into a single Cmd.resolvers() used by both the
dialer and the new resolver, instead of duplicating the default and writing back
through the option pointer.
Detection: 170 technologies over the reference set, no host at zero, stderr clean.
gofmt -l . clean, go vet ./... clean, go test -race ./... green, go mod verify OK
Two earlier commits each fixed part of the identity and left a seam between them. Probing www.hermes.com — DataDome in front of Cloudflare — surfaced both. setBrowserHeaders still emitted a hardcoded Sec-CH-UA in the Chrome-110 spelling: `"Google Chrome";v="N", "Chromium";v="N", "Not_A Brand";v="24"`, brands in the opposite order to what Chrome now sends, and the wrong bogus entry. So after "one coherent browser identity" the User-Agent finally matched between the probe and the browser while the Client Hints went on contradicting each other — the exact defect that work set out to remove, one header over. Both now derive from the same identity value, platform included, via identity.secChUa(). The second seam is the handshake. chromeHello was HelloChrome_Auto, an alias for whatever the uTLS build considers newest — 133 in v1.8.2 — while the presented User-Agent came from the installed browser, here 150. Measured against that origin, one round-trip per variant, two paced rounds: crypto/tls + UA 150 302 passed uTLS 133 + UA 150 403 blocked, captcha page <- what shipped uTLS 133 + UA 133 302 passed uTLS 131 + UA 150 302 passed uTLS 120 + UA 120 302 passed The only refused combination was the one being shipped: the vendor correlates the TLS fingerprint with the version the request claims. chromeHello is now pinned explicitly rather than tracking an alias a dependency bump can move, and identity.capToHello lowers the presented version to the one the handshake imitates, rewriting the UA string so the header, navigator.userAgent, navigator.userAgentData, Sec-CH-UA and the ClientHello all say 133. The trade-off, since it is a real one: the browser now reports a version older than its own engine, so a site comparing the User-Agent against JS features only present in the real build could notice. That check is rarer and more expensive than the TLS correlation, and claiming an older version is the safe direction — detectors flag claiming *newer* than the latest stable release. Capping only the probe was the alternative and was rejected: it puts the probe and the browser back in disagreement. Honest limit on the above: wappaGo itself was never observed being blocked on that host, before or after — 302 with 15 technologies and the real page title, Angular and TypeScript among them, so Chrome rendered the actual page rather than a challenge. The 403 reproduced 4/4 from an isolated probe carrying the same handshake and UA, and the http warm-up was ruled out as the difference, but the request volume of the investigation itself could not be excluded as a factor. Both fixes are coherence defects that stand on their own regardless. Drops (*Cmd).userAgent, now unused: probeIdentity supersedes it and returns the whole identity instead of one field, so a caller cannot pick up the User-Agent and forget the hints that go with it. Detection: 171 technologies over the 20-target reference set, no host at zero, stderr clean. gofmt -l . clean, go vet ./... clean, go test -race ./... green
Four remaining gaps, each verified by measurement before being acted on. Two of
the measurements contradicted claims this file made in earlier commits.
WebRTC was the serious one, and it is not fingerprinting but deanonymisation.
No WebRTC flag was passed at all, so a scanned page could open an
RTCPeerConnection to a public STUN server and read the host's real public address
— over UDP, which does not traverse an HTTP proxy, so straight past -proxy. The
LAN address was never exposed; Chrome already hides it behind an mDNS name.
The obvious remedy does not work. Measured on Chrome 150, reproduced twice:
(none) host=1 srflx=1
--force-webrtc-ip-handling-policy=disable_non_proxied_udp host=1 srflx=1
--force-webrtc-ip-handling-policy=default_public_interface_only host=1 srflx=1
--webrtc-ip-handling-policy=default_public_interface_only host=0 srflx=1
--webrtc-ip-handling-policy=disable_non_proxied_udp host=0 srflx=0
Only the older flag name closes it, and it leaves RTCPeerConnection defined, so
the leak goes without a "browser with no WebRTC" tell taking its place. Applied
when -proxy is set, which is when hiding the address is the point.
Headless is no longer the default. It costs an 800x600 screen whatever the window
size — a 1400x900 viewport claiming to sit on a smaller display — and it brands
its own User-Agent. With a real display both are simply correct
(screen=1920x1200, clean UA) and no metrics override is issued at all, one fewer
emulated surface. -headless remains for hosts without a display, and a failed
launch now points at xvfb-run rather than failing with a bare error.
Runtime.enable is closed, and the reason it was deferred was wrong. The previous
commit claimed fixing it "imposes patching chromedp (fork or replace)". It does
not: chromedp calls runtime.Enable() once at tab attach, and the execution-context
IDs it derives are read only by its selector-based actions. wappaGo uses none of
them — the complete list is Evaluate, Navigate, Title, CaptureScreenshot, Sleep,
ActionFunc, ListenTarget, and Title is itself an EvaluateAsDevTools. Since enable
only controls events while Runtime.evaluate is a command that works regardless,
runtime.Disable() at the head of the action chain closes the leak and leaves all
3155 js patterns running in the main world, which they require. One line, no fork.
window.outerWidth/outerHeight stay 0, and the explanation given for it twice was
wrong both times — first "inherent to headless", then "caused by
setDeviceMetricsOverride". Measured through chromedp, every combination reports 0:
headless or display, override or not, with or without Browser.setWindowBounds.
Raw Chrome opening a URL does report the window size, so it comes from the tab
chromedp attaches to via Target.createTarget, which has no window. Nothing over
CDP changes it; the injected-script workaround would leave a getter that no longer
reports [native code], a broader tell than the one it fixes. Documented as the
residual it is, with the right reason this time.
On software rendering, the earlier reasoning was incomplete. "Do not randomise,
because randomisation is itself a signal" is true but insufficient: a stable value
that is a *known* value identifies just as well, and the graphics backend does
move both fingerprints —
real GPU canvas2D d0c568f8 webgl 5796a780 35 exts
SwiftShader canvas2D f4d87128 webgl b30b83b0 36 exts
--disable-gpu canvas2D 693631e8 webgl b30b83b0 34 exts
so canvas 2D is not GPU-independent as assumed. There is no CDP override for
WebGL, so masking it means patching JS. Measured against the hosted detector,
forcing SwiftShader changed no verdict — the same greens as a real GPU. That is a
negative result from one detector, not proof of safety, so nothing is patched and
the limitation is documented instead.
The harness gains a WebRTC probe that reports its ICE candidates back over HTTP,
since gathering is asynchronous and chromedp.Evaluate would only ever catch
"pending". It also records the no-proxy case, where the mitigation is deliberately
off: 0 candidates with -proxy against 2 (1 server-reflexive) without, which proves
the probe can actually observe a leak rather than passing vacuously.
Detection: 179 technologies over the 20-target reference set, no host at zero,
stderr clean. 60 tests.
gofmt -l . clean, go vet ./... clean, go test -race ./... green
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.