feat(ax): auto-enable Chromium web accessibility tree - #3
Open
caipira113 wants to merge 2 commits into
Open
Conversation
`AXNode::app` decided whether to wait for web accessibility with `!has_web_area(..) && looks_chromium(..)`. Both are correct, but they cost very different amounts on a native app. `has_web_area` searches twelve levels for an `AXWebArea`. A native app has none, so the probe cannot short-circuit and walks the entire subtree before returning false — and Word materialises accessibility elements on demand, so with a large document open that walk is the whole cost of *every* command, including ones that never touch the document. `looks_chromium` reads `AXDOMClassList` at most four levels down and returns at the first match, and a native app has no such classes at all. Swapping the operands means a native app is rejected by the cheap test and never runs the expensive one. The conjunction is unchanged, so Chromium apps still wait exactly as before. Measured with a 74k-character document open in Word: snapshot --depth 1 15.61s -> 1.10s No regression elsewhere: Teams still resolves its `webarea` (0.28s) and Finder is unchanged (0.08s).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Chromium/Electron apps keep their web-content accessibility tree switched off until an assistive client asks for it. Without that opt-in, walking such an app bottoms out at the native window chrome, so
snapshotreturns only the title bar and menu items and every locator against web content fails.Before this change, on Microsoft Teams:
This affects Teams, Discord, Slack, VS Code, and Chrome-family browsers, which is a large share of what people actually want to automate.
Change
AXNode::app()is the single place every command resolves an application through, so the opt-in goes there.Enable. Set
AXManualAccessibility(the Electron opt-in) andAXEnhancedUserInterface(the legacy flag some builds expose instead). Both are set unconditionally and theirAXErrorresults are ignored: Chromium acts on the attempt while still returningkAXErrorAttributeUnsupported(-25205) orkAXErrorNotImplemented(-25208), so the return code says nothing about whether the tree will appear. Native Cocoa apps reject both and are unaffected.Settle. The renderer publishes the tree asynchronously over IPC, so a walk starting immediately still sees only the chrome. Poll for an
AXWebAreadescendant, up to 1.5s.Gate the wait. Only wait when the app looks Chromium-backed, detected via
AXDOMClassListentries on the native chrome (RootView,NonClientView,ClientView). These appear as soon as the window exists, well before web content, which distinguishes "web content is still loading" from "there is no web content" and keeps native apps from paying the timeout.Verification
Tested on macOS 27.0 (build 26A5388g, arm64). For each run the app was reset to a clean state (
AXManualAccessibility = false) so that a single fresh invocation is what gets measured, matching real CLI usage where each command is a new process.webarea+ full DOM, 0.645swebarea+#app-mount, 0.103sLocators resolve against web content on the first run:
cargo testpasses (54 tests).cargo clippyreports no new warnings for the changed lines.Notes
activateis needed first. This is orthogonal to this change but easy to mistake for it while testing.