You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 786d689
Browse filesBrowse the repository at this point in the historyBrowse files
Port every test in upstream's tests/registry.md to tests/async and
tests/sync, named after its id and title, or list it in
tests/upstream_not_ported.txt with the reason.
scripts/check_test_registry.py checks this in CI. The older tests move
to tests/{async,sync}/legacy.
Fix the client bugs that the ports found, so that the same tests pass
as with Node.js, and roll the driver to 1.65.0-alpha-2026-10-17.
CI runs the tests in parallel with pytest-xdist, on fewer bots.
test.sh runs the library tests under xvfb.
Copy file name to clipboardExpand all lines: .claude/skills/playwright-roll/SKILL.md
+11-5Lines changed: 11 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -282,11 +282,17 @@ After the script settles, run `pre-commit run --all-files` once more to confirm
282
282
283
283
### 6. Add tests
284
284
285
-
For each PORT, port the upstream test that covers it, as an async test and a matching sync test. Conventions:
285
+
Copy the upstream registry of the new release, then bring the Python tests in line with it:
286
286
287
-
- The upstream file decides the Python file: `tests/page/expect-misc.spec.ts` becomes `tests/async/test_expect_misc.py` and `tests/sync/test_expect_misc.py`. Create the file if it does not exist.
288
-
- The upstream title decides the test name: the `describe` title and the test title, in snake_case. `test('should support pseudo element')` in `test.describe('toHaveCSS')` becomes `test_to_have_css_should_support_pseudo_element`.
289
-
- When no upstream test covers the change, extend an existing test before you add a new one.
-**New identifiers**: port each new upstream test, as an async test and a matching sync test. `python scripts/check_test_registry.py --missing <code or file glob>` prints the file and the expected name of each. The upstream file decides the Python file (`tests/page/expect-misc.spec.ts` becomes `tests/async/page/test_expect_misc.py` and `tests/sync/page/test_expect_misc.py`); the identifier and the title decide the name (see [Tests](../../../AGENTS.md#tests)).
293
+
-**Changed titles**: the checker names the expected name; rename the Python test and port the changed body.
294
+
-**Removed identifiers**: delete the Python test (the checker reports it as not in the registry).
295
+
-**Not ported**: a test that can not exist in Python goes in `tests/upstream_not_ported.txt`, with the reason. The checker fails on a rule that no longer matches a test.
290
296
- Use `from playwright.async_api import …`, **not**`from playwright._impl._page import Page` (the impl class doesn't have the public wrappers like `expect_console_message`).
291
297
- For event-info objects, `await message_info.value` (it's an `async` property).
292
298
- Don't write tests that hang the page (e.g. `page.evaluate(... fetch slow ...)` followed by `page.close()` from a fixture) — the request task gets a `TargetClosedError`. Use `page.on("event", handler)` to capture state at event time instead.
@@ -308,7 +314,7 @@ mypy playwright # 2 pre-existing errors in _json_pipe.py and _ar
Then a smoke regression on a few neighboring suites (`tests/async/test_browser*.py`, `test_cdp_session.py`, `test_tracing.py`, `test_dialog.py`, `test_page_*.py`) to make sure nothing inherent to the port shifted.
317
+
Then run `python scripts/check_test_registry.py`, and a smoke regression on a few neighboring suites (for example `tests/async/library/test_browser*.py`, `tests/async/library/chromium/test_session.py`, `tests/async/library/test_tracing.py`, `tests/async/page/test_page_*.py`) to make sure nothing inherent to the port shifted.
Copy file name to clipboardExpand all lines: AGENTS.md
+9-7Lines changed: 9 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ Python bindings for [Playwright](https://playwright.dev). The Python client talk
17
17
-`_protocol_types.py` has the TypedDicts for initializers, results, events and nested types, named as upstream's `utils/generate_channels.js` names them (`PageInitializer`, `FrameGotoResult`, `PageRouteEvent`, …).
18
18
-`_protocol.py` has the parameters of every command. With `PW_CHECK_PROTOCOL=1`, `playwright/_impl/_protocol_check.py` checks each message that the client sends against it. The test suite sets this, and fails a test that sends an unknown command, an unknown or missing parameter, or a value of the wrong type.
19
19
-`scripts/codegen/compare_api.py` — compares two generated files (signatures, annotations, call bodies, docstrings) and lists every difference. Use it to check that a generator change does not change the API.
20
-
-`tests/async/`, `tests/sync/` — pytest suites, ported from the upstream tests. Most new tests are added to the async file with a sync mirror. See [Tests](#tests).
20
+
-`tests/async/`, `tests/sync/` — pytest suites, ported test for test from the upstream tests, which `tests/upstream_registry.md` lists by identifier. See [Tests](#tests).
21
21
-`DRIVER_VERSION` — the single source of truth for which Playwright release the driver is assembled from (one line, the `playwright-core` npm version, e.g. `1.61.0`, no `v` prefix). Read by `setup.py`, `scripts/build_driver.py`, and CI. The wheel build downloads `playwright-core` at this version from npm plus the matching Node.js binary and assembles the per-platform bundles — no source build. The version is baked into the staged bundle filenames (`driver/playwright-<version>-<suffix>.zip`), so it doubles as the build cache key.
22
22
-`NODE_VERSION` — the Node.js version bundled with the driver (one line, e.g. `24.16.0`). Maintained at roll time by `scripts/update_node_version.py` (latest LTS, mirroring upstream's `utils/build/update-playwright-node.mjs`).
23
23
-`scripts/build_driver.py` — assembles the per-platform driver bundles into `driver/` by downloading the `playwright-core` npm package (`DRIVER_VERSION`) and the official Node.js binaries (`NODE_VERSION`). Fetches `playwright-core` with `npm pack` (needs Node.js/npm on PATH; honours a root `.npmrc`) and the Node.js binaries over plain HTTP; invoked from `setup.py`'s `bdist_wheel` with the target platform's suffix (no arg builds all six).
@@ -50,14 +50,16 @@ The public API comes from the upstream docs. After `./scripts/update_api.sh`, ru
50
50
51
51
## Tests
52
52
53
-
The tests are a port of the upstream tests in `microsoft/playwright` (`tests/page/`, `tests/library/`). Keep them close to upstream, so that each upstream test has one place in this repo.
53
+
The tests are a port of the upstream tests in `microsoft/playwright` (`tests/page/`, `tests/library/`), test for test. Upstream gives each test an identifier, the first token of its title: `it('CH-001 should check the box', ...)`. `tests/upstream_registry.md` is a copy of upstream's `tests/registry.md`, which lists every identifier with its file and title.
54
54
55
-
**First rule: follow the upstream names and files.**
55
+
**First rule: every upstream test is ported, or listed as not ported.**
56
56
57
-
- When a change needs a test, find the upstream test for the same behavior and port it. Do not write a new test when an upstream test exists.
58
-
- Put the test in the Python file that matches the upstream file: `tests/page/expect-misc.spec.ts` becomes `tests/async/test_expect_misc.py`. If that file does not exist, create it.
59
-
- Name the test after the upstream title, in snake_case, with the `describe` title in front: `test('should support pseudo element')` in `test.describe('toHaveCSS')` becomes `test_to_have_css_should_support_pseudo_element`.
60
-
- Add the sync mirror in `tests/sync/`, with the same file name and test name.
57
+
- Port each upstream test twice, to `tests/async/` and to `tests/sync/`, in the file that matches the upstream file: `tests/page/page-check.spec.ts` becomes `tests/async/page/test_page_check.py` and `tests/sync/page/test_page_check.py`; `tests/library/chromium/oopif.spec.ts` becomes `tests/async/library/chromium/test_oopif.py`.
58
+
- Name the test `test_<ID>_<title>`: the identifier with `_`, then the `describe` titles and the title in snake_case. `test('EX-181 toHaveCount pass')` in `test.describe('toHaveCount')` becomes `test_EX_181_to_have_count_to_have_count_pass`. A test that a loop declares several times is one function with `@pytest.mark.parametrize`.
59
+
- Port faithfully: the same steps, assertions and skip conditions. A test that fails because of a bug in the Python client stays, marked `@pytest.mark.xfail(reason="python: ...", strict=True)`.
60
+
- A test that can not exist in Python (a JavaScript-only API, Node.js client internals, tools that run in the driver, `@playwright/test` features) goes in `tests/upstream_not_ported.txt`, with the reason.
61
+
-`python scripts/check_test_registry.py` checks all of this. `--missing <code or file glob>` prints the expected names of the tests still to port; `--report` prints the coverage per subsystem.
62
+
- Do not write a new test when an upstream test exists. `tests/async/legacy/` and `tests/sync/legacy/` hold older ports named after old upstream titles; do not add tests there.
61
63
62
64
**Second rule: keep the number of tests small.** This applies to every test, and most of all to Python-only behavior that has no upstream test (the sync API, asyncio, the driver).
0 commit comments