Skip to content

test: Deflake test_error_handler_can_access_page on Windows CI#2018

Open
vdusek wants to merge 3 commits into
masterfrom
test/deflake-error-handler-can-access-page
Open

test: Deflake test_error_handler_can_access_page on Windows CI#2018
vdusek wants to merge 3 commits into
masterfrom
test/deflake-error-handler-can-access-page

Conversation

@vdusek

@vdusek vdusek commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

test_error_handler_can_access_page fails intermittently on the Windows CI runner (example run). Under xdist load, Playwright's Page.goto occasionally fails with net::ERR_NO_BUFFER_SPACE before the page is navigated. That attempt's error handler then receives a BasicCrawlingContext (recorded as None) instead of a PlaywrightCrawlingContext, so the exact-list assertion [HELLO_WORLD, HELLO_WORLD] breaks with [None, HELLO_WORLD].

The crawler behaves correctly here (it retries, and a non-navigated attempt legitimately yields a basic context), so this is a test flake, not a product bug. The same net::ERR_NO_BUFFER_SPACE failure is already documented for three sibling tests (test_resource_management, test_adaptive_crawling_pre_nav_change_to_context, test_stagehand_browser_controller), which handle it with @pytest.mark.flaky(reruns=3).

Rather than retry, this makes the assertion robust to spurious pre-navigation failures: it ignores attempts that never reached the page (None) and asserts that at least one attempt did, and that every attempt that did exposes the page HTML. Removes the test's sensitivity to the flake deterministically while preserving what it verifies (that the error handler can read the page). No product code changes.

@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Jul 3, 2026
@vdusek vdusek self-assigned this Jul 3, 2026
@vdusek vdusek marked this pull request as ready for review July 3, 2026 10:41
@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.34%. Comparing base (b850cd6) to head (1513919).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2018      +/-   ##
==========================================
- Coverage   93.36%   93.34%   -0.02%     
==========================================
  Files         179      179              
  Lines       12488    12488              
==========================================
- Hits        11659    11657       -2     
- Misses        829      831       +2     
Flag Coverage Δ
unit 93.34% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek requested a review from Pijukatel July 3, 2026 10:49
Comment thread tests/unit/crawlers/_playwright/test_playwright_crawler.py Outdated
@vdusek vdusek requested a review from Pijukatel July 3, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants