Skip to content

test: fix order-dependent and stale tests surfaced by the full suite run - #3681

Merged
vybe merged 1 commit into
devfrom
fix/test-suite-order-pollution
Oct 13, 2026
Merged

vybe merged 1 commit into
devfrom
fix/test-suite-order-pollution

Conversation

@trinity-ability

Copy link
Copy Markdown
Contributor

Summary

A full tests/run-full.sh run on a local stack failed several tests that pass when run alone. Every one of them is a test-side problem: order dependence (pytest-randomly shuffles the order when it is installed), state that leaks between tests, or an assertion that went stale after an intended product change. This PR fixes them in the tests only. No product code changes.

The two real product bugs from the same run were filed separately as #3676 and #3677.

Integration / API tier

Test Cause Fix
test_nevermined_permissions.py::test_get_config_no_config, test_nevermined_payments.py::test_info_no_config, test_shared_folders.py::test_folders_default_values, test_read_only_mode.py::test_get_read_only_status_default_is_disabled Each asserts "fresh agent" defaults on the module-scoped created_agent, which sibling tests change. Under a shuffled order the change can come first. Use the per-test isolated_agent fixture.
test_agent_rename.py::test_rename_agent_name_sanitization It always renamed to test-agent-with-spaces. Deleting the renamed agent soft-deletes its ownership row, and that row keeps reserving the name and its volume base by design (#1664/#1671). Every run after the first got 409. Derive the target from the fixture agent's unique suffix.
test_subscriptions.py::TestSubscriptionAutoAssign round-robin and tiebreak Since #2409, auto-assign ranks by cached headroom. A real operator subscription has a reading and the suite's fake tokens never do, so it always wins. Skip when operator-owned subscriptions exist, the same guard the no-subscription test already uses (#2420). The tiebreak subscriptions now carry the suite's test- prefix.
test_operator_queue.py::test_respond_to_pending_item Answering a platform-raised item lands acknowledged, not responded (#2372). The test picks whichever pending row comes first. Pin both shapes. acknowledged must also set acknowledged_at.

Unit tier

Each fix is in the test that leaked state, not in the test that failed.

Victim Polluter and mechanism Fix
test_ent814_served_seat.py test_brain_orb_postprocess.py: dispatch_postprocess lazily imports task_execution_service while database.db is patched. The unit conftest evicts that module after each test, so its fresh import bound the stub. An autouse fixture pre-imports the service before each patch.
test_2973_depth_new_roots.py test_monitoring_dormant_skip.py: a bare sys.modules["services.monitoring_alerts"] = stub that was never undone. monkeypatch.setitem.
test_2572_credentialless_adoption.py (apply-phase restarts) test_subscription_auto_switch_no_cred_import.py: the restore fixture fixed the sys.modules slot but left a stale module on the parent package attribute. The victim then patched the wrong copy of _restart_agent. Snapshot and restore the parent-package attribute too.
test_ec_loop_service_edges.py::test_m16_infinite_budget_never_trips[postgres] Not pollution. On PostgreSQL the cost columns are float4, so writing 1e300 raises NumericValueOutOfRange. That is the known bug #3441. Strict xfail on the Postgres variant only, citing #3441. It will flip when the bug is fixed.

Test plan

  • Each unit polluter/victim pair fails before the fix and passes after, run with -p no:randomly.
  • The touched unit files plus their victims pass under -p randomly with seeds 1, 7, 42, 12345, 987654 and 2026: 188 passed, 1 xfailed each time.
  • The seven touched API test files pass against a live local stack under -p randomly. Seed 1 gave 149 passed, 3 skipped. Seed 424242 had one transient ReadTimeout on the first run and gave 149 passed, 3 skipped on the re-run.
  • The [postgres] xfail was not executed locally, because it needs the disposable Postgres tier.

The 3 skips are the subscription precondition guard. They fire because the test stack has a real operator subscription.

🤖 Generated with Claude Code

Full run-full.sh on a local stack (pytest-randomly installed, so test order
is shuffled) failed several tests that pass in isolation. None of these are
product bugs; each fix is test-side.

Integration/API tier:
- Default-state tests (Nevermined no-config x2, shared-folders defaults,
  read-only default) asserted "fresh agent" state on the module-scoped
  created_agent that sibling tests mutate. They now use isolated_agent.
- test_rename_agent_name_sanitization renamed to a fixed name. A deleted
  agent's soft-deleted ownership row keeps that name and its volume base
  reserved by design (#1664/#1671), so every run after the first got 409.
  The target is now derived from the fixture agent's unique suffix.
- TestSubscriptionAutoAssign round-robin/tiebreak: since #2409 auto-assign
  ranks by cached headroom, so a real operator subscription always beats the
  suite's fake tokens. Skip on foreign subscriptions like the existing
  no-subscription test (#2420); the tiebreak subs now carry the suite prefix.
- test_respond_to_pending_item: answering a platform-raised item lands
  `acknowledged` (#2372); pin both shapes.

Unit tier (fixed at the polluter, not the victim):
- test_brain_orb_postprocess: a lazily imported task_execution_service bound
  the patched database.db stub; pre-import it before each patch.
- test_monitoring_dormant_skip: bare sys.modules assignment of a stub
  services.monitoring_alerts; use monkeypatch.setitem.
- test_subscription_auto_switch_no_cred_import: restore the parent-package
  attribute as well as the sys.modules slot.
- test_ec_loop_service_edges m16: the [postgres] variant fails on the known
  float4 REAL-column bug, not pollution; strict xfail citing #3441.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge-train: batch validated on train/20261013-2303 (#3685)

@vybe
vybe merged commit 75b3ab9 into dev Oct 13, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants