Skip to content

Tests mutate tracked fixture files (scenarios/test_pipelines/*.yaml) on every run #431

Description

@jeremymanning

Found while verifying an unrelated change.

Evidence

Running the suite rewrites tracked files in place:

 M tests/scenarios/test_pipelines/concurrent_network.yaml
 M tests/scenarios/test_pipelines/network_recovery.yaml

The diff is an ephemeral port number being substituted into the fixture:

-            url: "http://localhost:58202/test1"
+            url: "http://localhost:63696/test1"

Impact

  • git status is dirty after any test run, so real changes are easy to lose in the noise — this actually caused churn to be swept into unrelated commits during this session.
  • The committed port number is meaningless: it records whatever ephemeral port one machine happened to get.
  • Any CI job that checks for a clean tree will fail spuriously.

Proposed fix

The test should render the fixture to a temp path (tmp_path) with the port substituted, and leave the tracked file untouched. The tracked fixture should carry a placeholder (e.g. {{ port }} or a sentinel) rather than a real port number.

Acceptance criteria

  • git status --porcelain is empty after a full test run
  • The tracked fixtures contain no machine-specific port numbers

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions