Gold context contains /workspace-absolute and mixed-relative-base paths that don't resolve against the repo tree

[jbendercb]

Two path forms appear in gold_context that cannot resolve against the repository at the task's own base_commit. Both look like collection artifacts, and both put entries in the File-coverage denominator that no system can match.

1. Container-absolute paths

1,509 gold paths across the release begin with /workspace (one with /tmp). gold.py:17–20 strips /workspace/ and the following path component, which usually recovers a real path — /workspace/django/db/models/enums.py → django/db/models/enums.py. Two forms defeat it:

• Bare filenames — nothing follows the prefix but the file itself, so the strip leaves nothing resolvable. 48 (task, path) pairs across 42 tasks:

  /workspace/reproduce.cpp     (Multi-SWE-Bench__cpp__maintenance__bugfix__6a5b64d, and others)
  /workspace/reproduce.sh      (Multi-SWE-Bench__c__maintenance__bugfix__5adf8efc)
  

  These look like files a solver created while working in /workspace. Checked specifically: none is named in the task's patch or test_patch, so they are not materialised by applying either.

• Non-repo second components — where the eaten component is not the repo directory (/workspace/temp/…, /workspace/output/…), the blind two-component strip yields a plausible-but-wrong repo-relative path rather than an obviously broken one. 13 paths. These are the harder cases to notice, since the result looks legitimate.

2. Mixed relative bases inside a single task

SWE-Bench-Verified__python__maintenance__bugfix__9db89f:

django/contrib/contenttypes/fields.py            ← resolves
django/db/models/enums.py                        ← resolves
django/db/models/fields/related_descriptors.py   ← resolves
django/template/base.py                          ← resolves
django/tests/template_tests/test_callables.py    ← django has no django/tests/ (tests live at tests/)

Four paths are repo-relative; the fifth carries a spurious repo-name prefix. Similar package-prefix duplication appears in transformers and requests tasks.

Scale — our measurement, method stated so you can check it

Fetching each task's tree at its own base_commit via the GitHub API (1,133 of 1,136 have non-empty gold) and testing every gold path, we count 88 tasks (7.8%) with at least one unresolvable path, and a dataset-wide mean File-coverage ceiling of 0.9798 (worst case 0.3333). Distribution: cpp 16.9%, ts 16.0%, js 14.5%, go 12.5%, java 10.5%, rust 9.7%, c 2.9%, python 2.0%.

The count depends on which normalisations are applied, and #1 shows normalisation here is subtle — so please treat the figure as ours rather than authoritative. The mechanism and the individual examples are what I would stand behind. Happy to share the per-task list.

Note: the ceiling binds File and Line only. Span and Def denominators come from byte_spans(repo_dir), and line_to_byte returns None on OSError, so unreachable files silently drop out of those denominators rather than capping them.