A moved payload shape is not an empty listing; CI tests 3.14

[jehrr]

Follows up the 2026-09-30 audit: its point 1 (the endpoints are undocumented BAPI) and the CI half of point 3. What the audit described as a risk turned out to be a live gap. I renamed fields in a real capture, and both ways a shape can move went through silently:

moved in a real P2P capture	before	after
row container data → items, page 1	exit 4, "the listing has nothing in it". The same response still counted 187 adverts	exit 5, nothing written; the log says the page was served but none of its rows were read
the same, on page 2 of copy-trading	exit 0, complete	exit 6, partial, stop_reason: parser_found_nothing, pages_failed: [2]
adv.price → unitPrice	exit 0, every row priceless, one warning in the log	still exit 0 (the rows are written), and the sidecar carries core_field_shortfall: {"1": {"price": 0.0}}; the canary fails on it

How the rule was set

A page counts as unread when its own total puts rows on it and no row was parsed. That relies on the endpoints serving every page their total implies. I measured this live on 2026-09-30 with plain requests:

endpoint	total → pages	last page	the page after
P2P USDT/EUR	228 → 12	8 rows	0 rows
copy-trading 30D	9,093 → 304	3 rows	0 rows
announcements	2,275 → 46	25 rows	0 rows

So the real end of a listing is still end_of_listing. The total is read from the page being judged, so a listing that shrank during the run carries its new total.

One case it cannot catch: if the total is renamed too, the page looks like an empty listing. The canary's daily row floor is what catches that.

A test whose premise could not happen

check_a_multi_page_run_merges_in_page_order_and_ends_on_data modelled "the listing shrank" as the page-999 capture served as page 3, with its total of 8,921 unchanged. That means page 3 claims rows and has none, which the live endpoints never answer. It now states a shrunk total. The dedupe check next to it had the same premise. It passed before for the right reason and would have kept passing afterwards for the wrong one (partial rather than complete), so it now also asserts the run is complete.

CI

The matrix is now 3.9 + 3.14 (was 3.9 + 3.12). I did not act on the audit's suggestion to drop 3.9: it is the Python macOS still ships, and it is the one the audit itself ran on. The reason is written next to the matrix.

Evidence

• Suite: 361 checks pass in a core-only venv, 366 with Playwright installed.
• Controls, run on a copy with no bytecode cache. Each went red on the named check and none crashed:
  • unread never set: red 4 times, and the page-2 case read complete / exit 0, which is the old behaviour;
  • shortfall not carried to the sidecar: red 1 time;
  • page-range guard dropped: red 5 times, including the end-of-listing test.
• Live, Playwright, 3 pages each, all complete / exit 0 with core_field_shortfall: {}:
  • p2p: 60 rows of 231;
  • copytrading: 90 of 9,092;
  • announcements: 150 of 2,275.

Not changed

• No new exit code: a moved shape falls into the existing 5 or 6 through the rule that a run which did not complete is never an empty result.
• A renamed column stays exit 0. The listing was read and the rows are real; the sidecar says what is missing.
• The rest of the audit is out of scope here: lock files, webhooks and checkpoints, an HTTP-only engine (a separate PR).

🤖 Generated with Claude Code