Skip to content

test: the object-storage loops name the case each iteration tests (#982, 2 of 8) - #989

Merged
jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:fix/982-the-objstore-loop-continuations
Sep 12, 2026
Merged

test: the object-storage loops name the case each iteration tests (#982, 2 of 8)#989
jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:fix/982-the-objstore-loop-continuations

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Second of #982's eight files, and the one that lost the most records. objstore_module: six checks sharing three keys, now zero.

Measured against a control run on clean main, same suite, same box:

clean main    30 records   24 distinct keys   3 colliding   6 lost
this branch   30 records   30 distinct keys   0 colliding   0 lost

Both rc=0, 30 checks, 0 FAIL. The record count is unchanged, which is how you can see this renames rather than adds or removes.

The two loops show both sub-shapes of the same defect

This file is a better pattern-setter than I expected, because its two loops fail differently.

Loop one — the headline already interpolated the scheme and only the continuation was short:

check     "a s3 URL reports an object-storage error, not a missing file"
check_num "and does NOT report it as a missing file"        # identical three times

Loop twoneither name carried the metacharacter, so three iterations produced one key for each of two checks. The headline collided as well, which loop one's did not.

So #984's rule covers both, with one extension: the continuation carries the same discriminator its headline namesand where the headline names none either, it gains one.

The discriminators were already in scope

From the loop variable, by parameter expansion — no new data, no invented labels:

scheme="${url%%:*}"                 # s3, gs, https
meta="${pat#s3://bucket/a}"; meta="${meta%.parquet}"   # *, ?, [0-9]

giving, from the run:

PASS  a remote glob (*) is handled remotely (an object-storage error, not a local one)
PASS  and the * glob is NOT reported as a local filesystem miss
PASS  a remote glob ([0-9]) is handled remotely (an object-storage error, not a local one)
PASS  and the https URL is NOT reported as a missing file

This is the mechanical quarter of #982 — six of the twenty-four. The remaining eighteen are hand-written sites where the discriminator has to be chosen rather than read off a loop variable.

One small thing, called out so it is not mistaken for scope creep

Loop one's headline now reads its scheme from $scheme rather than from a cut subshell, because both names need it. The resulting check name is byte-identical (a s3 URL reports... either way), so no ledger row moves on account of it and no name changed that did not have to.

No ledger change at all

objstore_module is not one of the two suites the ledger covers — those are harness_selftest and native_join_runtime_filter. So its check names have no rows, and this needs no regeneration.

That is also the sharp end of why #982 matters for #432 rather than for today's census: twenty-one of the twenty-four collisions are in suites that become covered only when the 240 are seeded, and they would go missing at that moment with every number still reconciling.

Gate

objstore_module   rc=0   30 checks   0 FAIL   on both trees
docs_style        rc=0   9 checks    0 FAIL
shellcheck        0 findings on the changed file
ledger + budget   untouched

A note on this PR's own commit message

It had to be written to a file rather than passed with -m: an apostrophe in the inline argument closed the quote and the shell then tried to execute the glob examples. Third inline-quoting fault of my session, against a rule that is mine — write the script, do not inline it. Recording it because the examples that broke it are the same metacharacters the suite is testing, which is a pleasing way to be reminded.

Part of #982. Six of twenty-four down; six more files.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

jdatcmd pushed a commit that referenced this pull request Sep 12, 2026
hilbert_cluster held the second-largest loss: one premise repeated before four
separate arms, and two pairs of (d) checks repeated across two fixtures.

    clean main    181 records  176 distinct keys  3 colliding  5 lost
    this branch   181 records  181 distinct keys  0 colliding  0 lost

Record count unchanged, so this renames and nothing else.

A THIRD FORM OF THE RULE, and the files keep supplying them. #984: a continuation
carries the discriminator its headline interpolates. #989: where the headline
names none either, it gains one. Here NOTHING interpolates anything -- all eight
sites are hand-written -- so the discriminator comes from the check's own VALUE
EXPRESSION, the table it queries:

    "$(pgc_is_columnar_scan 'SELECT * FROM s3hi')"   -> ... digested for s3hi ...
    "$(pgc_is_columnar_scan 'SELECT * FROM av_hi')"  -> ... digested for av_hi ...

That keeps the name and the assertion in agreement, which is worth more than
brevity: a reader can check one against the other without leaving the line. The
word doing the colliding was "here" -- which names the site to someone reading
top to bottom and names nothing at all to a key.

The two (d) pairs take the table their own count(*) and physlayout name, for the
same reason: moved s4d1's layout, no row was lost from s4d2.

No ledger change: hilbert_cluster is not one of the two suites the ledger covers,
which is why 21 of the 24 collisions matter for #432's seeding and not for the
census today.

    hilbert_cluster 181 checks 0 FAIL on both trees | docs_style 9/9
    shellcheck 0 findings | ledger and budget untouched

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
…mmandprompt#982)

objstore_module lost the most records of any suite to key collapsing: two loops
of three iterations, where the names did not carry what the iteration varies.

    clean main    30 records  24 distinct keys  3 colliding  6 lost
    this branch   30 records  30 distinct keys  0 colliding  0 lost

Record count unchanged, so this adds and removes no checks.

THE TWO LOOPS SHOW BOTH SUB-SHAPES. In the first the headline already
interpolated the scheme and only the continuation was short. In the second
NEITHER name carried the metacharacter, so three iterations produced one key for
each of TWO checks -- the headline collided too, which the first loop did not.

Both are fixed by the rule commandprompt#984 proposed: the continuation carries the same
discriminator its headline names, and where the headline names none either, it
gains one. The discriminators come from the loop variable by parameter expansion,
so they were already in scope:

    a remote glob [*] is handled remotely, an object-storage error not a local one
    and the [*] glob is NOT reported as a local filesystem miss
    and the https URL is NOT reported as a missing file

The first loop's headline now reads its scheme from a variable rather than a cut
subshell because both names need it; the resulting check name is byte-identical,
so no ledger row moves on account of it.

No ledger change: objstore_module is not one of the two suites the ledger covers,
which is why 21 of the 24 collisions matter for commandprompt#432's seeding rather than for
today's census.

    objstore_module 30 checks 0 FAIL on both trees | docs_style 9/9
    shellcheck clean | ledger and budget untouched

The commit message for this change had to be written to a file rather than passed
inline: an apostrophe in a -m argument closed the quote and the shell then tried
to execute the glob examples above. Third inline-quoting fault of the session and
the rule is mine: write the script, do not inline it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
@OffgridwithJD
OffgridwithJD force-pushed the fix/982-the-objstore-loop-continuations branch from db657b7 to 0f6e1ef Compare September 12, 2026 01:46

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 0f6e1ef7. Verified with the same derived-helper method as #992 (write-up there, including two under-reporting failures of my own).

objstore_module.sh   names 23 -> 23
                     literal 22 -> 19, interpolated 1 -> 4

Three names move from literal to interpolated — the loop fix exactly: the continuation now carries the scheme its headline already had in scope, and the variable was already there. Name count unchanged, so this renames and neither adds nor drops an assertion.

Worth recording that my static sweep reported 0 literal duplicates on main for this file, against your measured 3 colliding. Both true: the continuation is one source line inside a three-iteration loop. A static count under-reports loops exactly as it over-reports interpolation — which is the third distinct way I have had that method be wrong today, and the reason #983's guard must count over emitted records.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants