Repository navigation
refactor(grading): rename older-problem support to pre-existing problem - #138
Merged
Merged
Conversation
The owner decided on 2026-10-13 to call a fault that was in a project before the pull request under review a pre-existing problem. No grade, admitted problem or saved verdict holds the earlier field names yet, so the fields, decision types, audit strata, candidate kind, module, doc and panel take the new name now. The three proposal files under bench/rubric keep their names and bytes: earlier blind sessions read them and the proposed policy pins them. Part of #135.
…at it earned credit A pre-existing problem is not a point system: a review raises one, and the scorecard shows the count without scoring it. The panel is "Pre-existing problems raised", the doc is docs/pre-existing-problems.md and the card's counts are named raised. The decision type is pre-existing-raise, because pre-existing-raised is already an audit stratum. Credit rulings in credits.json and the two wording facts they share with known problems keep their names. Part of #135.
…r the current decision types The admission round on the preparation branch for #135 opened at 32d5258 and holds before-records written as older-admission and older-grouping. A before-record is written once and pinned by hash, so it cannot be rewritten to the new names. read_before maps the three earlier spellings to the current ones for a record that pins that round's opening, and for no other record. Part of #135.
…earlier decision types SavedRecordsTest read each saved before-record as written, so it would refuse older-admission and older-grouping once the round opened before the rename lands on main, although read_before supports them. Part of #135.
… stand The three proposal files were read before the rename and tell graders to write older_problems, older_candidate and kind older-problem, which validation now refuses. Activation needs new versions written with the current field names. Part of #135.
kamui
commented
Oct 14, 2026
kamui
left a comment
Owner
Author
There was a problem hiding this comment.
Approved (advisory) — no findings.
Reviewed 6f00a74. All 40 changed files reviewed, including schema/producer/consumer alignment, saved-ruling readers, frozen validator dependencies and explorer displays. No actionable findings.
Review details
Intent: Rename dormant older-problem support to pre-existing problems raised, while preserving grading and scoring behavior.
Issue fit: Fits the terminology-only scope of #135; activation, new policy versions and grading remain separate work.
Reviewed: 6f00a74 against merge-base 32d5258.
Checks:
- Exact-head CI passes: 988 grading tests (18 skipped), five self-tests, 68 web tests, 27 export/evidence tests, client compatibility, current/claim/import validation, build and typecheck. The native-model-instructions test also passes in CI.
- Locally: 117 focused Python tests pass across the initial run and affected ruling-test rerun after restoring hash-pinned dossier responses; 37 scoring tests and typecheck pass. The live fixture panel correctly shows raised, zero and unavailable results.
- Compatibility checks cover all three legacy aliases, exact-pin refusal, unchanged saved bytes, v1/v2/v3 mapping and ordinary score isolation. The external 22-record round and the claimed all-198-fingerprint comparison were not independently reproduced; their absence does not leave the rename behavior untested.
- Other PR-reported runs were treated as supporting assertions unless corroborated above; roster, scorecard, proposal-example and external-round output was not supplied. Published as a non-gating comment under the authenticated author identity.
5 of 11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
"Older problem" was ambiguous, and "credit" suggested a point system. The concept is now a pre-existing problem: a fault that was in the project before the change and that the change did not cause. A review raises one; that is its own kind of finding, shown and not scored. The support merged dormant in #137, and no grade or saved verdict holds its fields yet, so this is the cheapest moment to rename words and identifiers together.
This is a rename only. No rule, condition or default changes.
Scope
bench/tools/older_grading.py→pre_existing_grading.py(and its test);docs/older-problem-credit.md→docs/pre-existing-problems.md, with its four links.pre_existing_problems,pre_existing_candidate,pre_existing_problem_id,pre_existing_candidate_id; fingerprint inputpre_existing_admissions; candidate kindpre-existing-problem; dimensionpre-existing-admission; decision typespre-existing-admission,pre-existing-raise,pre-existing-grouping.pre-existing-raised,pre-existing-not-raised,pre-existing-unresolved, unitpre-existing-review, unit id infix#pre-existing:.src/lib/scoring.ts,src/lib/data.ts,MethodologyViews.tsx,EvidenceDrawer.tsx:preExistingProblems, card fieldsraisedin place ofcreditsandcatches, panel "Pre-existing problems raised".ruling_record.read_beforereads the records of the one round opened before the rename (older-admission,older-credit,older-grouping) under the current names, keyed on that round's opening pin.SavedRecordsTestreads through it.Not touched:
docs/research/**,bench/grading/rulings/**,bench/runs/**, saved assessments and audit records, the text in force (*.next.md), and the three proposal filesbench/rubric/*.older-problems.v1.md, whose bytes are pinned as what earlier blind sessions read.docs/pre-existing-problems.mdnow says they cannot be pinned as they stand.Tradeoffs
credits.json,credit_subject,creditFact).docs/finding-threshold.mdand the id prefixOP-stay: the first names an earlier scope ruling, the second is used as a group name by written records.Blast Radius
Dormant support only: nothing on main holds a renamed field, all 198 batch fingerprints match main, and current records and schema examples validate unchanged. The three proposal files now disagree with the validator's field names, so activation must write new grader, rubric and rule files, which it does anyway.
Verification
bun run test: bench suite 988 tests, 1 failure, 18 skipped. The failure istest_codex_grading.test_catalog_preserves_native_model_instructions, the known host-only failure. The rest of the chain run separately: five self-tests pass,test:clientpasses,test:web68 and 27 pass.bun run typecheck(afterbun run build),bun run verify:current,bun run verify:claims,bun run verify:import,python3 bench/tools/roster.py,bun run scorecard: pass. The scorecard prints "Pre-existing problems raised: unavailable (No approved pre-existing problems in this selection.)".ruling_record.pyandroutethrough the alias, in a scratch copy outside the repository; the saved-records test was red on them before the alias reader and green after.Part of #135