Skip to content

fix(reward): extract boxed answer in RLPR WebInstruct math grading (#170)#61

Open
galuis116 wants to merge 1 commit into
mini-router:mainfrom
galuis116:sn74-galuis116-agreement-diagnostic
Open

fix(reward): extract boxed answer in RLPR WebInstruct math grading (#170)#61
galuis116 wants to merge 1 commit into
mini-router:mainfrom
galuis116:sn74-galuis116-agreement-diagnostic

Conversation

@galuis116

@galuis116 galuis116 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Closes #170

Summary

_check_rlpr_webinstruct in src/trinity/orchestration/reward.py grades WebInstruct-verified RLPR items. For a math answer it fed the raw candidate text into math_equal / normalize_math_answer without first extracting the answer — unlike _check_math, which uses extract_boxed / extract_last_number. Since normalize_math_answer does not strip \boxed{...}, a correct boxed answer (the format format_hint("rlpr") tells the worker to use) scored 0.

score_text("rlpr", r"The area is \boxed{15}.", {"ground_truth":"15","source":"WebInstruct-verified-val_Avg2"})
# -> 0.0 (should be 1.0); the same answer under math500 -> 1.0

What Changed

  • src/trinity/orchestration/reward.py: extract the answer before the math compare — cand_math = extract_boxed(cand) or extract_last_number(cand) or cand, gold_math = extract_boxed(gold) or gold — mirroring _check_math.
  • tests/test_rlpr_webinstruct_math.py (new, offline): boxed answer scores 1.0, plain number still works, wrong boxed scores 0, the letter path is unchanged, and RLPR agrees with the math path on a boxed answer.
  • docs/JOURNAL.md: entry per repository protocol.

Why This Matters

Every WebInstruct-verified RLPR math item answered in the requested \boxed{...} format was a false negative, understating accuracy and corrupting the reward signal for those items. Verified offline: the new tests fail on the old code and pass on the fix; 47 existing rlpr/reward tests still pass.

Notes

@github-actions github-actions Bot added docs Documentation changes eval Evaluation changes miner Miner contribution PR train Training or routing changes labels Jul 8, 2026
@tmimmanuel tmimmanuel added the feature New feature label Jul 9, 2026
@galuis116 galuis116 closed this Jul 11, 2026
@galuis116 galuis116 reopened this Jul 11, 2026
@galuis116
galuis116 force-pushed the sn74-galuis116-agreement-diagnostic branch from a25eacc to 6698b45 Compare July 12, 2026 04:00
@tmimmanuel tmimmanuel added the not planned This feature is not planned. label Jul 12, 2026
_check_rlpr_webinstruct fed the raw candidate text into math_equal without
extracting the answer, unlike _check_math. Since normalize_math_answer doesn't
strip \boxed{...}, a correct boxed answer (the format the worker is told to use)
scored 0: score_text('rlpr', r'The area is \boxed{15}.', {...'15'...}) returned
0.0 while the same answer under math500 returned 1.0.

Extract extract_boxed(cand) or extract_last_number(cand) or cand (and
extract_boxed(gold) or gold) before comparing, mirroring _check_math. Adds
offline tests/test_rlpr_webinstruct_math.py; 47 existing rlpr/reward tests pass.
Distinct from the RLPR MMLU-Pro choice issues (mini-router#116/mini-router#122). Logs in docs/JOURNAL.md.
@galuis116
galuis116 force-pushed the sn74-galuis116-agreement-diagnostic branch from 6698b45 to 29ea75b Compare July 13, 2026 07:22
@galuis116 galuis116 changed the title feat(analysis): per-question model-agreement diagnostic + contested-subset export (#60) fix(reward): extract boxed answer in RLPR WebInstruct math grading (#170) Jul 13, 2026
@galuis116

Copy link
Copy Markdown
Contributor Author

Note: I've repurposed this PR onto a distinct, self-contained fix — RLPR WebInstruct math grading now extracts \boxed{} answers (Closes #170). The force-push above swaps the contents; the agreement-diagnostic work can be reopened separately if wanted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation changes eval Evaluation changes feature New feature miner Miner contribution PR not planned This feature is not planned. train Training or routing changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reward: RLPR WebInstruct math grader ignores \boxed{...} answers, scoring correct answers 0

2 participants