Search results

Per-role decoding config is loaded but never applied (dead config)

Status: Open.

#189 In mini-router/minirouter;

· real-venus opened on Jul 13, 2026

reward: BFCL multi-call grading misaligns candidate/gold calls, scoring correct answers 0

Status: Open.

#180 In mini-router/minirouter;

· philluiz2323 opened on Jul 13, 2026

reward: RLPR WebInstruct math grader ignores \boxed{...} answers, scoring correct answers 0

Status: Open.

#170 In mini-router/minirouter;

· philluiz2323 opened on Jul 13, 2026

worker: one-shot (non --loop) run exits 1 on success, so supervisors see every processed job as a failure

Status: Open.

#168 In mini-router/minirouter;

· RealDiligent opened on Jul 13, 2026

validate_bundle trusts summary.json n_total over the canonical ParamSpec, so a wrong-size theta passes

Status: Open.

#167 In mini-router/minirouter;

· philluiz2323 opened on Jul 13, 2026

cost ledger: model ids with a quote or backslash write malformed JSON and are dropped from the cost total

Status: Open.

#165 In mini-router/minirouter;

· RealDiligent opened on Jul 13, 2026

reward: normalize_math_answer leaves a trailing backslash on LaTeX-escaped set braces \{...\}

Status: Open.

#162 In mini-router/minirouter;

· philluiz2323 opened on Jul 13, 2026

CI red on main: test_wrong_theta_length_fails not updated for the warning-only size check (#150)

Status: Open.

#160 In mini-router/minirouter;

· philluiz2323 opened on Jul 13, 2026

benchmarks: rlpr / ifeval / bfcl_simple load() all raise TypeError (same bug as #86, masked by the same test pattern)

Status: Open.

#152 In mini-router/minirouter;

· philluiz2323 opened on Jul 13, 2026

Infra/bugfix PRs auto-closed by validator before maintainer can merge

Status: Open.

#143 In mini-router/minirouter;

· RealDiligent opened on Jul 13, 2026

Multiple-choice grading picks the first listed option, not the model's answer

Status: Open.

#124 In mini-router/minirouter;

· rust-toml opened on Jul 12, 2026

RLPR MMLU-Pro is scored as 4-option (A–D), so any correct answer in E–J is always marked wrong

Status: Open.

#122 In mini-router/minirouter;

· rust-toml opened on Jul 12, 2026