Skip to content

Validate quality measurements and exercise attention oracles on CPU - #65

Draft
CompleteDotTech wants to merge 1 commit into
mainfrom
fix/reviewed-quality-and-cpu-oracles-20261026
Draft

CompleteDotTech wants to merge 1 commit into
mainfrom
fix/reviewed-quality-and-cpu-oracles-20261026

Conversation

@CompleteDotTech

Copy link
Copy Markdown
Collaborator

Imported evaluation rows could report accuracy above one or omit negative confidence from reliability bins. Validate finite, nonnegative measured fields and bound correctness/confidence before aggregation and bootstrap sampling. Preserve soft targets and optional ordinal error. Also run nine existing strict attention-oracle cases on CPU while retaining their CUDA cases and unchanged tolerances.

Related: #5 and #3. These changes improve reporting integrity and regression coverage; they do not establish representative quality, native-weight/CUDA acceptance or complete either issue.

Validation on this exact tree:

  • Combined Python suite: 908 passed, 55 explicit skips; one upstream deprecation warning.
  • Independent full-diff reviews for both contributions: no blocking findings; independent malformed-report probes passed.
  • Compilation, critical Ruff, staged-source integrity, feature matrix, 24-configuration CPU FP32 numerical audit, local resume/calibration exercise, TypeScript tests, wheel/sdist and installed-wheel HTTP checks passed.
  • CPU-compatible dependency installation, not a locked installation. Hosted CI is not claimed as passed.

Signed commit tree: 5527dcc3e757af9a10d5dcd1e2c293f97cdcd9b3.

Remain draft/unmerged pending required checks and policy verification. The integration denied the live protection read with HTTP403; that operation was not retried or bypassed. A pre-existing extreme-finite-value overflow limitation remains in standalone summarize(); the seed-report path rejects the nonfinite aggregate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant