Skip to content

feat(perf): decide captured observed-performance policies offline - #26

Merged
plotarmordev merged 1 commit into
mainfrom
feat/observed-policy-24
Sep 10, 2026
Merged

feat(perf): decide captured observed-performance policies offline#26
plotarmordev merged 1 commit into
mainfrom
feat/observed-policy-24

Conversation

@plotarmordev

Copy link
Copy Markdown
Owner

Closes #24.

Delivered

  • Optional run --policy FILE validates and captures exact policy bytes before dispatch, with a skip-absent plan binding. Every baseline/candidate/repeat must have captured the same policy; no retrospective binding through decide.
  • Offline decide A B --reference A2 --json returns a versioned PASS / REGRESSION / INCONCLUSIVE / ERROR envelope, separate from measurement eligibility and conventional compare exits.
  • Policy explicitly pins collector/workload identities, every required cell/metric, minimum trials, practical loss tolerance and allowed reference spread. There are no implicit recipe tolerances.
  • Checked rational cross-products evaluate observed adverse bounds without comparing rounded percentages. Complete wave/lane coverage, qualified independent role evidence and reference variability precede a verdict. Missing data never becomes a favorable subset; a qualified regression cannot be averaged away.
  • Results bind verified outcome/session/evidence fingerprints and evaluator identity. Machine errors use bounded reason codes rather than raw paths/provider/deployment diagnostics. Legacy no-policy requests and receipts retain their byte contracts.

Claim boundary

PASS means complete observed evidence meets the declared practical policy. It does not establish statistical significance, causal effect, future performance, universal no-regression or model quality. Reference declarations and captured policy are not authenticated restoration or independent preregistration. Only a printed decision envelope constitutes a decision; process success from run/compare is not PASS.

Verification

Independent SIX, HELP and CYBERGLM review; one malformed-unbound-collector identity emission defect was reproduced, fixed and re-reviewed. HELP independently ran exact boundary/determinism/privacy regressions. Seventeen new real-CLI policy tests; one older comparison fixture now uses explicitly synthetic fixed observations instead of a scheduler-sensitive intersection premise. Exact-head fmt, Clippy -D warnings, full workspace tests and release build passed. Combined shared workflow was exercised on a clean-source native Linux ARM64 build using synthetic loopback responses.

No inference/server/SSH/deployment calls. Live recipe qualification remains separately coordinated, DeepSeek first and GLM independently later. No claim of actual source authenticity or absence of all historical tool defects.

@plotarmordev
plotarmordev merged commit 4922f99 into main Sep 10, 2026
2 checks passed
@plotarmordev
plotarmordev deleted the feat/observed-policy-24 branch September 10, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an explicit offline observed-performance decision policy

1 participant