Skip to content

Phase 2C.1: add reproducible teacher image quality benchmark - #29

Draft
xueyang-dev wants to merge 3 commits into
mainfrom
codex/phase-2c1-teaching-image-benchmark
Draft

Phase 2C.1: add reproducible teacher image quality benchmark#29
xueyang-dev wants to merge 3 commits into
mainfrom
codex/phase-2c1-teaching-image-benchmark

Conversation

@xueyang-dev

Copy link
Copy Markdown
Owner

Summary

Phase 2C.1 adds a reproducible, teacher-reviewable quality benchmark for the fixed Phase 2C SD 1.5 teaching-image path.

  • Defines 25 fixed-seed cases across all ten required teaching categories.
  • Adds strict schema and identity binding for the fixed Model Package, Workflow Pack, Runtime, and prompt profile.
  • Adds an interruptible, resumable, idempotent batch executor that uses only the controlled TeachingImageRequest entry point.
  • Verifies PNG technical integrity, SHA-256, provenance, request identity, fixed negative prompt, and Asset Manifest registration.
  • Adds a portable offline teacher review package with blank pending_review records and structured failure labels; no teacher scores are generated automatically.
  • Documents pilot and full real-run evidence without committing images, Runtime, model, cache, or reports.

Real-run evidence

  • Pilot: 3/3 real technical successes after using a persistent Runtime owner session; 3 review records pending.
  • Full run: 25/25 real technical successes, 0 technical failures, 0 pending/invalidated, one fixed seed per case.
  • Asset Manifest: 25 entries, all pending_review.
  • Teacher review: 25 pending records, 0 imported/reviewed; no teaching-quality conclusions are claimed.
  • An initial one-off Runtime starter produced a fail-closed ComfyUI BrokenPipeError; this was recorded and recovered by keeping the Runtime owner session alive. History/provenance checks were not weakened.

Validation

  • npm test: 654 API tests passed, 3 skipped; Vite build passed; Web state contract passed.
  • npm run test:e2e: 14 passed.
  • git diff --check: passed.
  • Focused benchmark/Ruff checks passed.
  • macOS arm64 opt-in Runtime/model lifecycle and real generation were exercised locally.

Scope boundaries

This PR does not tune prompts, replace the model, modify the fixed Workflow Pack, add LoRA/ControlNet/custom nodes/multi-model support, fabricate teacher ratings, add cloud fallback, or merge automatically. It remains Draft pending real teacher review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant