Skip to content

feat(debate): optional early-stop on convergence (closes #4)#38

Merged
ernestprovo23 merged 1 commit into
mainfrom
feat/debate-convergence
Jun 9, 2026
Merged

feat(debate): optional early-stop on convergence (closes #4)#38
ernestprovo23 merged 1 commit into
mainfrom
feat/debate-convergence

Conversation

@ernestprovo23

Copy link
Copy Markdown
Member

Implements PDD §9 #2 / issue #4: optional debate early-stop when answers converge, with a configurable convergence signal. Fixed-rounds behavior is preserved exactly when the feature is off (the default).

Convergence signal

Round-over-round answer stability. For each member with a usable answer in both the previous and current round, compute the difflib.SequenceMatcher ratio between its two answer texts (1.0 = identical), then average across those members. High mean = members stopped revising = the debate has stabilized.

Why this signal:

  • Dependency-free, deterministic, stdlib-only (difflib) → fully offline-testable, no heavy deps.
  • Self-stability faithfully captures "a debate converging" (members stop changing) and does not penalize a legitimate, stable disagreement between members the way cross-member agreement would.
  • Round 1 has no prior, so convergence is only evaluated after round ≥2 — early-stop can never fire before at least 2 rounds run.

Opt-in (mirrors the cache feature)

  • config: ConclaveConfig.converge_threshold: float | None (off/None by default; malformed/out-of-range values degrade to off with a warning).
  • library: Council.debate / Council.debate_sync / run_debate converge_threshold param — None defers to config (mirrors cache's None-defers-to-config).
  • CLI: --converge-threshold FLOAT (implies on) plus --converge/--no-converge (mirrors --cache/--no-cache). --converge with no explicit threshold uses a conservative default of 0.95; --no-converge forces off regardless of config.

Recorded on the result (minimal, mirrors cached)

  • CouncilResult.converged: bool and CouncilResult.convergence_score: float | None. Actual rounds run is already observable via len(result.rounds).

Cache key

converge_threshold is part of the debate cache key (make_key / Council._cache_key / _cached_run plumbing) so a converged run and a fixed run over otherwise-identical inputs never collide. Covered by a direct make_key unit test and an end-to-end no-collision test.

Hard constraints honored

  • call_model never-raises invariant and BYO-keys posture untouched (no key read/store/serialize; redact() path unchanged).
  • Convergence-detector failure or degenerate input (empty/missing answers) never crashes — it degrades to running the remaining fixed rounds.
  • Convergence logic lives in modes.py, not council.py.

Tests

Offline, transport-mocked via the existing patch_call_model fixture. 142 → 151 (9 new):

  • convergence OFF → exactly --rounds (unchanged behavior)
  • ON + converging answers → early-stop, records score + actual rounds
  • ON + never-converges → full --rounds, no false stop
  • degenerate input (all-fail) → no crash, fixed-rounds fallback
  • partial-degenerate (one member fails mid-debate) → scores survivors, no crash
  • converge_threshold=None defers to config
  • _round_convergence scorer unit test
  • cache-key: converged vs fixed produce different keys (unit + e2e two-file check)

Local: 151 passed, ruff check . clean, ruff format --check . clean.

🤖 Generated with Claude Code

Add an opt-in debate early-stop when answers converge, preserving the
historic fixed-rounds behavior exactly when disabled (default).

Signal: round-over-round answer stability -- per-member
difflib.SequenceMatcher ratio averaged across members usable in both the
previous and current round. Deterministic, stdlib-only, offline-testable.
A degenerate score (no comparable answers) or any scoring failure degrades
to running the remaining fixed rounds; it never crashes the debate.

Opt-in is consistent with the cache feature:
- config: ConclaveConfig.converge_threshold (float | None, off by default)
- library: Council.debate/debate_sync + run_debate converge_threshold param
  (None defers to config, mirroring cache's None-defers-to-config)
- CLI: --converge-threshold FLOAT plus --converge/--no-converge

Recorded on CouncilResult.converged + convergence_score (minimal, mirrors
how the cache feature added .cached); actual rounds run is len(rounds).

converge_threshold is added to the debate cache key (make_key / _cache_key /
_cached_run plumbing) so a converged run and a fixed run over identical
inputs never collide.

Convergence logic lives in modes.py (not council.py). 9 new offline,
transport-mocked tests cover: off=full rounds, on+converge=early stop with
recorded score, on+never-converge=full rounds, degenerate input fallback,
config-defer, the scorer unit, and a cache-key no-collision check
(142 -> 151 tests). ruff check + format clean.
@ernestprovo23 ernestprovo23 merged commit 3f14600 into main Jun 9, 2026
5 checks passed
@ernestprovo23 ernestprovo23 deleted the feat/debate-convergence branch June 10, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant