Skip to content

framework: high-confidence remediation suggestions (Phase 2b.4)#36

Merged
raeq merged 2 commits into
mainfrom
phase-2b-4-remediation
May 20, 2026
Merged

framework: high-confidence remediation suggestions (Phase 2b.4)#36
raeq merged 2 commits into
mainfrom
phase-2b-4-remediation

Conversation

@raeq

@raeq raeq commented May 20, 2026

Copy link
Copy Markdown
Owner

Summary

Second task of .plans/phase-2b-spec.md — when the framework knows what's wrong with high confidence (e.g. exactly two drivers on a short; a partner-class mismatch on a mate), the diagnostic ends with a Try: … line offering the most-likely fix. When confidence is low (three-way shorts, novel forbidden states, unknown refdes prefixes), the method returns None and the framework stays silent rather than fabricating a guess.

The shape end-to-end:

ShortCircuitError: wire() has multiple drivers ('y_1', 'y_2') — short circuit
  Why: Two OUT-direction ports on one net fight each other on the
  copper — current sinks through the losing output stage until the
  FETs overheat; one driver per shared conductor.
  Wired at: hello_led.py:14
  Try: Remove one of the two wire() calls connecting y_1 and y_2,
  OR insert a series element (resistor, diode) between them to
  break the direct conflict.

Design discipline

The teacher posture, explicitly pinned by tests:

  • Suggest only when one remediation is the right answer for >90% of cases.
  • Teaching-toned: explain what to do and offer the alternative with OR when more than one fix is valid.
  • Never propose bypassing a check, silencing the validation, or using bare ValueError / TypeError instead — banned phrases scanned by a parametrised test (bypass, silence, disable, except ValueError, etc.).

Mechanism

  • WirebenchError.suggested_remediation() -> str | None (default None); leaf classes override with confidence-gated per-shape logic.
  • Per-class structured kwargs carry the diagnosable shape: drivers for ShortCircuitError, kind+port_refs for FloatingNetError, partner/expected/actual for IncompatibleMateError, state_signature for ForbiddenStateError, port_types for SignalTypeMismatchError, kind+duplicate_refdes for RefdesError, port_domains for DomainCrossingError, port_refs for UnconnectedPinError.
  • Raise sites in wire/circuit/mate/port and nor_latch pass the structured data so the suggestion has something to key off.
  • __str__ renders Try: … as a fourth paragraph after base / Why / Wired at — append-only; existing regex extractors still hit the head line.
  • wirebench validate JSON output gains a remediation field in details when applicable; omitted entirely when the framework returned None.

Files

  • src/framework/errors.py — base suggested_remediation() + 8 per-class overrides + structured-field constructors.
  • src/framework/wire.py, src/framework/circuit.py, src/framework/mate.py, src/framework/port.py — raise sites pass structured kwargs.
  • src/components/chips/concepts/nor_latch.py — SR latch carries state_signature='sr_latch_both_active'.
  • src/cli/validate.py — new _details_with_remediation helper merges the structured-fields extractor output with the high-confidence hint.
  • tests/framework/test_remediation_suggestions.py — 31 new tests: 8 positive shapes, 9 low-confidence-returns-None shapes, 2 rendering/ordering, 9 banned-phrase discipline checks, 1 teaching-tone shape.
  • tests/cli/test_validate.py — assert remediation present in JSON for two-driver short; absent for three-driver short.

Test plan

  • uv run pytest — 4761 passed (31 new), 16 skipped
  • uv run mypy src/ demos/ — clean
  • Existing extractor / message-shape tests stay green (append-only design verified)
  • Manual smoke: two-driver short renders all four lines (base / Why / Wired at / Try)

When the framework knows what's wrong with high confidence — exactly
two drivers on a short, a known-canonical forbidden state, a
declared-partner-mismatch on a mate — it now offers the most-likely
fix as a `Try: …` line at the end of the diagnostic.  When confidence
is low, the method returns `None` and the framework stays silent
rather than fabricating a suggestion.

Design discipline (the teacher posture):

- Suggest only when one remediation is the right answer for >90% of
  cases.  Three-driver shorts, all-IN floats, unknown-prefix refdes
  errors, novel forbidden states → `None`.
- Remediations are teaching-toned — explain what to do and offer the
  alternative with "OR" when more than one fix is valid.
- No suggestion ever proposes bypassing a check, silencing the
  validation, or using a bare ValueError/TypeError instead.  Pinned
  with a parametrised banned-phrases test.

Mechanism:

- `WirebenchError` gains `suggested_remediation() -> str | None`
  (default `None`); leaf classes override with confidence-gated
  per-shape logic.
- Per-class structured kwargs carry the diagnosable shape: drivers
  for `ShortCircuitError`, kind+port_refs for `FloatingNetError`,
  partner/expected/actual classes for `IncompatibleMateError`,
  state_signature for `ForbiddenStateError`, port_types for
  `SignalTypeMismatchError`, kind+duplicate_refdes for `RefdesError`,
  port_domains for `DomainCrossingError`, port_refs for
  `UnconnectedPinError`.
- Raise sites in wire/circuit/mate/port and `nor_latch` pass the
  structured data so the suggestion has something to key off.
- `__str__` renders `Try: …` as a fourth paragraph after
  base / Why / Wired at — append-only, existing regex extractors
  still match the head line.
- `wirebench validate` JSON output gains a `remediation` field in
  `details` when applicable; omitted entirely when the framework
  returned `None`.

The shape end-to-end:

    ShortCircuitError: wire() has multiple drivers ('y_1', 'y_2') — short circuit
      Why: Two OUT-direction ports on one net fight each other on the
      copper — current sinks through the losing output stage until the
      FETs overheat; one driver per shared conductor.
      Wired at: hello_led.py:14
      Try: Remove one of the two wire() calls connecting y_1 and y_2,
      OR insert a series element (resistor, diode) between them to
      break the direct conflict.

Suite: 4761 passed (31 new), mypy clean.
Copilot AI review requested due to automatic review settings May 20, 2026 17:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds “high-confidence” remediation suggestions to framework errors and surfaces them in CLI JSON output, so diagnostics can end with a Try: … hint when the framework can infer a dominant fix and stay silent otherwise.

Changes:

  • Introduces WirebenchError.suggested_remediation() with per-exception overrides and updates __str__ to append a Try: paragraph when applicable.
  • Passes structured diagnostic context (drivers, kinds, port refs/types/domains, etc.) through raise sites to enable confidence-gated remediation.
  • Extends wirebench validate JSON details with an optional remediation field and adds comprehensive tests for positive/negative suggestion shapes and discipline constraints.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/framework/errors.py Adds suggested_remediation() API, stores structured fields on exceptions, and appends Try: to rendered error strings.
src/framework/wire.py Populates structured context when raising wiring/signal mismatch errors from wire().
src/framework/port.py Populates structured context when raising domain/type mismatch errors from port operations.
src/framework/mate.py Populates structured context for incompatible mating errors.
src/framework/circuit.py Populates structured context for validate-time shorts/floats/unconnected pins/refdes collisions.
src/components/chips/concepts/nor_latch.py Tags SR latch forbidden state with a state_signature and port names.
src/cli/validate.py Adds helper to merge extractor details with optional remediation and omits the key when None.
tests/framework/test_remediation_suggestions.py New unit tests covering high/low confidence shapes, rendering, banned phrases, and tone discipline.
tests/cli/test_validate.py Updates CLI schema assertions and tests presence/absence of remediation in JSON details.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/framework/errors.py
Comment on lines +288 to +296
def suggested_remediation(self) -> str | None:
if self.port_types:
return (
"Insert a comparator, ADC, or level-shifter between the "
"Analog and Digital ports — they can't share copper "
"directly because one carries a continuous voltage and "
"the other a logic level."
)
return None
…Digital shape

Copilot flagged that `SignalTypeMismatchError` fires from two distinct
paths:

  1. `wire()` joining ports with mismatched signal_types — the
     Analog↔Digital case the existing remediation addresses.
  2. `Port.drive()` failing to coerce a runtime value (e.g. a string
     onto a numeric port) — the conversion-element advice doesn't
     fit; the real fix is to supply a correctly-typed value, which
     depends on design intent the framework can't infer.

Firing the "Insert a comparator / ADC / level-shifter" advice for
path 2 was misleading.  Gate the remediation on the canonical shape:
both `'Analog'` and `'Digital'` must appear among `port_types`.
Otherwise (e.g. Port.drive() failures with `('<incoming>', 'str')`,
or two-Analog Volts-vs-Amps mismatches) the method returns `None`
and the framework stays silent — the *teacher posture* the discipline
calls for.

Regression test added: non-Analog↔Digital `port_types` (including
the Port.drive() failure shape) must return `None`.

Suite: 4762 passed (1 new), mypy clean.
@raeq

raeq commented May 20, 2026

Copy link
Copy Markdown
Owner Author

Good catch — fixed in 06312dc.

SignalTypeMismatchError does fire from two paths:

  1. wire() joining ports with mismatched signal types — the Analog↔Digital case the original remediation addresses.
  2. Port.drive() failing to coerce a runtime value to the port's signal type (e.g. ('<incoming>', 'str')) — where the fix is to supply a correctly-typed value, which is design-intent territory.

Gated the remediation on the canonical shape: both 'Analog' and 'Digital' must appear among port_types. Otherwise the method returns None and the framework stays silent — the teacher posture the discipline calls for.

Regression test (test_signal_type_mismatch_non_analog_digital_returns_none) pins two non-canonical shapes:

  • ('in', 'Digital') + ('<incoming>', 'str') — the Port.drive() failure case
  • ('a', 'Volts') + ('b', 'Amps') — two-Analog mismatch with no Digital party

Both return None now.

Suite: 4762 passed (1 new), mypy clean.

@raeq raeq merged commit dcead21 into main May 20, 2026
3 checks passed
@raeq raeq deleted the phase-2b-4-remediation branch May 20, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants