framework: high-confidence remediation suggestions (Phase 2b.4)#36
Conversation
When the framework knows what's wrong with high confidence — exactly
two drivers on a short, a known-canonical forbidden state, a
declared-partner-mismatch on a mate — it now offers the most-likely
fix as a `Try: …` line at the end of the diagnostic. When confidence
is low, the method returns `None` and the framework stays silent
rather than fabricating a suggestion.
Design discipline (the teacher posture):
- Suggest only when one remediation is the right answer for >90% of
cases. Three-driver shorts, all-IN floats, unknown-prefix refdes
errors, novel forbidden states → `None`.
- Remediations are teaching-toned — explain what to do and offer the
alternative with "OR" when more than one fix is valid.
- No suggestion ever proposes bypassing a check, silencing the
validation, or using a bare ValueError/TypeError instead. Pinned
with a parametrised banned-phrases test.
Mechanism:
- `WirebenchError` gains `suggested_remediation() -> str | None`
(default `None`); leaf classes override with confidence-gated
per-shape logic.
- Per-class structured kwargs carry the diagnosable shape: drivers
for `ShortCircuitError`, kind+port_refs for `FloatingNetError`,
partner/expected/actual classes for `IncompatibleMateError`,
state_signature for `ForbiddenStateError`, port_types for
`SignalTypeMismatchError`, kind+duplicate_refdes for `RefdesError`,
port_domains for `DomainCrossingError`, port_refs for
`UnconnectedPinError`.
- Raise sites in wire/circuit/mate/port and `nor_latch` pass the
structured data so the suggestion has something to key off.
- `__str__` renders `Try: …` as a fourth paragraph after
base / Why / Wired at — append-only, existing regex extractors
still match the head line.
- `wirebench validate` JSON output gains a `remediation` field in
`details` when applicable; omitted entirely when the framework
returned `None`.
The shape end-to-end:
ShortCircuitError: wire() has multiple drivers ('y_1', 'y_2') — short circuit
Why: Two OUT-direction ports on one net fight each other on the
copper — current sinks through the losing output stage until the
FETs overheat; one driver per shared conductor.
Wired at: hello_led.py:14
Try: Remove one of the two wire() calls connecting y_1 and y_2,
OR insert a series element (resistor, diode) between them to
break the direct conflict.
Suite: 4761 passed (31 new), mypy clean.
There was a problem hiding this comment.
Pull request overview
Adds “high-confidence” remediation suggestions to framework errors and surfaces them in CLI JSON output, so diagnostics can end with a Try: … hint when the framework can infer a dominant fix and stay silent otherwise.
Changes:
- Introduces
WirebenchError.suggested_remediation()with per-exception overrides and updates__str__to append aTry:paragraph when applicable. - Passes structured diagnostic context (drivers, kinds, port refs/types/domains, etc.) through raise sites to enable confidence-gated remediation.
- Extends
wirebench validateJSONdetailswith an optionalremediationfield and adds comprehensive tests for positive/negative suggestion shapes and discipline constraints.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/framework/errors.py |
Adds suggested_remediation() API, stores structured fields on exceptions, and appends Try: to rendered error strings. |
src/framework/wire.py |
Populates structured context when raising wiring/signal mismatch errors from wire(). |
src/framework/port.py |
Populates structured context when raising domain/type mismatch errors from port operations. |
src/framework/mate.py |
Populates structured context for incompatible mating errors. |
src/framework/circuit.py |
Populates structured context for validate-time shorts/floats/unconnected pins/refdes collisions. |
src/components/chips/concepts/nor_latch.py |
Tags SR latch forbidden state with a state_signature and port names. |
src/cli/validate.py |
Adds helper to merge extractor details with optional remediation and omits the key when None. |
tests/framework/test_remediation_suggestions.py |
New unit tests covering high/low confidence shapes, rendering, banned phrases, and tone discipline. |
tests/cli/test_validate.py |
Updates CLI schema assertions and tests presence/absence of remediation in JSON details. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def suggested_remediation(self) -> str | None: | ||
| if self.port_types: | ||
| return ( | ||
| "Insert a comparator, ADC, or level-shifter between the " | ||
| "Analog and Digital ports — they can't share copper " | ||
| "directly because one carries a continuous voltage and " | ||
| "the other a logic level." | ||
| ) | ||
| return None |
…Digital shape
Copilot flagged that `SignalTypeMismatchError` fires from two distinct
paths:
1. `wire()` joining ports with mismatched signal_types — the
Analog↔Digital case the existing remediation addresses.
2. `Port.drive()` failing to coerce a runtime value (e.g. a string
onto a numeric port) — the conversion-element advice doesn't
fit; the real fix is to supply a correctly-typed value, which
depends on design intent the framework can't infer.
Firing the "Insert a comparator / ADC / level-shifter" advice for
path 2 was misleading. Gate the remediation on the canonical shape:
both `'Analog'` and `'Digital'` must appear among `port_types`.
Otherwise (e.g. Port.drive() failures with `('<incoming>', 'str')`,
or two-Analog Volts-vs-Amps mismatches) the method returns `None`
and the framework stays silent — the *teacher posture* the discipline
calls for.
Regression test added: non-Analog↔Digital `port_types` (including
the Port.drive() failure shape) must return `None`.
Suite: 4762 passed (1 new), mypy clean.
|
Good catch — fixed in 06312dc.
Gated the remediation on the canonical shape: both Regression test (
Both return Suite: 4762 passed (1 new), mypy clean. |
Summary
Second task of .plans/phase-2b-spec.md — when the framework knows what's wrong with high confidence (e.g. exactly two drivers on a short; a partner-class mismatch on a mate), the diagnostic ends with a
Try: …line offering the most-likely fix. When confidence is low (three-way shorts, novel forbidden states, unknown refdes prefixes), the method returnsNoneand the framework stays silent rather than fabricating a guess.The shape end-to-end:
Design discipline
The teacher posture, explicitly pinned by tests:
ORwhen more than one fix is valid.ValueError/TypeErrorinstead — banned phrases scanned by a parametrised test (bypass,silence,disable,except ValueError, etc.).Mechanism
WirebenchError.suggested_remediation() -> str | None(defaultNone); leaf classes override with confidence-gated per-shape logic.driversforShortCircuitError,kind+port_refsforFloatingNetError,partner/expected/actualforIncompatibleMateError,state_signatureforForbiddenStateError,port_typesforSignalTypeMismatchError,kind+duplicate_refdesforRefdesError,port_domainsforDomainCrossingError,port_refsforUnconnectedPinError.wire/circuit/mate/portandnor_latchpass the structured data so the suggestion has something to key off.__str__rendersTry: …as a fourth paragraph after base / Why / Wired at — append-only; existing regex extractors still hit the head line.wirebench validateJSON output gains aremediationfield indetailswhen applicable; omitted entirely when the framework returnedNone.Files
src/framework/errors.py— basesuggested_remediation()+ 8 per-class overrides + structured-field constructors.src/framework/wire.py,src/framework/circuit.py,src/framework/mate.py,src/framework/port.py— raise sites pass structured kwargs.src/components/chips/concepts/nor_latch.py— SR latch carriesstate_signature='sr_latch_both_active'.src/cli/validate.py— new_details_with_remediationhelper merges the structured-fields extractor output with the high-confidence hint.tests/framework/test_remediation_suggestions.py— 31 new tests: 8 positive shapes, 9 low-confidence-returns-None shapes, 2 rendering/ordering, 9 banned-phrase discipline checks, 1 teaching-tone shape.tests/cli/test_validate.py— assertremediationpresent in JSON for two-driver short; absent for three-driver short.Test plan
uv run pytest— 4761 passed (31 new), 16 skippeduv run mypy src/ demos/— clean