feat(types,hir,mir,codegen): Optional[int] flow-sensitive narrowing (D-199, Part 2 of #747)

[rotnov]

Outstanding gate before merge (D-068): the pinned local reviewer
(ievo:deep-review) has not been run against this branch. Skill(skill: "ievo:deep-review") refuses model invocation
(disable-model-invocation: true; "reserved for explicit user invocation")
and this dispatched session has no Task/Agent-style tool to route
around that block — the identical capability gap already logged for
#763/PR #770 (docs/AGENT_RETROSPECTIVE.md, "2026-08-24 — A dispatched
subagent cannot satisfy D-068's local-reviewer dispatch requirement"). A
session that can invoke /ievo:deep-review needs to run it against the
full committed range and resolve any actionable findings before this
merges.

Summary

Implements flow-sensitive Optional[int] narrowing on a top-level
if name is None: / if name is not None: test — Part 2 of #747,
following on from #763/PR #770 (Part 1). Fixes #769.

• pycc_hir::optional_none_test / pycc_hir::definitely_terminates: a
  shared, environment-independent narrowing-test recognizer and a strict
  terminator predicate, consumed directly by pycc_mir (which cannot
  depend on pycc_types) and re-exported thinly by pycc_types::narrow.
• pycc_types: overlay-based narrowing state on Environment (clone/
  discard join semantics), applied both to if bodies and to the
  early-return-narrows-the-continuation shape (a guard clause whose body
  definitely terminates narrows every read after it).
• pycc_mir: a $narrowed:{name} scope-sentinel on the existing scopes
  stack, plus narrowing_snapshot/restore_narrowing/lower_scoped_body
  to recreate the checker's clone-and-discard semantics on MIR's single
  shared scope frame. Deviation from the original dispatched design,
  made and flagged per this task's own "resolve autonomously, flag the
  deviation" instruction: without this, narrowing established inside a
  nested body (e.g. a while loop inside a narrowed if) could leak past
  that body's own close on MIR's shared-frame model, which the checker's
  own clone-and-discard Environment never allows. Covered directly by
  a_nested_scoped_body_entered_while_already_narrowed_still_sees_the_narrowing
  in crates/pycc_mir/src/tests/narrow.rs.
• MirExpr::OptionalUnwrap (read-side counterpart of OptionalWrap),
  lowered to a single borrowed build_extract_value in codegen — no
  retain at the unwrap site itself. bigint_rc.rs's existing
  retain_if_int_duplicate/int_value_is_a_duplicate_reference
  duplicate-reference classification gained an OptionalUnwrap arm so a
  bigint payload duplicated out of a narrowed binding into a second name
  is still correctly refcounted.
• docs/decisions/D-199-optional-t-flow-sensitive-narrowing-part2.md,
  docs/TYPE_SYSTEM.md, docs/ROADMAP.md, and
  tests/fixtures/conformance-breadth-manifest.json updated (the
  manifest's narrowing row flips to proven, its existing ◐ subset
  marker left unpromoted).

Deliberately out of scope (documented in D-199/TYPE_SYSTEM.md/the
manifest): compound conditions (and/or), narrowing to None itself,
raise as an additional terminator alongside return, and any test more
complex than a top-level is/is not None comparison.

D-014 coverage gate

cargo llvm-cov --workspace --fail-under-lines 100 --fail-under-regions 100
initially failed at 1 missed region
(crates/pycc_codegen/src/bigint_rc.rs:281, the new OptionalUnwrap arm).
Root cause: Cargo compiles pycc_codegen under several distinct metadata
hashes depending on which package links it, and the one uncovered
instantiation (shared by issue_382_exceptions, slice1_codegen_depth,
and the pycc binary) is only reached when a bigint-valued narrowed
Optional[int] read is duplicated into a second binding — the fixture
extension alone didn't close it locally because that scenario in
pep_0604_union.py only reaches this instantiation through
tests/conformance.rs's CPython-oracle comparison, which is skipped
without a pinned python3.14 on PATH. Fixed with
tests/issue_769_optional_narrowing.rs, an oracle-independent
build-and-run integration test (bigint duplicate-binding, smallint
mirror, absent-optional cases), plus the same scenario added to
pep_0604_union.py for when the oracle is available. Final coverage: 0
missed lines/regions/functions across 42836 regions.

Testing

• cargo test -p pycc_hir --lib: 648 passed, 0 failed.
• cargo llvm-cov --workspace --fail-under-lines 100 --fail-under-regions 100:
  exit 0, 0 missed lines/regions/functions.
• cargo test --workspace: 60 test result: ok blocks, 0 failures.
• ruby scripts/check_roadmap_evidence.rb: passed.
• python3 scripts/generate_decisions_index.py docs/decisions docs/decisions/README.md --check: up to date.

See docs/sessions/2026-08-25-02-issue-769-optional-narrowing-part2.md
for the full session record.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[rotnov]

D-068 review response (98e8c41, b7d0340)

Addressed the pinned ievo:deep-reviewer pass on this branch. Summary below; does not merge.

Finding 1 (blocker, soundness) — fixed

join_if_branches/join_loop_body/join_match_branches (crates/pycc_types/src/lib.rs) and lower_scoped_body's join call sites (crates/pycc_mir/src/lib.rs) discarded a narrowing kill caused by a reassignment inside a nested branch, so a stale Optional narrowing could survive past that branch's own close — e.g.

def f(x: int | None, flag: bool) -> None:
    if x is not None:
        if flag:
            x = None
        print(x + 1)   # unsound: x could be None here

was previously accepted. Both crates now reconcile sibling-branch narrowing via a sound conservative intersection: a name stays narrowed only if every supplied branch narrows it to the exact same type. pycc_types gains narrow::join_narrowed; pycc_mir gains an independent join_narrowed twin (kept separate rather than shared, matching the existing pycc_hir-only-shares-what-both-need precedent, since pycc_mir doesn't depend on pycc_types). lower_scoped_body now returns each nested body's own ending narrowed state so If/While/ForRange/ForList in stmt.rs can join branches and re-apply the reconciled result. match/try call sites are unchanged deliberately: MIR only lowers HIR the now-sound checker already accepted, and the checker's fixed join rule for match/try is a provable subset of "unconditionally keep pre-branch state," so no soundness gap remains there.

Design tradeoff: intersection is not maximally precise (it doesn't special-case an unconditionally-terminating branch the way the ordinary .bindings join does), but it can only shrink or preserve the narrowing set relative to the old always-keep-pre-branch behavior — never grow it — so it cannot introduce new unsoundness. Chosen over a more precise but more intricate join because correctness was the review's actual bar.

Regression tests: pycc_mir::tests::narrow::a_reassignment_inside_a_nested_if_kills_narrowing_past_the_nested_ifs_own_close and a_reassignment_inside_a_nested_while_kills_narrowing_past_the_loops_own_close.

End-to-end verification beyond unit tests: pycc check on the exact repro above now correctly emits error[T0021]: operator Add is not defined for 'int | None' and 'int'.

Finding 2 (warning, completeness) — fixed

The checker's if/while fast-path helpers (check_if_branches_in_place, check_while_body_in_place, and their _in_function twins) bypassed narrowing-aware statement processing, spuriously rejecting a nested early-return guard inside an unrelated outer if, e.g.:

def f(cond: bool, x: int | None) -> int:
    if cond:
        if x is None:
            return 0
        return x
    return 0

All four now route through narrow::check_stmt_sequence/check_stmt_sequence_in_function, matching the slow/clone path's existing behavior.

Regression tests: 7 new checker-level tests in crates/pycc_types/src/tests.rs covering both findings plus join-reconciliation edge cases (mixed-type sibling narrowing, partial narrowing across branches).

End-to-end verification: pycc run on the shape above now builds and runs, printing 5, 0, 0 for f(True, 5), f(True, None), f(False, None).

Finding 3 (doc drift) — no change needed

Checked tests/fixtures/conformance-breadth-manifest.json and D-199's Consequences section against the fixed behavior; both are still accurate now that findings 1 & 2 are resolved.

Incidental fixes (discovered while verifying this branch would actually pass CI)

• Pre-existing clippy::ptr_arg CI failure in apply_post_if_narrowing/restore_narrowing (crates/pycc_mir/src/lib.rs), confirmed via live CI logs to already exist on the pushed branch before this review pass (traces to 14535c9, part of the original Optional[int] is None / is not None flow-sensitive narrowing (Part 2 of #747) #769 feature work), blocking build-test-coverage, native-build-test (all 4 platforms), and ci-gate regardless of the review findings. Fixed by narrowing both signatures from &mut Vec<HashMap<String, Ty>> to &mut [HashMap<String, Ty>] (verified neither function calls a Vec-specific method). cargo clippy --workspace --all-targets -- -D warnings now passes clean.
• llms.txt budget regression (D-207): this branch's own docs/ROADMAP.md/docs/PYTHON_STANDARDS.md additions pushed the non-optional-expansion aggregate from 260,922 bytes (main, already only 1,222 bytes under the 262,144-byte ceiling) to 264,533 bytes — a genuine PR-caused regression, not pre-existing. Trimmed both entries to the essential facts; local GITHUB_PAGES=true sh scripts/check-site.sh now passes at 261,976 bytes. The remaining 168-byte margin is thin — worth a follow-up (terser future roadmap entries, or revisiting the 256 KiB ceiling) but out of scope here; noting it rather than filing a new issue since it's a one-line heads-up, not an actionable defect yet.
• status-page-freshness still fails (this branch's ROADMAP.md change wasn't mirrored into site/status/index.html/site/index.html, per issue P1: Keep the GitHub Pages status/roadmap page in sync on every PR #401/D-156), but that workflow is explicitly documented as not yet a required branch-protection check, so it's flagged here rather than fixed to avoid scope creep into rewriting the status page.

Gate results

• cargo test --workspace: 0 failures (all test result: lines green, including the 2 new MIR regression tests and 7 new checker tests).
• cargo llvm-cov --workspace --fail-under-lines 100 --fail-under-regions 100: 100.00% lines / 100.00% regions (42,894 regions, 27,848 lines, 0 missed).
• ruby scripts/check_roadmap_evidence.rb: passes.
• GITHUB_PAGES=true sh scripts/check-site.sh: passes.
• cargo clippy --workspace --all-targets -- -D warnings: passes.

Commits: 98e8c41 (findings 1 & 2 + clippy fix), b7d0340 (llms.txt budget trim).

Not merging — this pass fixed the previously-reviewed findings; a fresh ievo:deep-reviewer pass over these new commits is still needed before merge, and I don't have Agent-tool access in this session to dispatch it myself.

[rotnov]

D-068 re-review round 3: kill-prescan for re-enterable narrowed bodies

This round fixes a soundness defect class in flow-sensitive Optional[T] narrowing found in two new places after two prior fix rounds already landed (through b7d03402): narrowing was established via a single left-to-right source-order pass, reconciled only at control-flow joins, which is unsound whenever a body can be entered/re-entered such that a statement earlier in execution order comes from later in source order than the read it should invalidate.

Repros verified empirically (before/after)

1. Loop re-entry — if x is not None: i=0; while i<2: print(x+1); x=None; i=i+1. Before this fix: accepted (unsound — print(x+1) on iteration 2 actually runs after iteration 1's x=None). After: rejected with T0021.
2. except-from-mid-try — if x is not None: try: x=None; raise ValueError("boom") except ValueError: return x+1. Before: accepted (unsound — the handler is reached only after the kill inside try). After: rejected with T0021.

Both confirmed as real .py files against cargo run --bin pycc -- check/run before any code change, then reconfirmed rejected after the fix.

Also fixed (warning-severity, not soundness): match case bodies and try/except/else/finally bodies were using raw per-statement loops instead of narrow::check_stmt_sequence[_in_function], spuriously rejecting a nested early-return guard that should have narrowed the rest of the same body.

Approach taken and why

Approach 1 (kill-prescan) over Approach 2 (scope-down fallback), per the task's explicit preference. Before checking/lowering a re-enterable body, recursively collect every name that body reassigns anywhere within it (new pycc_hir::killed_names) and drop those names' narrowing overlay entries for the entire body up front — a conservative, non-fixpoint, whole-body rule. Built cleanly with zero regressions on the first attempt, so the scope-down fallback was never needed.

Implemented identically in both pycc_types (checker, via narrow::apply_kill_prescan) and pycc_mir (MIR lowering) — pycc_mir cannot depend on pycc_types, so the shared killed_names primitive lives in pycc_hir, the common dependency.

MIR/checker consistency: grepped every caller of narrowed_ty in crates/pycc_mir/src/expr.rs — exactly one call site, automatically consistent with the checker's new rule since it reads the same scopes structure apply_kill_prescan mutates.

Files and functions changed

• crates/pycc_hir/src/lib.rs — new killed_names/collect_killed_names (canonical primitive: which statement kinds route a bare-name target through assignment/binding).
• crates/pycc_types/src/narrow.rs — new apply_kill_prescan.
• crates/pycc_types/src/lib.rs — apply_kill_prescan wired into While/ForRange/ForList (module-scope and function-scope) and enum-loop helpers; check_match case bodies routed through narrow::check_stmt_sequence[_in_function].
• crates/pycc_types/src/exception.rs — check_try_stmt's body/handler.body/orelse/finalbody loops routed through a new check_stmt_sequence_shared helper (replacing the old per-statement check_stmt_shared, now dead and removed); apply_kill_prescan applied to the try body before checking the handler.
• crates/pycc_mir/src/lib.rs, crates/pycc_mir/src/stmt.rs — mirrored kill-prescan application at the equivalent MIR lowering sites.

Tests added

• crates/pycc_types/src/tests.rs: a_narrowed_read_inside_a_while_loop_body_the_same_body_later_kills_is_rejected, an_except_handler_reached_after_a_try_body_kill_is_rejected, a_while_loop_body_that_reads_but_never_kills_the_narrowed_name_stays_narrowed (completeness guard), plus 5 warning-fix regression tests (nested_early_return_guard_narrows_the_rest_of_the_same_match_case_body, ..._the_try_body, ..._an_except_handler_body, ..._the_else_body, a_plain_narrowed_read_inside_the_finally_body_still_type_checks).
• crates/pycc_mir/src/tests/narrow.rs: a_while_body_that_reads_then_kills_the_narrowed_name_does_not_unwrap_the_read, a_while_test_reading_a_name_the_body_kills_does_not_unwrap_the_test, a_while_body_that_reads_but_never_kills_the_narrowed_name_still_unwraps_the_read (completeness guard), an_except_handler_reached_after_a_try_body_kill_does_not_unwrap_the_read.
• crates/pycc_hir/src/tests.rs: 14 direct unit tests for killed_names/collect_killed_names covering every statement-kind match arm, added to close a coverage gap the new function introduced.

Gates

• cargo test --workspace: 0 failures.
• cargo llvm-cov --workspace --fail-under-lines 100 --fail-under-regions 100: 100.00% lines / 100.00% regions, exit 0.
• cargo clippy --workspace --all-targets -- -D warnings: exit 0.
• ruby scripts/check_roadmap_evidence.rb: pass.
• GITHUB_PAGES=true sh scripts/check-site.sh: pass (llms.txt budget not tripped).

Decision log

New entry D-201 narrows D-199's Consequences claim ("a narrowing fact never survives a reassignment of the narrowed name" — true only for the constructs D-199 covered, not in general once loops/try-except are in scope). D-199 itself is left unedited per this repo's decision-log convention. docs/decisions/README.md regenerated.

Retrospective

Added a 2026-08-25 entry to docs/AGENT_RETROSPECTIVE.md: three consecutive D-068 review rounds against this PR found the same defect class in new constructs because each round fixed only what was flagged instead of characterizing the general invariant up front. Lesson: audit every construct where execution order can diverge from source order as part of original design work, not reactively per review round.

Still needed before merge

Per D-068, another fresh pinned-reviewer (ievo:deep-reviewer) pass is required over this round's diff before merge, same as the prior two rounds. This session does not have Agent-tool access to invoke it directly — a session that does should run it against the current head (18f303cb) before merge. Per D-024, this PR is not being merged by this session; it is left for the repository owner / a session with reviewer access.

[rotnov]

D-068 re-review round 4: doc-completion follow-up

A fourth pinned-reviewer pass against this PR's kill-prescan soundness fix (round 3, 088ec5f0..18f303cb) returned zero blockers — "The soundness claim now holds." It found two should-fix documentation warnings and one optional, explicitly non-blocking note. This round addresses all three.

Warnings fixed

1. docs/TYPE_SYSTEM.md's "Narrowing & flow typing" section was stale — it only described D-199's original if/else narrowing scope. Added a paragraph describing the round-3 (D-201) extension: narrowing now survives into while/for loop bodies (including the loop's own re-evaluated test), try/except handler bodies, and match case bodies, via a conservative whole-body kill-prescan (pycc_hir::killed_names) rather than a fixpoint analysis — plus a one-line note on why finally needs no separate prescan (it's checked against the strict-intersection join of the try body / handlers / orelse). Linked to D-201 following this doc's existing citation convention for other decision entries. Commit 8bab55cd.

2. tests/fixtures/conformance-breadth-manifest.json's PEP 604 row — inspected and left unchanged. The manifest's proven bullets are strictly evidenced by a registered conformance fixture (every existing evidence value across the whole manifest is a tests/fixtures/*.py path); pep_0604_union.py itself does not exercise any loop/try/except/match narrowing. The round-3 kill-prescan capability is covered by Rust unit tests in crates/pycc_types/src/tests.rs (e.g. nested_early_return_guard_narrows_inside_an_unrelated_outer_while_in_function_scope, a_narrowed_read_inside_a_while_loop_body_the_same_body_later_kills_is_rejected, nested_early_return_guard_narrows_the_rest_of_the_same_match_case_body, nested_early_return_guard_narrows_the_rest_of_the_try_body), not by a conformance fixture, so adding a proven bullet citing pep_0604_union.py as evidence would overclaim relative to this manifest's own evidentiary discipline. No change made.

Optional item: D-201's finally justification

Corrected in place (commit ee5810ea). The reviewer traced the actual mechanism precisely: finally is checked against joined, an Environment built by folding body_env (the full try-body walk in source order, including dead code after a raise) through join_loop_body, then intersecting each handler's and the orelse's end-state via join_if_branches — and it's narrow::join_narrowed's strict intersection that actually excludes any killed name, not merely "not a narrowing-sensitive path" as D-201 previously said. Verified this against check_try_stmt in crates/pycc_types/src/exception.rs (lines 102–111) and join_narrowed's doc comment in crates/pycc_types/src/narrow.rs before editing. D-201 is not yet an accepted decision — it lands in this same PR — so AGENTS.md's append-only rule (which applies only to already-accepted entries) doesn't block correcting it in place; a superseding entry is unnecessary here.

Gates — independently re-run this round

• cargo test --workspace: all green (1312-test unit suite plus all integration suites, 0 failed).
• cargo clippy --workspace --all-targets -- -D warnings: clean — only the 3 pre-existing, unrelated warnings in tests/slice1_codegen_depth.rs (multi-line string escape lints), not gated, untouched.
• cargo llvm-cov --workspace --fail-under-lines 100 --fail-under-regions 100: 100.00% lines / 100.00% regions, independently re-run (previous round's report was self-reported-only).
• GITHUB_PAGES=true sh scripts/check-site.sh: passed, independently re-run. docs/TYPE_SYSTEM.md is not referenced by site/llms-txt-context-manifest.json's non-optional documents at all, so the doc addition carries no llms.txt byte-budget risk despite the tight margin noted in the previous round.

All commits are pushed to feat/issue-769-optional-narrowing:

• 8bab55cd — docs(types): describe kill-prescan narrowing scope in TYPE_SYSTEM.md
• ee5810ea — docs(decisions): correct D-201's finally-needs-no-prescan mechanism

Believed merge-ready pending the repository owner's own final look. Not merging myself per D-024.