Skip to content

Fix stale Output Interpretation numbers in underdiagnosis-bias.md - #566

Merged
yakew7 merged 1 commit into
yakew7:mainfrom
propcgamer20-png:fix/underdiagnosis-bias-stale-output
Sep 10, 2026
Merged

Fix stale Output Interpretation numbers in underdiagnosis-bias.md#566
yakew7 merged 1 commit into
yakew7:mainfrom
propcgamer20-png:fix/underdiagnosis-bias-stale-output

Conversation

@propcgamer20-png

Copy link
Copy Markdown
Collaborator

Closes #523.

The Output Interpretation prose under the detection-code block quoted numbers from an earlier version of the code that were never regenerated after an edit - the same stale-paste pattern as the parallel reject-inference.md issue.

Actual output of the file's own python block

Seeded with np.random.seed(42), byte-identical across repeated runs:

figure doc claimed code actually prints
true_recall, Group A vs B 80.5% vs 41.2% 62.5% vs 29.2%
true false-negative gap 39.3 points 33.3 points
obs_recall, Group A vs B ~0.85 vs ~0.76 ~0.64 vs ~0.33
Q4 biomarker-band diagnosis rate 0.887 / 0.536 0.785 / 0.497

Change

  • Rewrote the two interpretation bullets with the real figures.
  • Adjusted the item 1 framing: observed-label recall already shows a Group A/B gap in the real output, so the "the model appears to perform decently across both groups" line no longer described what the code prints. The core teaching point (standard evaluation scores the true-disease false negatives as correct True Negatives) is unchanged.
  • Regenerated underdiagnosis-bias.html, faircode/_explainers/underdiagnosis-bias.md, and the llms-full.txt section via scripts/build_explainers.py.

check_em_dash.py, check_broken_links.py, check_explainer_count.py, check_generated_files_current.py all pass.

…oses yakew7#523)

The 'Output Interpretation' prose cited figures from an earlier version of
the detection-code block that were never regenerated after the code was
edited. Running the file's own python block (seeded with
np.random.seed(42), byte-identical across repeated runs) actually prints:

- true_recall 0.625 (Group A) vs 0.292 (Group B) - a 33.3-point gap, not
  the claimed 80.5% / 41.2% / 39.3-point gap
- obs_recall ~0.64 vs ~0.33, not the claimed ~0.85 / ~0.76
- Q4 biomarker-band recorded-diagnosis rate 0.785 / 0.497, not 0.887 / 0.536

Updated the two interpretation bullets to the real numbers and adjusted
the framing (the observed-label recall already shows a gap here, so the
'looks fine on observed labels' line no longer fit). Regenerated
underdiagnosis-bias.html, the package mirror copy, and the llms-full.txt
section from the build script.
@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

@propcgamer20-png is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown

@yakew7 @Shreyash0712 - new PR touching a path you own, please review.

@yakew7
yakew7 merged commit c28b524 into yakew7:main Sep 10, 2026
18 of 19 checks passed
yakew7 added a commit that referenced this pull request Sep 10, 2026
Documents the newest explainer (#565), the CODEOWNERS grant to
@propcgamer20-png, 8 doc/code fixes (#557-564, #566, #567), and the
two direct fixes made during this review: the README.md regression
from #557's broad substring replacement, and the recurred
llms.txt/ROADMAP.md explainer-count drift after #565.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

underdiagnosis-bias.md's "Output Interpretation" doesn't match its own code

2 participants