Skip to content

docs: a walkthrough that runs, and a front page whose output is not typed - #32

Merged
sotashimozono merged 1 commit into
mainfrom
docs/the-walkthrough-runs-rather-than-being-typed
Sep 9, 2026
Merged

docs: a walkthrough that runs, and a front page whose output is not typed#32
sotashimozono merged 1 commit into
mainfrom
docs/the-walkthrough-runs-rather-than-being-typed

Conversation

@sotashimozono

Copy link
Copy Markdown
Member

Measured before writing anything: docs/src had 32 fenced ```julia blocks whose output was typed
by hand
, against one @example and one `jldoctest`. Two of those 32 had already been caught
producing output the code cannot produce (#26) — a report in the wrong order, and a frame count
written from a simplified trace.

Worse, the single most important output in the documentation — the exit summary, the thing the
front page exists to sell — was a ```console block reading … your output …. It could not be an
`@example`: the summary fires at exit, and a documentation build does not exit between blocks. That
is precisely why it was typed.

One source, three consumers

examples/walkthrough.jl is a Literate source:

consumer what it gets
Literate at docs-build time docs/src/walkthrough.md — gitignored, so no second drifting copy is committed
a reader julia --project=examples examples/walkthrough.jl runs standalone
test/test_examples.jl runs it and asserts what the page claims

It walks one small package through the whole arc — mark, entered, @entered, record, reach,
audit, compare/isbreaking — and every step carries its control, because a claim without
one is worth little here:

  • a marked definition the run never touched is absent, not reported as zero
  • a caller with nothing unvalidated behind it gives an empty record, not a small one
  • the same removal is breaking or not depending on whether the notice was in the source

The exit summary, actually run

The page runs examples/sweep.jl as its own process and prints what came back. That made a
property visible that was nowhere in the documentation:

the summary goes to stderr, the result to stdout — so julia sweep.jl > result.dat keeps
the data clean and still puts the notice in front of whoever ran it. The test asserts it, and fails
if the summary ever changes streams.

Two things the page says because they were measured

  • :clean is version-sensitive. A lengths that called tanh was :clean on 1.12.2 and
    :unknown on 1.14.0-DEV — resolving tanh reaches Base internals that moved. The walkthrough
    uses a formulation that is :clean on both and states the sensitivity, since that is one of
    the reasons reach is itself declared @experimental.
  • Pinax is not the subject. It has zero marks today, so it could demonstrate audit but not
    the headline, and depending on it would put a [sources] rev in the docs environment. Its real
    audited numbers stay in the prose of adopting.md.

🤖 Generated with Claude Code

…yped

Measured before writing any of this: `docs/src` had 32 fenced ```julia blocks whose output was
typed by hand, against one `@example` and one `jldoctest`. Two of those 32 had already been caught
producing output the code cannot produce — a report shown in the wrong order, and a frame count
written from a simplified trace.

Worse, the single most important output in the documentation — the exit summary, the thing the
front page exists to sell — was in a ```console block reading `… your output …`. It could not be
an `@example`: the summary fires at exit and a documentation build does not exit between blocks.
That is exactly why it was typed.

`examples/walkthrough.jl` is a Literate source with three consumers and one copy of the truth:

  * Literate generates `docs/src/walkthrough.md` at build time — gitignored, so no second drifting
    copy is committed
  * `julia --project=examples examples/walkthrough.jl` runs it standalone
  * `test/test_examples.jl` runs it and asserts what the page claims

It walks one small package through the whole arc — mark, `entered`, `@entered`, `record`, `reach`,
`audit`, `compare`/`isbreaking` — and every step carries the control, because a claim without one
is worth little here: a definition the run never touched is *absent* rather than zero, a caller
with nothing unvalidated behind it gives an empty record, the same removal is breaking or not
depending on whether the notice was in the source.

The exit summary is shown by running `examples/sweep.jl` as its own process and printing what came
back. That also made a property visible that was nowhere in the documentation: the summary goes to
`stderr` and the result to `stdout`, so `julia sweep.jl > result.dat` keeps the data clean. The
test asserts it, and would fail if the summary ever moved streams.

Two things the page says because they were measured, not assumed:

  * `:clean` is version-sensitive. A `lengths` that called `tanh` was `:clean` on 1.12.2 and
    `:unknown` on 1.14.0-DEV, because resolving `tanh` reaches Base internals that moved. The
    walkthrough uses a formulation that is `:clean` on both AND states the sensitivity, since it
    is one of the reasons `reach` is itself declared `@experimental`.
  * Pinax is not the subject. It has zero marks today, so it could demonstrate `audit` and not the
    headline, and making the docs depend on it would put a `[sources]` rev in the docs environment.
    Its real audited numbers stay in the prose of `adopting.md`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

📚 Docs preview: https://codes.sota-shimozono.com/ExperimentalAPI.jl/previews/PR32/

(updates on each push to this PR)

@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sotashimozono
sotashimozono merged commit 931879a into main Sep 9, 2026
14 checks passed
@sotashimozono
sotashimozono deleted the docs/the-walkthrough-runs-rather-than-being-typed branch September 9, 2026 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant