docs: a walkthrough that runs, and a front page whose output is not typed - #32
Merged
sotashimozono merged 1 commit intoSep 9, 2026
Merged
Conversation
…yped
Measured before writing any of this: `docs/src` had 32 fenced ```julia blocks whose output was
typed by hand, against one `@example` and one `jldoctest`. Two of those 32 had already been caught
producing output the code cannot produce — a report shown in the wrong order, and a frame count
written from a simplified trace.
Worse, the single most important output in the documentation — the exit summary, the thing the
front page exists to sell — was in a ```console block reading `… your output …`. It could not be
an `@example`: the summary fires at exit and a documentation build does not exit between blocks.
That is exactly why it was typed.
`examples/walkthrough.jl` is a Literate source with three consumers and one copy of the truth:
* Literate generates `docs/src/walkthrough.md` at build time — gitignored, so no second drifting
copy is committed
* `julia --project=examples examples/walkthrough.jl` runs it standalone
* `test/test_examples.jl` runs it and asserts what the page claims
It walks one small package through the whole arc — mark, `entered`, `@entered`, `record`, `reach`,
`audit`, `compare`/`isbreaking` — and every step carries the control, because a claim without one
is worth little here: a definition the run never touched is *absent* rather than zero, a caller
with nothing unvalidated behind it gives an empty record, the same removal is breaking or not
depending on whether the notice was in the source.
The exit summary is shown by running `examples/sweep.jl` as its own process and printing what came
back. That also made a property visible that was nowhere in the documentation: the summary goes to
`stderr` and the result to `stdout`, so `julia sweep.jl > result.dat` keeps the data clean. The
test asserts it, and would fail if the summary ever moved streams.
Two things the page says because they were measured, not assumed:
* `:clean` is version-sensitive. A `lengths` that called `tanh` was `:clean` on 1.12.2 and
`:unknown` on 1.14.0-DEV, because resolving `tanh` reaches Base internals that moved. The
walkthrough uses a formulation that is `:clean` on both AND states the sensitivity, since it
is one of the reasons `reach` is itself declared `@experimental`.
* Pinax is not the subject. It has zero marks today, so it could demonstrate `audit` and not the
headline, and making the docs depend on it would put a `[sources]` rev in the docs environment.
Its real audited numbers stay in the prose of `adopting.md`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📚 Docs preview: https://codes.sota-shimozono.com/ExperimentalAPI.jl/previews/PR32/ (updates on each push to this PR) |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
sotashimozono
deleted the
docs/the-walkthrough-runs-rather-than-being-typed
branch
September 9, 2026 07:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measured before writing anything:
docs/srchad 32 fenced ```julia blocks whose output was typedby hand, against one
@exampleand one `jldoctest`. Two of those 32 had already been caughtproducing output the code cannot produce (#26) — a report in the wrong order, and a frame count
written from a simplified trace.
Worse, the single most important output in the documentation — the exit summary, the thing the
front page exists to sell — was a ```console block reading
… your output …. It could not be an`@example`: the summary fires at exit, and a documentation build does not exit between blocks. That
is precisely why it was typed.
One source, three consumers
examples/walkthrough.jlis a Literate source:docs/src/walkthrough.md— gitignored, so no second drifting copy is committedjulia --project=examples examples/walkthrough.jlruns standalonetest/test_examples.jlIt walks one small package through the whole arc — mark,
entered,@entered,record,reach,audit,compare/isbreaking— and every step carries its control, because a claim withoutone is worth little here:
The exit summary, actually run
The page runs
examples/sweep.jlas its own process and prints what came back. That made aproperty visible that was nowhere in the documentation:
the summary goes to
stderr, the result tostdout— sojulia sweep.jl > result.datkeepsthe data clean and still puts the notice in front of whoever ran it. The test asserts it, and fails
if the summary ever changes streams.
Two things the page says because they were measured
:cleanis version-sensitive. Alengthsthat calledtanhwas:cleanon 1.12.2 and:unknownon 1.14.0-DEV — resolvingtanhreaches Base internals that moved. The walkthroughuses a formulation that is
:cleanon both and states the sensitivity, since that is one ofthe reasons
reachis itself declared@experimental.auditbut notthe headline, and depending on it would put a
[sources]rev in the docs environment. Its realaudited numbers stay in the prose of
adopting.md.🤖 Generated with Claude Code