diff --git a/docs/src/adopting.md b/docs/src/adopting.md index cf67d21..88608c5 100644 --- a/docs/src/adopting.md +++ b/docs/src/adopting.md @@ -9,10 +9,21 @@ and no idea which of them anybody meant. ## Start by looking -```julia -julia> using MyPackage, ExperimentalAPI +```@setup adopting +using ExperimentalAPI +module MyPackage +using ExperimentalAPI +public settled, unfinished, undescribed +"Settled and documented." +settled(x) = x +"Documented, and not finished." +@experimental "the tolerance is a guess" unfinished(x) = x +undescribed(x) = x +end +``` -julia> ExperimentalAPI.audit(MyPackage) +```@repl adopting +ExperimentalAPI.audit(MyPackage) ``` Two numbers matter. `undocumented` is the backlog. `dangling` should be zero on day one, because diff --git a/docs/src/analysing.md b/docs/src/analysing.md index 3f7c79c..54474c7 100644 --- a/docs/src/analysing.md +++ b/docs/src/analysing.md @@ -7,15 +7,22 @@ CurrentModule = ExperimentalAPI [`entered`](@ref) and [`record`](@ref) say what a run *did*. [`reach`](@ref) says what a caller *could* do — before running it, and including through code that never names the marked thing. -```julia -julia> r = ExperimentalAPI.reach(analyse, Tuple{Model,Float64}); - -julia> ExperimentalAPI.verdict(r) -:depends +```@setup analysing +using ExperimentalAPI +module MyModel +using ExperimentalAPI +public energy, inner, sweep, analyse +@experimental "convergence not established below β ≈ 0.1" energy(β::Float64) = -log(2cosh(β)) / β +inner(β::Float64) = energy(β) * 2 +sweep(βs::Vector{Float64}) = (t = 0.0; for β in βs; t += inner(β); end; t) +analyse(βs::Vector{Float64}) = round(sweep(βs); digits = 4) +end +``` -julia> r.reached -1-element Vector{ExperimentalAPI.Reached}: - ExperimentalAPI.Reached(MyModel.energy via analyse → sweep → inner → energy) +```@repl analysing +r = ExperimentalAPI.reach(MyModel.analyse, Tuple{Vector{Float64}}); +ExperimentalAPI.verdict(r) +r.reached ``` The model is Lean's `sorry`: a proof that uses one is not a proof, however many layers down it diff --git a/docs/src/checking.md b/docs/src/checking.md index e91f289..172d298 100644 --- a/docs/src/checking.md +++ b/docs/src/checking.md @@ -23,13 +23,23 @@ This is the part the rest of the package exists for. A mark says the shape is unsettled. That is never a reason to say nothing about what the name does, so a marked name with no prose is a finding exactly as an unmarked one is. -```julia -julia> ExperimentalAPI.audit(Archeion) -Public surface of Archeion — 36 names - documented 33 - experimental 0 - undocumented 3 ← no docstring - FTPSTransport, pull_file, push_dir +```@setup checking +using ExperimentalAPI +module Archeion +using ExperimentalAPI +public ingest, fetch_record, FTPSTransport, pull_file, push_dir +"Ingest a document." +ingest(x) = x +"Fetch one record." +fetch_record(x) = x +struct FTPSTransport end +pull_file(t::FTPSTransport, p) = p +push_dir(t::FTPSTransport, p) = p +end +``` + +```@repl checking +ExperimentalAPI.audit(Archeion) ``` The list is never truncated. A coverage report that elides its tail reads as if the tail were @@ -110,15 +120,28 @@ see exactly which names it would have to argue about. Empty means the two agree. ## The method-level half -```julia -julia> ExperimentalAPI.audit(Downstream).contributed_methods -4-element Vector{Method}: - fetch_value(::Ising, ::Energy) … - ⋮ - -julia> ExperimentalAPI.unaccounted_methods(Downstream) # neither a docstring nor a mark -2-element Vector{Method}: - ⋮ +```@setup checking2 +using ExperimentalAPI +module Upstream +struct Ising end +struct Energy end +fetch_value(m, q) = 0.0 +end +module Downstream +using ExperimentalAPI +using ..Upstream: Upstream, Ising, Energy +public Widget +struct Widget end +"Documented, on somebody else's generic." +Upstream.fetch_value(::Ising, ::Energy) = 1.0 +@experimental "extrapolated; no reference value" Upstream.fetch_value(::Ising, ::Int) = 2.0 +Base.show(io::IO, ::Widget) = print(io, "") +end +``` + +```@repl checking2 +ExperimentalAPI.audit(Downstream).contributed_methods +ExperimentalAPI.unaccounted_methods(Downstream) # neither a docstring nor a mark ``` Docstrings are keyed by signature, so [`isdocumented`](@ref)`(::Method)` is a real question and @@ -138,15 +161,28 @@ under the ones Julia adds next. `require_methods = :all` widens it. The mark records where it was written, and `--code-coverage` records a count per line. Joining the two answers the worst case a marked definition can be in: -```julia -julia> ExperimentalAPI.unverified(MyPackage) # marked AND never executed by the suite -1-element Vector{ExperimentalAPI.Mark}: - ExperimentalAPI.Mark(MyPackage.never_called, "shipped without ever being called") +```@setup checking3 +using ExperimentalAPI +module MyPackage +using ExperimentalAPI +public exercised, never_called +"Run by the suite." +@experimental "reference value not cross-checked" exercised(x) = x + 1 +"Never run." +@experimental "shipped without ever being called" never_called(x) = x * 0 +end +MyPackage.exercised(1) +``` -julia> ExperimentalAPI.coverage(MyPackage, :half_exercised) -0.6 +```@repl checking3 +ExperimentalAPI.unverified(MyPackage) # marked AND never entered by this process +ExperimentalAPI.coverage(MyPackage, :exercised) ``` +`unverified` reads the probe, which is exact and needs no coverage run. `coverage` is the partial +fraction and answers `missing` here, because a documentation build has no `--code-coverage` +counters — which is the next paragraph's point, shown rather than described. + [`coverage`](@ref) answers `missing`, never `0.0`, when the run has no coverage data: a run without `--code-coverage` has nothing to say, and reporting zero would flag every marked definition in every ordinary run. The counters are flushed from the running process rather than diff --git a/docs/src/declaring.md b/docs/src/declaring.md index d8a77c6..6b87072 100644 --- a/docs/src/declaring.md +++ b/docs/src/declaring.md @@ -37,13 +37,22 @@ Why a name is not settled is knowledge only the author has. A reader can see tha odd; they cannot see that it is odd *because* a refactor upstream has not landed. So the reason is required, and the mark carries it everywhere the name goes: -```julia -julia> ExperimentalAPI.mark(Archeion, :ingest) -Archeion.ingest — experimental - reason: signature will be wrapped once the write-back refactor settles - since: v0.1.4 - tracking: https://github.com/QAtlasHub/Archeion.jl/issues/12 - declared: /…/Archeion.jl/src/ingest.jl:153 +```@setup declaring +using ExperimentalAPI +module Archeion +using ExperimentalAPI +public ingest +@experimental( + "signature will be wrapped once the write-back refactor settles", + since = v"0.1.4", + tracking = "https://example.invalid/issues/12", + ingest(config; doc = nothing) = config, +) +end +``` + +```@repl declaring +ExperimentalAPI.mark(Archeion, :ingest) ``` `since` and `tracking` are optional and go between the reason and the subject: @@ -62,14 +71,25 @@ shape they need. ## What it attaches to `function`, short-form `f(x) = …`, `struct`, `mutable struct`, `abstract type`, -`primitive type`, `macro` (recorded as `Symbol("@name")`), `const`, and plain assignment. +`primitive type`, `macro` (recorded as `Symbol("@name")`), `const`, plain assignment, a definition +wrapped in `@inline` and its neighbours, and a method on **another module's** generic — +`Base.show(io, ::Widget) = …`. That last one is not on `names(m)` and never can be, so it is +reported by [`contributed_methods`](@ref) rather than by the name audit. Anything else is **refused with a message naming the alternative**, never guessed at: -```julia -julia> @experimental "why" Base.sum(x::Int) = x -ERROR: ArgumentError: @experimental: `Base.sum` defines a name owned by another module, -which is not part of this module's public surface +```@example declaring +try + @eval module Refused + using ExperimentalAPI + module Sub + g(x) = x + end + @experimental "why" Sub.g + end +catch e + showerror(stdout, e isa LoadError ? e.error : e) +end ``` The refused cases and why: @@ -77,8 +97,9 @@ The refused cases and why: | | | |---|---| | `module M … end` | Julia requires it as a direct top-level statement, so it cannot be wrapped — use the name-list form | -| `Base.foo(x) = …` | adds a method to a name this module does not own; it is not on your surface | -| `@somemacro …` | the macro cannot know which name the expansion defines — name it explicitly | +| a bare `Sub.g` | names somebody else's generic without saying **which** method — mark the definition | +| `begin f(x)=x; g(x)=x end` | two names, one mark; marking the first and dropping the second is a covenant that omits a definition | +| `@somemacro …` | outside the annotating allowlist the macro cannot know which name the expansion defines | Guessing in any of these cases would produce a mark on the wrong symbol, which is worse than no mark: `audit` would then report it as `dangling` and the author would be debugging this package @@ -109,17 +130,16 @@ A mark says nothing about visibility. A name still has to be `export`ed or decla be part of the surface, and a mark on a name that is neither promises nothing to anyone — [`audit`](@ref) reports it as `dangling`: -```julia +```@example declaring module M using ExperimentalAPI @experimental "not settled" helper(x) = x # never exported, never public end +nothing # hide ``` -```julia -julia> ExperimentalAPI.audit(M).dangling -1-element Vector{Symbol}: - :helper +```@repl declaring +ExperimentalAPI.audit(M).dangling ``` That check needs no reference to be right. The module is disagreeing with itself. diff --git a/docs/src/observing.md b/docs/src/observing.md index f7f0b27..62b278d 100644 --- a/docs/src/observing.md +++ b/docs/src/observing.md @@ -5,6 +5,19 @@ DocTestSetup = quote end ``` +```@setup observing +using ExperimentalAPI +module MyModel +using ExperimentalAPI +public energy, step, sweep, simulate +@experimental "convergence not established below β ≈ 0.1" energy(β::Float64) = -log(2cosh(β)) / β +step(β::Float64) = energy(β) + tanh(β) +sweep(βs::Vector{Float64}) = (t = 0.0; for β in βs; t += step(β); end; t) +simulate(βs::Vector{Float64}; steps::Int = 1) = (t = 0.0; for _ in 1:steps; t += sweep(βs); end; t) +end +βs = collect(0.05:0.05:2.0) +``` + # Observing A docstring can say a name is unfinished. It cannot tell you that *this run* went through it. @@ -44,16 +57,14 @@ Three properties, each of them a decision: [`entered`](@ref) returns the same thing the summary prints, as a `Vector{`[`Entry`](@ref)`}`: -```julia -julia> ExperimentalAPI.entered() -1-element Vector{ExperimentalAPI.Entry}: - ExperimentalAPI.Entry(Main.energy, "convergence not established below β ≈ 0.1") +```@repl observing +MyModel.energy(0.5) +ExperimentalAPI.entered(MyModel) ``` -The display above is a transcript rather than a doctest on purpose: `Entry` prints its module, and -Documenter evaluates doctests in a sandbox whose module does not print as `Main`, so a doctest here -would show a line no reader ever sees at their own REPL. The fields it is read for do not depend on -where it ran, so those are checked: +Asked about a module rather than about the whole process, because the process-wide answer includes +every marked package loaded — on this page, the fixtures the documentation built. The fields it is +read for do not depend on where it ran, and those are checked as a doctest: ```jldoctest julia> using ExperimentalAPI @@ -116,21 +127,21 @@ table above. ## Recording: counts, paths and time -```julia -r = ExperimentalAPI.record() do - simulate(model; steps = 10_000) +```@example observing +r = ExperimentalAPI.record(; paths = false) do + MyModel.simulate(βs; steps = 500) end +nothing # hide ``` -```julia -julia> r -Record — 1 marked definition entered in 0.42s - MyModel.energy ×10000 — convergence not established below β ≈ 0.1 - inclusive 0.31s exclusive 0.28s - via record → sweep → step → energy - recorder overhead ≈ 4.1% +```@example observing +r ``` +`paths = false` here because a captured path is the whole call stack, and on this page that starts +at `_start` and runs through `makedocs`. The timings are `missing`: a documentation build does not +load `Profile`, and `record` reports what it measured rather than a zero. + [`record`](@ref) returns a `Vector`-like of [`Hit`](@ref), so `isempty(r)` and `r[1].count` read the way they look — with the properties an empty vector could not carry: @@ -147,25 +158,24 @@ the way they look — with the properties an empty vector could not carry: [`@entered`](@ref) is the same question asked about an expression, and it knows two things a closure cannot — the source text of the call and the line it is written on: -```julia -julia> ExperimentalAPI.@entered sweep(model; βs = 0.05:0.05:2.0) -┌ @entered sweep(model; βs = 0.05:0.05:2.0) at sweep.jl:42 -│ MyPkg.correlator × 500 — edge cases at zero separation untested -│ MyPkg.energy ×10000 — convergence not established below β ≈ 0.1 -└ 15 of 17 observable marked definitions were not entered -0.42713… +```@example observing +report = IOBuffer() +value = ExperimentalAPI.@entered report MyModel.sweep(βs) +print(String(take!(report))) ``` +The footer counts every observable marked definition **loaded in the process**, not only the ones +in your package. + It returns the value of the expression, so it drops into existing code the way `@time` does — with one divergence `@time` does not have, since `record` takes a function and the expression therefore runs inside a closure: a `return` inside it returns from the closure, and `@entered y = f(x)` binds `y` inside the closure. Write `y = @entered f(x)`. The last line is what makes a clean answer mean anything: -```julia -julia> ExperimentalAPI.@entered publish(result) -┌ @entered publish(result) at sweep.jl:57 -└ entered nothing marked — 17 observable marked definitions were loaded +```@example observing +ExperimentalAPI.@entered report round(1.0; digits = 2) +print(String(take!(report))) ``` "Entered nothing" and "nothing is marked anywhere" are different states, and a package that has diff --git a/examples/walkthrough.jl b/examples/walkthrough.jl index e5ce85c..e1c9638 100644 --- a/examples/walkthrough.jl +++ b/examples/walkthrough.jl @@ -115,8 +115,19 @@ record(() -> Ising.normalise(βs, 0.5); paths=false, timing=false) # # [`@entered`](@ref) asks the same question about a single expression, and returns its value, so # it drops into existing code the way `@time` does. +# +# Written here as `@entered expr` and printed, because Documenter's `@example` renders a +# block's **value** when it has one and its captured output only when the value is `nothing` — +# measured with a control, a plain `println` followed by `99` loses its line the same way. Run +# from a terminal, `ExperimentalAPI.@entered Ising.report(βs)` prints the report itself. + +report = IOBuffer() +value = ExperimentalAPI.@entered report Ising.report(βs) +print(String(take!(report))) + +#- -value = ExperimentalAPI.@entered Ising.report(βs) +value # ## How often, and how much of the run #