Skip to content

2026.9.30.2: a build that exits, a status row that counts the work, planning that walks each tree once, #744 and #732 - #745

Merged
speak-agent merged 15 commits into
mainfrom
feat/build-count-plan-reuse-module-scope
Sep 30, 2026
Merged

speak-agent merged 15 commits into
mainfrom
feat/build-count-plan-reuse-module-scope

Conversation

@speak-agent

@speak-agent speak-agent commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Five reports on 2026.9.30.1 while building xlings, analysed and planned in .agents/docs/2026-09-30-build-wall-time-progress-count-and-hang-plan.md (measurements in sections 1 and 2, plan in 4 and 5, self-review in 7, tasks in 9, implementation record in 10).

  • W1: a build no longer hangs after ninja. The stack animation could spawn pieces onto cells it already held once its stack reached the right edge short of its target; the loop never ended while the ticker held the line lock, and close_region() joined it for ever. Every loop now grows the stack or ends. A property test over every animation and game fails on 2026.9.30.1 and passes in 4 s.
  • W2: the status row counts the work of the build. 1195 counted steps on a clean build of xlings were 503 cache placements, 460 dependency scans and 232 compiles and links; the row read 967/1195 at the first compile. Ninja passes now have a kind: the cache pass is read but not counted, scans that wait on no action run first as Scanning f/t, and Building f/t is the main pass (0/232 at the first compile). The fast path runs the same passes.
  • W3: vendored xlings: the note that no newer source is available can be false, and it is printed once per configuration load #744. One function chooses the vendored xlings's source (the override, else the newer of the released copy and the PATH copy); Updating and Note are stated once per process; the vendored binary's version is kept per process and under the home, keyed by path, size and modification time, so a planned command no longer runs xlings --version (0.35 s).
  • W8: planning walks each tree once. A source pattern with an empty literal prefix walked its whole package tree; libarchive's 35 directories were opened 13,406 times per plan. A walk is kept per tree for the command and revalidated by directory modification times.
  • W9: planning states its phases under build/stage, and the finish phase its steps.
  • W10: artifacts dependency rejects duplicate C++ module names across independent executables #732. A module name is unique within one program, not within one build: GCC and clang name a module's entities and initializer after it, so one program cannot link two modules of one name, and two programs may each have one. Imports are resolved in the importer's closure by one resolver; two BMIs of one name lie below their packages' directories and each compile is bound to its provider (a module map for GCC, -fmodule-file= for clang, /reference for MSVC; mcpp dyndep --module-map). Without a collision, build.ninja and compile_commands.json of the xlings workspace are byte-identical to 2026.9.30.1's.
  • Found by CI: a BMI served from the global cache waits for the modules it imports that the build compiles. xpkg's build program generates lua_stdlib below the consumer's target directory, so the package is staged with its other five units and lua_stdlib compiles in every build; nothing ordered a consumer of the staged executor BMI after that compile, and the aarch64-linux-musl cross build of xlings failed with failed to read compiled module. The defect predates this branch (it needs a warm cache and a fresh build directory; the scan pass changed the schedule). Such a stage edge now waits for those BMIs and leaves the aggregate every compile waits for (section 10.6; unit test and e2e 849, each failing on the previous binary).
  • W4 (plan reuse) is deferred on the plan's own gate: after W3 and W8, planning is 0.43 s of an 8.0 s edit (section 10.3).
  • Version 2026.9.30.2, CHANGELOG, and the guides (status row counts; the module name rule; workspace members) in both languages. The xlings pin is already the latest release. The xlings start-up cost is stated as xlings --version spends 0.35 s in start-up work before it prints the version openxlings/xlings#638.

Refs #744, #732.

Measured on xlings (this machine)

Build 2026.9.30.1 this branch
Clean, wall time 35.4 s 34.4 s
Clean, ninja starts at about 4.5 s about 1.3 s
Edit of one source 10.5 s 8.0 s
No-op 0.05 s 0.05 s
Planning of that edit 3.05 s 0.43 s

Test plan

  • Unit tests: dots screen (property test), xlings version (source choice, memo, once per process), modgraph (module resolution), pack interface, dyndep, build progress (pass kinds).
  • e2e 846 (vendored xlings: the note that no newer source is available can be false, and it is printed once per configuration load #744), 847 (artifacts dependency rejects duplicate C++ module names across independent executables #732, under GCC 16 and clang 22), 848 (MSVC, runs on the Windows leg), 687, 842, 843, 845, 801, 805, 806, 19, 172, 196, 114 locally.
  • Revert probes: the property test, e2e 846 and e2e 847 fail on 2026.9.30.1.
  • Byte comparison of build.ninja and compile_commands.json of the xlings workspace against 2026.9.30.1 (no collision): identical.
  • check_version_pins, check_docs_structure, check_docs_style, check_file_lengths, check_modules_wiring, check_narrow_conversions, check_reason_tokens, the design index.
  • e2e 849 (fails at its graph criterion and, alone, at its ninja criterion on the previous binary); on xlings, the ninja probe of section 10.6 reproduces the CI error with the previous binary and builds with this one.
  • e2e 846 on Windows: the test's PATH holds System32, where where lies.
  • GalTranslPP verification (Sunrisepeak/GalTranslPP, branch verify/mcpp-2026.9.30.2) on this head.
  • CI on Linux, macOS and Windows.

Every phase of prepare_build logs its duration, as the backend's own steps do,
whenever the log file or --verbose would show it. The backend's stage lines are
recorded under the same condition, not only under --verbose. A planned edit of
one source spent 3.05 s before ninja that could only be attributed from gaps
between unrelated log lines.
A piece that lands with a cell outside the screen is not spawned, and a piece
that adds no cell ends the fill loop, so every iteration grows the stack or ends
the loop. Before, a stack whose holes reached the right edge short of the
fraction's target spawned pieces onto cells it already held, the loop never
ended while the ticker held the line lock, and the build hung after ninja.

A property test drives every animation over 2000 seeds and every game over 500
under a watchdog. It fails on the previous stack animation (the stack leg does
not return within 120 s) and passes in 4 s.
…ocess (#744)

One function chooses the source: MCPP_VENDORED_XLINGS when set, otherwise the
newer of the xlings released with this mcpp and the xlings on PATH, the
released one on a tie. The check that decides whether to replace the vendored
binary and the copy that replaces it both use its answer, so the version stated
is the version copied. Before, the check took the first source that existed, so
a released copy older than the pin hid a newer xlings on PATH and mcpp stated
that no newer source was available.

A home settled once in a process is not examined again, and Updating and Note
are each stated at most once. The version of the vendored binary is asked once
per process and kept under the home, keyed by the binary's path, size and
modification time, so a command that loads the configuration no longer runs
xlings --version (0.35 s measured) when the binary has not changed.

e2e 846 covers the three source cases; it fails on 2026.9.30.1 with the false
note of #744.
A glob's walk is kept per root and start, and each pattern is matched against
the kept listing, first by the literal text after its last star. Every directory
the walk entered is examined again before the listing is reused, and a directory
modified within two seconds of the walk is never trusted, so files that planning
writes are seen.

Before, a pattern with an empty literal prefix walked the whole package tree:
the 127 source patterns of compat.libarchive, expanded about three times per
plan, opened its 35 directories 13,406 times. Measured on a planned edit of one
xlings source: 30,372 directory opens of installed packages fall to 1,749, the
scan phase from 0.91 s to 43 ms, and ninja starts at 0.66 s instead of 3.05 s.
A module name identifies one module within one program: GCC and clang name a
module's entities and its initializer after it, so a program cannot link two
modules of one name (measured: multiple definition of value@common()), and two
programs may each have one. A build holds several programs, a package and the
programs it ships through artifacts or the members of a workspace, and mcpp
refused a name that two packages of one build provided whichever programs they
belonged to.

The prepare phase computes each compiled package's closure (code and member
edges; not artifacts, tools or build-dependencies; no closure for a
workspace's virtual root). One resolver, choose_provider and resolve_provider
in mcpp.modgraph, answers which provider an import means, and the scanner,
the interface check, the packer and the backend's std reachability all use
it. Two providers of one name are refused only when one closure holds both,
and one file reached twice is recognised by identity rather than spelling.

When two packages provide a name, their BMIs lie below their packages'
directories and every unit that may import the name is bound to its provider:
a module map for GCC, whose mapper does not fall back for an unlisted name,
-fmodule-file= for clang and /reference for MSVC; mcpp dyndep reads the same
map through --module-map. With every name provided once nothing changes:
build.ninja and compile_commands.json of the xlings workspace are
byte-identical to 2026.9.30.1's. A package that provides a collided name is
compiled rather than served from the global cache. The plan's unread module
map is removed.

e2e 847 (GCC and clang) and 848 (MSVC) cover an artifacts updater, two
workspace members, the reported layout, and the two refusals; 847 fails on
2026.9.30.1 with the error of #732.
The finish phase took 301 ms of a planned edit of one xlings source; its steps
now state their durations under build/stage like the phases: make plan 112 ms
and the dependency cache 176 ms.
A clean build of xlings counted 1195 steps on its status row, of which 503 were
placements of the cache pass and 460 were dependency scans, and read 967/1195
when its first compile began. Ninja passes now have a kind. The cache pass is a
placement pass: its Cached lines are written as before and its steps are not
counted. The scans that wait on no action run in a pass of their own, before
the main pass, shown as Scanning f/t; a scan that waits on a package's prepare
or check action stays in the main pass, which keeps such an action from
holding back every compile. Building f/t is the main pass: 0/232 at the first
compile of the same build. The fast path runs the same passes. A failed scan
pass ends the build with its own output. A build of named goals scans in its
main pass.

Measured on xlings: the scan pass takes 20 ms on an edit and about 0.3 s on a
clean build, whose scans all ended in the first quarter second of the main
pass before.
Every package scans in the scan pass, and naming each package that finished a
step when that pass ended put a package before the one it imports (e2e 842). A
package that only scanned is named when the main pass ends, as before.
The version is 2026.9.30.2 in mcpp.toml and MCPP_VERSION. The CHANGELOG entry is
the release's notes. The commands guide states what the status row counts, the
dependencies guide states that a module name is unique within one program, and
the workspace guide no longer refuses two members that share a module name and
no program; both languages. The design record states the measurements, the plan,
the tasks across repositories, and the implementation record, including W4's
deferral on its own gate.
Eight sections against the published release: the three #732 cases, index
packages built with it, the status row's count against the units the cache
placed, #744's source choice, and the planning timers. Run locally against the
branch: 8 passed; against 2026.9.30.1 as the control, every section marked
CHANGE fails.
The ninth section builds the largest consumer of mcpp, xlings, from its main
branch with the release under test and runs the program. Locally against the
branch: 9 passed.
… upgrade, the GCC map

- A `**/` also matches no directory, so the literal tail of `**/name` does not
  start with its slash; a root-level match of `**/main.cpp` was dropped.
- Under -k the scans run in the main pass; the scan pass takes
  MCPP_NINJA_DEBUG, and -j precedes its goal.
- The versions of the vendored xlings's candidate sources are read through the
  memo, and an upgrade copies beside the binary and renames over it, so a
  failed copy keeps the old binary.
- The GCC module map also lists a name a unit imports that no unit provides,
  at GCC's own path; the clang and MSVC paths use native separators.
… one configuration

When two packages of one configuration provide a module name, both units list
it, and a unit that may import it carries the binding the build uses in its
arguments; a consumer that looks providers up by name may take the other
program's module. With one provider per name the document is unchanged (#732).
Under --verbose the thirty planning lines reached the terminal, and e2e 198 on
the Linux-to-Windows leg read one of them, `plan finish windows resources`,
as a non-PE build speaking about resources. The planning phases and the
finish phase's steps are recorded with log::info, in the log file that
--verbose or MCPP_LOG_LEVEL=info enables; a record to read afterwards is not
output of the build. The verification reads them from the file.
A package's cache entry holds the units below its root. A module its build
program generates lies below the consumer's target directory and compiles in
every build, also when the rest of the package is staged (xpkg's lua_stdlib,
imported by its cached executor). The consumer's dyndep named the staged BMI
only and the stage edge had no input but the cache entry, so a fresh build
could compile the consumer first: the aarch64-linux-musl cross build of
xlings failed with `failed to read compiled module`. The stage edge of such a
BMI now waits for the BMIs it imports that compile here, and leaves the
aggregate every compile waits for, which would otherwise form a cycle
(NinjaBackend.AStagedBmiWaitsForTheModulesItImportsThatCompileHere, e2e 849).

e2e 846: on Windows the test's PATH holds System32, where `where` lies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant