Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,062 changes: 1,062 additions & 0 deletions .agents/docs/2026-09-30-build-wall-time-progress-count-and-hang-plan.md

Large diffs are not rendered by default.

163 changes: 163 additions & 0 deletions .agents/docs/2026-09-30-build-wall-time-verify.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
#!/usr/bin/env bash
# Sandbox verification of mcpp 2026.9.30.2 (.agents/docs/2026-09-30-build-
# wall-time-progress-count-and-hang-plan.md), run against the PUBLISHED release
# inside an xlings sandbox:
#
# B64=$(base64 -w0 .agents/docs/2026-09-30-build-wall-time-verify.sh)
# xlings subos new v9302 2>/dev/null || true
# xlings subos use v9302 --sandbox --cmd "echo $B64 | base64 -d > /tmp/v.sh && VER=2026.9.30.2 bash /tmp/v.sh"
#
# VER selects the release under test. Running it with VER=2026.9.30.1 is the
# control: every section marked CHANGE must fail there, and every other section
# must pass on both.
#
# Every probe directory is removed at the start of its section, because the
# sandbox's $HOME persists between runs of the same subos. A section that does
# not run is reported as SKIP and counted separately from a pass.
set -u
VER="${VER:-2026.9.30.2}"
W="$HOME/v9302"
fails=0; passes=0; skips=0
pass() { echo "PASS $1"; passes=$((passes+1)); }
fail() { echo "FAIL $1"; [ -n "${2:-}" ] && [ -f "$2" ] && tail -15 "$2"; fails=$((fails+1)); }
skip() { echo "SKIP $1"; skips=$((skips+1)); }

# ── 0. the release under test, from the published channel ──────────────────
if [ -n "${MCPP_OVERRIDE:-}" ]; then
MCPP="$MCPP_OVERRIDE"
else
xlings config --mirror CN >/dev/null 2>&1 || true
xlings update >/dev/null 2>&1 || true
xlings install "mcpp@$VER" -y > /tmp/v9302-install.log 2>&1 || true
MCPP="$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp"
fi
if [ ! -x "$MCPP" ]; then
echo "FATAL: mcpp $VER is not installable from the index"; tail -20 /tmp/v9302-install.log; exit 2
fi
got=$("$MCPP" --version 2>&1 | head -1)
case "$got" in *"$VER"*) pass "0 installed: $got";; *) fail "0 version: $got";; esac
"$MCPP" self config --mirror CN >/dev/null 2>&1 || true
mkdir -p "$W"

module_file() { printf 'export module boost;\nexport int value() { return %s; }\n' "$2" > "$1"; }
main_file() { printf '#include <cstdio>\nimport boost;\nint main() { std::printf("%%d\\n", value()); }\n' > "$1"; }
bin_of() { find "$1" -path '*/bin/*' -name "$2" -type f | head -1; }

# ── 1. CHANGE: #732, an artifacts program with its own module of one name ──
rm -rf "$W/s1"; mkdir -p "$W/s1/app/src" "$W/s1/updater/src"; cd "$W/s1"
module_file app/src/boost.cppm 1; main_file app/src/main.cpp
module_file updater/src/boost.cppm 2; main_file updater/src/main.cpp
printf '[package]\nname = "updater"\nversion = "0.1.0"\n\n[targets.updater]\nkind = "bin"\nmain = "src/main.cpp"\n' > updater/mcpp.toml
printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[dependencies]\nupdater = { path = "../updater", artifacts = ["updater"] }\n\n[targets.app]\nkind = "bin"\nmain = "src/main.cpp"\n' > app/mcpp.toml
if (cd app && "$MCPP" build > ../s1.log 2>&1) \
&& [ "$("$(bin_of app/target app)")" = 1 ] && [ "$("$(bin_of app/target updater)")" = 2 ]; then
pass "1 CHANGE: the app and its artifacts updater each have their own module boost"
else fail "1 CHANGE: two programs, one module name" s1.log; fi

# ── 2. #732, two providers in one program are refused ───────────────────────
rm -rf "$W/s2"; mkdir -p "$W/s2/lib1/src" "$W/s2/lib2/src" "$W/s2/prog/src"; cd "$W/s2"
module_file lib1/src/boost.cppm 1; module_file lib2/src/boost.cppm 2
for l in lib1 lib2; do printf '[package]\nname = "%s"\nversion = "0.1.0"\n' "$l" > $l/mcpp.toml; done
printf 'int main() { return 0; }\n' > prog/src/main.cpp
printf '[package]\nname = "prog"\nversion = "0.1.0"\n\n[dependencies]\nlib1 = { path = "../lib1" }\nlib2 = { path = "../lib2" }\n\n[targets.prog]\nkind = "bin"\nmain = "src/main.cpp"\n' > prog/mcpp.toml
if (cd prog && "$MCPP" build > ../s2.log 2>&1); then fail "2 two providers in one program were accepted" s2.log
elif grep -q "module 'boost' is provided by package" s2.log; then pass "2 two providers in one program are refused"
else fail "2 the refusal does not name the module" s2.log; fi

# ── 3. CHANGE: #732, two workspace members with one module name ─────────────
rm -rf "$W/s3"; mkdir -p "$W/s3/one/src" "$W/s3/two/src"; cd "$W/s3"
module_file one/src/boost.cppm 5; main_file one/src/main.cpp
module_file two/src/boost.cppm 6; main_file two/src/main.cpp
printf '[workspace]\nmembers = ["one", "two"]\n' > mcpp.toml
for m in one two; do printf '[package]\nname = "%s"\nversion = "0.1.0"\n\n[targets.%s]\nkind = "bin"\nmain = "src/main.cpp"\n' "$m" "$m" > $m/mcpp.toml; done
if "$MCPP" build --workspace > s3.log 2>&1 \
&& [ "$("$(bin_of target one)")" = 5 ] && [ "$("$(bin_of target two)")" = 6 ]; then
pass "3 CHANGE: two workspace members each have their own module boost"
else fail "3 CHANGE: two workspace members, one module name" s3.log; fi

# ── 4. index packages build and run with the release ───────────────────────
rm -rf "$W/s4"; mkdir -p "$W/s4/src"; cd "$W/s4"
printf '[package]\nname = "eco"\nversion = "0.1.0"\n\n[dependencies]\n"compat.zlib" = "*"\n"mcpplibs.cmdline" = "*"\n\n[targets.eco]\nkind = "bin"\nmain = "src/main.cpp"\n' > mcpp.toml
cat > src/main.cpp <<'EOF'
#include <cstdio>
#include <zlib.h>
import mcpplibs.cmdline;
int heavy();
int main() { std::printf("zlib %s\n", zlibVersion()); return heavy() == 0; }
EOF
# One unit that takes a few seconds to compile, so that section 5's build
# outlives the half second before the status row is first drawn.
cat > src/heavy.cpp <<'EOF'
#include <format>
#include <regex>
#include <string>
int heavy() {
std::regex r("([a-z]+)-([0-9]+)");
std::smatch m;
std::string s = std::format("{}-{}", "mcpp", 2026);
return std::regex_match(s, m, r) ? static_cast<int>(m.size()) : 0;
}
EOF
if "$MCPP" build > s4.log 2>&1 && "$(bin_of target eco)" | grep -q '^zlib '; then
pass "4 compat.zlib and mcpplibs.cmdline from the index build and run"
else fail "4 index packages" s4.log; fi

# ── 5. CHANGE: the status row counts the work of the build ─────────────────
# A pty makes the status row appear; the project of section 4 has a
# dependency the global cache serves after its first build.
cd "$W/s4"
if command -v script > /dev/null 2>&1 && [ -f s4.log ] && grep -q 'Finished' s4.log; then
"$MCPP" clean > /dev/null 2>&1
MCPP_PROGRESS=plain script -qefc "stty cols 160 rows 40; $MCPP build" s5.pty > /dev/null 2>&1
rows=$(sed 's/\x1b\[[0-9;?]*[A-Za-z]//g' s5.pty | tr '\r' '\n')
# The units the cache placed, from the `Cached ... (N units)` lines; the
# Building total must not include them (2026.9.30.1 counted each one).
placed=$(echo "$rows" | grep -a '^ *Cached ' | grep -ao '[0-9]* units\?)' | grep -o '^[0-9]*' \
| awk '{s += $1} END {print s + 0}')
building=$(echo "$rows" | grep -ao 'Building [0-9]*/[0-9]*' | head -1)
total=${building##*/}
if [ -n "$total" ] && [ "$placed" -gt 0 ] && [ "$total" -lt "$placed" ]; then
pass "5 CHANGE: Building counts $total steps, not the $placed units placed from the cache"
else fail "5 CHANGE: the status row (first Building: '$building', units placed: $placed)" s5.pty; fi
else skip "5 no script(1) or section 4 did not build"; fi

# ── 6. CHANGE: #744, the vendored xlings comes from the newest source ──────
rm -rf "$W/s6"; mkdir -p "$W/s6/rel/bin" "$W/s6/rel/registry/bin" "$W/s6/pathbin"; cd "$W/s6"
NINJA=$(ls "$HOME"/.mcpp/registry/data/xpkgs/xim-x-ninja/*/ninja 2>/dev/null | head -1)
REAL="$HOME/.mcpp/registry/bin/xlings"
if [ -n "$NINJA" ] && [ -x "$REAL" ]; then
older=$("$NINJA" --version | head -1)
export_home="$W/s6/home"
cp "$MCPP" rel/bin/mcpp
cp "$NINJA" rel/registry/bin/xlings
cp "$REAL" pathbin/xlings
MCPP_HOME="$export_home" MCPP_OFFLINE=1 rel/bin/mcpp self env > setup.log 2>&1 || true
mkdir -p "$export_home/registry/bin"; rm -f "$export_home/registry/bin/xlings"; cp "$NINJA" "$export_home/registry/bin/xlings"
env -u MCPP_VENDORED_XLINGS MCPP_HOME="$export_home" MCPP_OFFLINE=1 PATH="$W/s6/pathbin:/usr/bin:/bin" \
rel/bin/mcpp self env > s6.out 2> s6.err || true
if grep -q "vendored xlings $older -> .* from PATH" s6.err; then
pass "6 CHANGE: a newer xlings on PATH replaced the vendored one past an older released copy"
else fail "6 CHANGE: #744" s6.err; fi
else skip "6 no ninja payload or vendored xlings to stand in"; fi

# ── 7. planning states its phases ──────────────────────────────────────────
cd "$W/s4" && touch src/main.cpp
LOGFILE="$HOME/.mcpp/log/mcpp.log"
before=$(wc -c < "$LOGFILE" 2>/dev/null || echo 0)
if MCPP_LOG_LEVEL=info "$MCPP" build > s7.log 2>&1 \
&& tail -c +$((before + 1)) "$LOGFILE" 2>/dev/null | grep -q 'build/stage: plan scan'; then
pass "7 CHANGE: planning states its phases under build/stage"
else fail "7 CHANGE: the planning phase timers" s7.log; fi

# ── 8. the largest consumer: xlings builds from its main branch ────────────
rm -rf "$W/s8"; mkdir -p "$W/s8"; cd "$W/s8"
if git clone -q --depth 1 https://github.com/openxlings/xlings.git xlings > s8-clone.log 2>&1; then
cd xlings
if "$MCPP" build > ../s8.log 2>&1 && "$(bin_of target xlings)" --version 2>/dev/null | grep -q '^xlings '; then
pass "8 xlings builds from its main branch and runs: $(grep -a 'Finished' ../s8.log | tail -1 | sed 's/\x1b\[[0-9;]*m//g; s/^ *//')"
else fail "8 xlings from its main branch" ../s8.log; fi
else skip "8 xlings could not be cloned"; fi

echo
echo "RESULT: $passes passed, $fails failed, $skips skipped (mcpp $VER)"
[ "$fails" -eq 0 ]
4 changes: 3 additions & 1 deletion .agents/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded
---
```

319 records.
320 records.

## By subject

Expand All @@ -30,6 +30,7 @@ Records that declare one. Everything else is listed by date below.

### design

- [The build's wall time, its progress count, a hang after the build, and #732 and #744: measurements and a remediation plan](2026-09-30-build-wall-time-progress-count-and-hang-plan.md) — landed
- [Build output, revision 3: every package that does work is named, the live display is one line drawn in one write, and a repeated warning is stated once per file](2026-09-30-build-output-refinement-design.md) — landed
- [The workspace as the unit of build: one graph per configuration, one scheduler, product directories, and a reusable graph module](2026-09-29-workspace-build-graph-design.md) — landed
- [Build progress: each step's line states its outcome, and one status line states the build](2026-09-29-build-progress-display-design.md) — landed
Expand Down Expand Up @@ -110,6 +111,7 @@ Records that declare one. Everything else is listed by date below.

### 2026-09

- [The build's wall time, its progress count, a hang after the build, and #732 and #744: measurements and a remediation plan](2026-09-30-build-wall-time-progress-count-and-hang-plan.md) — landed
- [Build output, revision 3: every package that does work is named, the live display is one line drawn in one write, and a repeated warning is stated once per file](2026-09-30-build-output-refinement-design.md) — landed
- [The workspace as the unit of build: one graph per configuration, one scheduler, product directories, and a reusable graph module](2026-09-29-workspace-build-graph-design.md) — landed
- [Build progress: each step's line states its outcome, and one status line states the build](2026-09-29-build-progress-display-design.md) — landed
Expand Down
69 changes: 69 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,75 @@
> Each `## [<version>]` section is that release's notes. Entries are written in English
> from 2026.9.28.3 on; earlier entries remain as written.

## [2026.9.30.2] - 2026-09-30

This release answers five reports on 2026.9.30.1 while building xlings
(`.agents/docs/2026-09-30-build-wall-time-progress-count-and-hang-plan.md`): a
build that did not exit after its status row stopped, a status row whose count
was mostly bookkeeping, planning that preceded every edit's compile by three
seconds, mcpp#744, and mcpp#732. On a clean build of xlings the command starts
ninja at 1.3 s instead of 4.5 s; an edit of one source builds in 8.0 s instead
of 10.5 s; a build with nothing to do is unchanged at 0.05 s.

### Fixed

- **A build no longer hangs after ninja.** The stack animation could spawn
pieces onto cells it already held once its stack reached the right edge
short of its target, and its loop then never ended while the status row held
the terminal's line lock; the build joined the row's thread and waited for
ever (about 1% of interactive builds). Every loop of the animation now grows
the stack or ends. A property test drives every animation over 2000 seeds and
every game over 500 under a watchdog; it fails on the previous animation.
- **The status row counts the work of the build.** A clean build of xlings
counted 1195 steps, of which 503 placed files the global cache serves and 460
were dependency scans, and read 967/1195 when its first compile began. The
cache pass is reported by its `Cached` lines and not counted; the scans that
wait on no action run first, shown as `Scanning f/t`; and `Building f/t`
counts the compiles, links, archives and actions: 0/232 at the first compile
of the same build. The fast path runs the same passes (e2e 842, 843).
- **A vendored xlings is replaced from the newest source (mcpp#744).**
`MCPP_VENDORED_XLINGS` when set, otherwise the newer of the xlings released
with mcpp and the xlings on `PATH`; the note that no newer source is
available was false when a newer xlings was on `PATH`. `Updating` and `Note`
are each stated once per process (e2e 846).
- **A module name is unique within one program, not within one build
(mcpp#732).** An `artifacts` program, or a workspace member that shares no
program with another, may provide a module of the same name as another
program of the build. An import is resolved in the importing package's
closure; two BMIs of one name lie below their packages' directories, and each
compile is told which one a name means (a module map for GCC,
`-fmodule-file=` for Clang, `/reference` for MSVC). Two providers that one
program links are refused, naming that program's package, and one file
reached as two packages is recognised whatever its spelling. When every name
has one provider, `build.ninja` and `compile_commands.json` are byte-identical
to 2026.9.30.1's (e2e 847, 848).
- **A BMI served from the global cache waits for the modules it imports that
the build compiles.** A module that a package's build program generates lies
below the consumer's target directory and is compiled in every build, also
when the rest of the package is staged from the cache (xpkg's `lua_stdlib`,
imported by its cached `executor`). Nothing ordered a consumer of the staged
BMI after that compile, so a fresh build could compile the consumer first and
fail with `failed to read compiled module`. The stage edge of such a BMI now
waits for those BMIs (e2e 849).

### Changed

- **Planning walks each package tree once.** A source pattern with an empty
literal prefix walked the whole package tree: the 127 patterns of
`compat.libarchive`, expanded about three times per plan, opened its 35
directories 13,406 times. A walk is now kept per tree for the command and
revalidated by its directories' modification times. A planned edit of one
xlings source opens 1,749 package directories instead of 30,372, and its
scan phase takes 43 ms instead of 0.91 s.
- **The version of the vendored xlings is asked once.** It is kept per
process, and under the home keyed by the binary's path, size and
modification time, so a command that loads its configuration no longer runs
`xlings --version` (0.35 s) when the binary has not changed.
- **Planning states where its time goes.** Each phase of planning, and each
step of its last phase, logs its duration under `build/stage` in the log
file, which `--verbose` or `MCPP_LOG_LEVEL=info` enables. The backend's own
stage lines are recorded in the file under the same condition.

## [2026.9.30.1] - 2026-09-30

This release revises what a build prints, from a report on `mcpp build` in the
Expand Down
29 changes: 29 additions & 0 deletions docs/05-dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,35 @@ updater = { path = "../updater", artifacts = ["updater"] }
> by nothing that made a decision: writing it produced a manifest that loaded,
> no diagnostic, and no effect.

### One module per name in each program (mcpp 2026.9.30.2+)

A module name identifies one module within one program. The compilers name a
module's entities and its initializer after the module, so a program cannot
link two modules of one name. A build can hold several programs (a package and
the programs it ships through `artifacts`, the members of a workspace), and
each of them may have its own module of one name.

- An import is resolved within the importing package's closure: the package
and every package it reaches through its dependencies. The closure does not
follow `artifacts`, `tools` or `[build-dependencies]` edges, whose programs
are built separately.
- Two packages that one program links may not provide the same name. The build
is refused, and the message names the package whose closure holds both.
- When two packages of one build provide a name, each BMI lies below its
package's directory in the build directory, and every compile that may import
the name is told which one it means: through a module map with GCC, through
`-fmodule-file=` with Clang, and through `/reference` with MSVC. When every
name has one provider, the build directory and every command are as they
were before.
- A package that provides such a name is compiled in the project, not served
from the global dependency cache.
- clangd finds a module by its name in the compilation database, so for a name
two packages provide it may show the other program's module. The build is
not affected.

A module that several programs share is best provided by one package that the
others depend on; it is then compiled once.

## Current limitations

- **Two things need the network, and only two:** resolving a branch that has no
Expand Down
8 changes: 5 additions & 3 deletions docs/07-workspace.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,9 +451,11 @@ member that several members use is compiled once.
`compile_commands.json` once, as the union of their databases (2026.9.29.5+).
- **No-op builds.** A command repeated with nothing changed is answered by one
check per configuration, without planning.
- **Module names.** Members built in one graph share one module namespace:
two members that each provide a module of the same name cannot be built in
one `--workspace` command; build each with `-p`.
- **Module names.** A module name is unique within one program, not within one
graph (2026.9.30.2+). Two members that share no program may each provide a
module of the same name, and one `--workspace` command builds both; a member
that links both is refused. See
[05 — One module per name in each program](05-dependencies.md#one-module-per-name-in-each-program-mcpp-20269302).

## 6. Directory Layout

Expand Down
Loading
Loading