From 27908ddb31736f961473c043676945d639472e05 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 04:17:35 +0800 Subject: [PATCH 01/11] Workspace build graph, first part: E1 removed, the directory named by the configuration, linked placement, the virtual root and member edges --- ...2026-09-29-workspace-build-graph-design.md | 499 ++++++++++++++++++ modules/manifest/src/dep_spec.cppm | 5 + modules/manifest/src/types.cppm | 5 + src/build/ninja_backend.cppm | 6 +- src/build/plan.cppm | 10 - src/build/prepare.cppm | 5 + src/build/prepare/features.cpp | 10 +- src/build/prepare/graph.cpp | 62 ++- src/build/prepare/graph_load.cpp | 3 +- src/build/prepare/manifest.cpp | 243 ++++++--- src/build/prepare/plan.cpp | 190 +------ src/build/prepare/scan.cpp | 37 +- src/build/prepare/state.cppm | 31 +- src/build/prepare/target_side.cpp | 213 +++++++- src/build/prepare/toolchain.cpp | 4 + src/build/prepare_inputs.cppm | 324 +----------- src/build/stage.cppm | 33 +- src/modgraph/scanner.cppm | 4 + src/project.cppm | 183 +++++++ .../830_a_shared_member_is_compiled_once.sh | 118 ----- tests/unit/test_build_profile.cpp | 61 ++- tests/unit/test_build_stage.cpp | 42 ++ 22 files changed, 1315 insertions(+), 773 deletions(-) create mode 100644 .agents/docs/2026-09-29-workspace-build-graph-design.md delete mode 100755 tests/e2e/830_a_shared_member_is_compiled_once.sh diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md new file mode 100644 index 00000000..d3047da1 --- /dev/null +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -0,0 +1,499 @@ +# The workspace as the unit of build: one graph per configuration, one scheduler, product directories, and a reusable graph module + +- Status: design, agreed; implementation in progress (revision 4) +- Date: 2026-09-29 +- Origin: the #734 validation on a five-member Windows workspace (the validation + project's pull request), and the discussion that followed it +- Replaces: E1 of `.agents/docs/2026-09-28-build-cost-foreign-toolsets-and-library-surface-design.md` + (a shared member built once through a per-consumer admission check) + +Revision 2 settles the configuration key field by field (§3), places each +member's products in its own directory instead of refusing shared names (§5), +adds hard-linked placement (§5.3), folds the build key over the graph module's +order (§9), and adds the self-review (§12). Revision 3 records a planning +regression of 2026.9.28.3 and its cause (§1.1). Revision 4 drops the separate +patch release that revision 3 proposed: the regression and the cost underneath +it are both fixed by this design, in one release (§14), and §15 divides the +work into tasks with their dependencies across the repositories. + +## 0. Scope + +`mcpp build --workspace` builds the members one after another, each as an +independent root with its own plan and its own ninja. This document replaces that +loop with a workspace-level build: the members and everything they depend on are +nodes of one graph per configuration, and one scheduler runs all graphs of a +command. It also extracts the graph algorithms the engine has written by hand +several times into one module under `modules/`. + +Out of scope: the parallelism an action carries inside itself (a `cmake --build +--parallel` inside a `prepare` action), which is a resource declaration of actions +and needs its own design (§10). + +## 1. Readings + +| Reading | Value | Source | +|---|---|---| +| `mcpp build --workspace`, 2026.9.28.3, vcpkg binaries cached | 1828 s | validation project, run 36464299728 | +| the same with 2026.9.28.2 | 1726 s | run 36378870254 | +| compilations of the core library in one workspace build | 3 (itself, CLI, GUI) | the build log: `Compiling gpp.core (path)` in each consumer | +| E1 on that workspace | admitted `gpp.version`, refused the core library | the build log; the reason is printed under `-v` (the run that follows revision 1) | +| lines printed per consumer by E1's admission | the member's full header: `Workspace building member`, the toolchain resolution, the build-program lines | the build log | +| `mcpp pack --format release`, two members | 209 s | run 36464299728 | +| what the build directory's name hashes today | the toolchain, the triple, the standard library, the C++ standard, the runtime contract, the std BMI, the mcpp version, the lock, and **every package's** flags (`canonical_compile_flags` of the root plus `canonical_package_build_metadata` of all packages) | `modules/toolchain-model/src/fingerprint.cppm`, `src/build/prepare/scan.cpp` | +| ninja in the xlings index | 1.12.1 only; the jobserver client arrived in ninja 1.13 | `xim-pkgindex/pkgs/n/ninja.lua` | +| how a ninja edge names a file | by the declared string: an absolute and a relative path to the same bytes are two nodes | `.agents/docs/2026-08-29-build-rule-package-spec.md` §11.4 | +| the working directory of every edge | the build directory; advice and `${mcpp.target_file:}` are build-dir-relative | `src/build/advice.cppm`, the same spec | +| MinGW `as`, `ld`, `collect2`, `ar` in a directory outside the ANSI code page | work only with relative paths (measured) | `.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md` | +| hand-written graph algorithms | module order (Kahn, `modgraph/graph.cppm`, reports the nodes left rather than the cycle), unit order by imports (iterative DFS, `prepare/features.cpp`), host-module order (post-order DFS with a cycle path, same file), dependency closure (`build/dep_graph.cppm`), the build-key fold (`prepare/plan.cpp`) | the sources | + +### 1.1 The planning regression of 2026.9.28.3 + +A user's nine-member workspace took 6.3 s for a build with nothing to do under +2026.9.28.2 and 79 s under 2026.9.28.3, while compiling took about one second. +Reproduced locally (Linux, released binaries, a build with nothing to do): + +| Workspace | 2026.9.28.2 | 2026.9.28.3 | +|---|---|---| +| nine members, eight programs using one library | 4.8 s, 9 toolchain resolutions | 5.2 s, 17 | +| five libraries in a chain and one program, `--workspace` | 7.6 s, 6 | 36.8 s, 63 | +| the same chain, `-p app` only | not measured | 24.1 s, 32 (2^5) | + +**Cause.** E1 plans a shared member with a new `BuildOverrides`, whose +`tool_depth` is 0, so the nested plan meets E1's own condition and plans that +member's member dependencies again, recursively. A chain of n members costs 2^n +full plans for `-p` on its end, and each plan resolves the toolchain, resolves +dependencies, checks the xlings payloads and the build programs. + +**The cost underneath.** Even under 2026.9.28.2, `--workspace` takes no fast +path: every member is planned in full on every command (about 1.2 s per member +here), because the fast path is recorded per root and the workspace loop builds +each member as a root. + +How other build systems place outputs, for §5: + +| Tool | Intermediate outputs | Final outputs | Runtime files | +|---|---|---|---| +| Cargo | `target//deps/`, file names carrying a metadata hash | flat in `target//`; a shared name is a warning and the later one wins | none; `cargo run` sets the library path | +| Bazel | `bazel-bin//` | `bazel-bin//` | one `.runfiles/` tree per binary (symbolic links) | +| CMake | the source directory's mirror under the build directory | the same, or one flat directory on Windows (`CMAKE_RUNTIME_OUTPUT_DIRECTORY`) so that DLLs are shared | install rules | +| Meson | the source directory's mirror | the same | install rules | + +## 2. Principles + +1. **A build directory is one configuration and holds one ninja graph whose + paths are relative to it.** This is the convention of CMake, Meson and GN + (`ninja -C `), and three facts of this engine depend on it: file + identity in ninja is the declared string, every edge runs in the build + directory, and the MinGW tools need relative paths. The design keeps it. +2. **The workspace is the unit of build.** A command on a workspace plans the + workspace; `-p` selects goals inside that plan. +3. **A node is a (package, configuration) pair.** A package appears once per + configuration; deduplication is a property of the graph, not a check. +4. **No configuration is refused.** Members that differ in toolchain, target, + standard or runtime contract build, in separate graphs. +5. **A program's directory is its runtime closure.** `mcpp run` sees what + `mcpp pack` stages. Each member's products therefore live in the member's own + product directory. +6. **One mechanism, reused.** The planner already builds a root with its + dependencies in one graph, each package with its own flags, features and + build program; the workspace plan is that planner with a virtual root. + Graph algorithms live in one module. + +## 3. Configurations + +A **configuration** is the set of inputs that every node of one graph must share, +because the compiled outputs of two configurations cannot be linked or imported +together. Everything else is an attribute of a package node: it enters that +package's command lines and its build key, and it does not decide which graph a +package is in. + +| Input (field of today's fingerprint) | Class | Why | +|---|---|---| +| compiler, its version and driver identity | configuration | one toolchain per graph | +| target triple | configuration | one target per graph | +| standard library and its version | configuration | objects must agree on it | +| C++ runtime contract (`/MD` against `/MT`, static against shared runtime) | configuration | a whole-program property (docs/20) | +| the std module BMI | configuration | one per graph; every importer must match it | +| C++ standard (`[package] standard`) | configuration | a BMI is refused across standards | +| dialect flags (`dialect_cxxflags`) | configuration | they must reach `import std` as well | +| macOS deployment target | configuration | the std module and the ABI follow it | +| the C-environment realisation (`[c-abi]` tokens, the openkal kernel ABI) | configuration | objects disagree otherwise (the design of 2026-09-18 §3.4) | +| profile (`opt`, `debug`, `lto`), defined at the workspace root | configuration | selected per command, one per graph | +| mcpp version | configuration | one per build | +| `cflags`, `cxxflags`, `ldflags`, defines, include directories, per-glob flags | package | per-package flags exist today (e2e 50) | +| `c_standard` | package | per-package since 2026.9.26 | +| features | package | per-package since the start | +| `mcpp.lock` | not an input | the lock is not hashed today (`dependencyLockHash` is empty); a dependency's version reaches its own commands through its source paths | + +Members whose configuration inputs are equal share one graph; members whose +inputs differ get separate graphs, each compiled correctly. + +**The directory name.** Today the directory hashes every package's flags, so +editing one `cxxflag` of one member moves the whole graph to a new directory and +rebuilds everything. In a workspace graph that would rebuild every member for an +edit to one. The directory name is therefore the configuration key alone; +a change to a package's flags changes that package's command lines, and ninja +rebuilds exactly the edges whose commands changed (it records them in +`.ninja_log`). Criterion: an edit to one member's `cxxflags` recompiles that +member's units and nothing else (§11). + +## 4. One graph per configuration + +For each configuration the planner builds one graph with a **virtual workspace +root**: + +- the root has no sources; it depends by `path` on every member of that + configuration; +- the link units are the members' targets (programs, libraries, tests); +- every package, member or dependency, appears once, with its own flags, + features, target rows and build program, as dependencies appear today; +- the build directory is `/target///`, + with paths relative to it. + +A shared member is compiled once because it is one node. E1's admission check, +its nested plans, its stage edges from member directories and its lock are no +longer needed (§7). + +A workspace whose root is itself a package (#725) has that package among the +members of its configuration. + +## 5. Where outputs go + +### 5.1 Two kinds of output + +| Kind | Place | Shared by | +|---|---|---| +| intermediate: objects, BMIs, static libraries, generated sources | `/obj//`, as today | every consumer in the graph | +| products: a member's programs and shared libraries, with their deployed files and DLLs | the member's **product directory** `/bin//` | that member only | + +The product directory keeps principle 5: the files a program reads at run time +and the DLLs it loads are beside it, exactly as `mcpp pack` stages them. Two +members with a program of the same name do not collide, and the deployment sets +of two programs never mix. + +### 5.2 Names + +- A member's product directory is named by its **package name** (`bin/cli/`, + `bin/gui/`). +- When two members of one configuration share a package name (they are in + different namespaces), both use the qualified name `namespace.name`. +- A package that is not in a workspace, and the root package of a rooted + workspace, keep `bin/` itself, so every single-package project keeps its paths. +- A member's tests keep the placement they have today, under its product + directory. + +### 5.3 Hard-linked placement + +A runtime file that several programs need (a Qt DLL, a data tree) is placed in +each product directory that needs it. On one file system the placement links +instead of copying: the bytes exist once and the placement costs no copy. The +placement already writes out of place and renames, so replacing the file at one +path never changes the other. Where a link cannot be made (another volume, a file +system without links, a permission) it copies, which is today's behaviour. +Criterion: two product directories that place the same file share its inode or +file index where the file system allows it (§11). + +## 6. One scheduler for all configurations + +A command may need several configurations: members with different toolchains, +and host tools built for the build machine. The configurations form a small +dependency graph: a host tool must exist before the build program that uses it +runs, because build programs run while planning. + +- **Order.** Configurations are planned and built in that dependency order; + independent configurations are independent tasks. +- **Concurrency.** Independent configurations run at the same time, each as its + own `ninja -C `, drawing on one pool so that the total stays + within the command's `-j`: with ninja 1.13 or later mcpp is the jobserver; with + ninja 1.12.1, the index's version today, the pool is divided statically. +- The common case, every member in one configuration, is one ninja and needs + none of this. + +## 7. Commands and compatibility + +### 7.1 Commands on a workspace + +| Command | Behaviour | +|---|---| +| `mcpp build --workspace` | every configuration's graph, all goals | +| `mcpp build -p X` | X's configuration's graph, planned for X and what X reaches | +| `mcpp build` in a member's directory | as `-p X` for that member | +| `mcpp build` at the root of a rooted workspace | as `-p` for the root package | +| `mcpp run -p X`, `mcpp test -p X`, `mcpp pack -p X` | the same graph; the artifact is in X's product directory | +| a package that is not in a workspace | unchanged | + +`-p X` plans X and what X reaches, not the whole workspace, in the same build +directory as `--workspace`. Revision 3 planned the whole graph for `-p X`; the +ecosystem review (§12) refuted that: the package index is a virtual workspace +of 172 example members whose CI runs `mcpp test -p ` once per member, +so a whole-graph `-p` would resolve and fetch every example's dependencies on +every run, and one member whose planning fails would fail every other +member's `-p`. The build directory is still shared: a node's path and command +are functions of the package, the configuration and the package's features, so +switching between `-p X`, `-p Y` and `--workspace` rebuilds only a package whose +unified features differ between the selections, as in Cargo. + +Supporting changes: one fast-path record per configuration, so a workspace +build with nothing to do costs one check per configuration instead of one full +plan per member (§1.1); one compile database per configuration covering every +member (S1); output with one header per configuration and one line per package +compiled. + +### 7.2 Compatibility and upgrade + +- **Build directories move.** A member's outputs move from + `/target/...` to `/target///bin//`. + The first build after the upgrade is a full build; `mcpp clean --stale` + removes the old member directories. A script that reads a member's `target/` + directly reads the product directory instead; `mcpp build --message-format + json` and `mcpp run -p` name the artifacts. +- **Editing one member's flags no longer rebuilds the others** (§3). +- **E1 is removed.** Its behaviour is subsumed, and it has no user-facing + interface; its printed line disappears with it. +- **Unchanged:** a package outside a workspace; the build-program protocol; the + manifest vocabulary; `-p` naming. + +## 8. Cross-platform notes + +- Paths stay relative to the build directory on every host (principle 1). +- A product directory adds one segment (`bin//`); long paths are covered + by the extended-length form the engine already uses on Windows. +- Links: NTFS, ext4, APFS and XFS support hard links; FAT and exFAT, and a + destination on another volume, fall back to copying. +- A Windows program loads its DLLs from its own directory, which is its + product directory. +- The jobserver of §6 is a named semaphore on Windows and a FIFO elsewhere in + ninja 1.13; the static division works everywhere. + +## 9. A reusable graph module (`modules/graph`, `mcpp.graph`) + +One module over index-addressed directed graphs, with no engine types: + +| Operation | Used by | +|---|---| +| stable topological order (ties in input order), with a cycle reported as the path of its nodes | module order (`modgraph`), host-module order, unit order by imports, the configuration scheduler, the build-key fold | +| dependency levels | the configuration scheduler | +| transitive closure and direct neighbours | `build/dep_graph.cppm`, kept as its API and implemented on the module | + +The build-key fold in `prepare/plan.cpp` keeps its computation (a value per +package from its dependencies' values, with the local-source taint); what it +takes from the module is the order and the cycle report, so the recursive walk +becomes a loop over that order. The module has its own unit tests (orders, ties, +cycles, levels, closure). + +## 10. Outside this design + +| Item | Why | +|---|---| +| The parallelism inside an action (`cmake --build --parallel` in a `prepare` action) | an action's resource declaration; it oversubscribes ninja's `-j` today as well | +| Running the build programs of one configuration in parallel while planning | a planning-time optimisation, measurable after this lands | +| Adding ninja 1.13 to the xlings index | an ecosystem step; §6 is correct without it | + +## 11. Criteria + +| Criterion | Test | +|---|---| +| Each package is compiled once per configuration in a workspace build | e2e: a shared member, counted from the one `.ninja_log` | +| `-p` and `--workspace` share one build directory | e2e: `--workspace`, then `-p X` compiles nothing | +| Units of different members overlap | e2e: a unit of one member starts before another member's units finish (`.ninja_log` start times) | +| An edit to one member's flags recompiles that member only | e2e | +| Members of two configurations both build | e2e: a member with another standard or target row beside a default member | +| Two members with a program of the same name both build, in their own product directories | e2e | +| Two members with the same package name in different namespaces get qualified product directories | e2e | +| A file placed in two product directories shares its storage where links are possible | e2e (inode on POSIX; file index on Windows) | +| `mcpp clean --stale` removes the old member directories | e2e | +| `mcpp.graph` | unit tests | +| A workspace build with nothing to do is one fast-path check per configuration | e2e: the chain of §1.1 with nothing changed prints no toolchain resolution and finishes in well under a second | +| Planning is linear in the members | e2e: the chain of §1.1, one resolution per configuration when something changed | +| The validation project | its Windows CI: `mcpp build --workspace` against 1828 s, the core library compiled once, the Release layout unchanged | + +## 12. Self-review + +Each angle states what was checked and what changed because of it. + +**Architecture.** +- The design has two levels, configurations and packages, and each has one owner: + the scheduler owns configurations (§6), the planner owns packages (§4). No + mechanism spans both. +- *Found:* the planner treats `packages[0]` as a root with sources, targets, a + toolchain pin, hooks and a build program. A virtual root has none of these. + *Resolution:* the implementation enumerates every read of the root in the + planner (a table in the implementation record) and states for each whether it + reads the workspace (toolchain pin, profile, lock) or the members (targets, + hooks, build programs). +- *Found (revision 4, ecosystem review):* planning the whole graph for `-p X` + multiplies the cost of every `-p` by the workspace and couples every member + to the planning of every other; the package index runs `mcpp test -p` once + for each of its 172 members. *Resolution:* `-p X` plans X's closure in the + shared build directory (§7.1); the shared state is kept by node identity + rather than by one `build.ninja`. + +**Compatibility.** +- Workspace members' output paths change; this is the one visible break, stated + in the CHANGELOG with the replacement (§7.2). Single-package projects keep + every path. +- *Found:* taking per-package flags out of the directory name changes when a + full rebuild happens. *Resolution:* it only removes rebuilds (§3), and a + criterion checks that an edited member is rebuilt. +- E1 shipped in 2026.9.28.3 without an interface; removing it breaks no manifest. + +**Elegance and simplicity.** +- E1's admission check, nested plans, member stage edges and lock are removed; + the workspace uses the dependency machinery the planner already has. +- Four hand-written graph algorithms become calls to one module. +- *Found:* revision 1 refused shared program names, a rule that existed only + because of a layout choice. *Resolution:* product directories remove the case + instead of guarding it. + +**Cross-platform.** +- Relative paths are kept, so the MinGW and MAX_PATH constraints are unaffected. +- Hard links have a copy fallback on every host (§8). +- *Found:* concurrency across configurations needs a jobserver that the index's + ninja lacks. *Resolution:* static division now, the jobserver when ninja 1.13 + is in the index; the common single-configuration case needs neither. + +**Usability.** +- `run`, `test` and `pack` with `-p` work as before; `mcpp build` in a member's + directory means that member, as in Cargo. +- A workspace build prints one header per configuration and one line per + package, instead of a header per member and per shared dependency. +- Switching between `-p` targets rebuilds nothing. + +**Semantic clarity.** +- Three terms carry the design, each defined once: configuration (§3), node + (§2), product directory (§5). +- "The directory is the configuration" and "a program's directory is its + runtime closure" are stated as principles and each has a criterion. +- *Found:* revision 1 called the per-package inputs "removed from the key", + which read as if they stopped mattering. *Resolution:* §3 names them package + attributes and states where they go. + +## 13. Decisions + +| # | Decision | Status | +|---|---|---| +| 1 | The configuration key is the table of §3; the directory name is that key alone | agreed | +| 2 | The workspace build directory is `/target///`; member directories become stale | agreed | +| 3 | Products in `bin//`; qualified names when two members share a name; single packages and root packages keep `bin/` | agreed (package name) | +| 4 | Configurations run concurrently under one pool; static division until ninja 1.13 is in the index | agreed | +| 5 | E1 is removed | agreed | +| 6 | `mcpp.graph` holds order, levels and closure; the build-key fold keeps its computation and uses the module's order | clarified | +| 7 | Hard-linked placement with a copy fallback | agreed | +| 8 | No separate patch release: E1's removal and the planning cost land with this design (§14) | agreed | + + +## 14. The planning cost, fixed at its cause + +Revision 3 proposed two steps: a patch release that removed E1 and restored the +planning of 2026.9.28.2, then this design. Both steps address one cause, a +workspace planned as many roots, and the first step would have shipped the +cost of §1.1 (one full plan per member on every command) again. This design +removes the cause in one release: + +| Cost | Cause | What removes it | +|---|---|---| +| 2^n plans for a chain of n members (2026.9.28.3) | E1 plans a shared member as the root of a nested build, and the nested build meets E1's condition again | E1 is removed (§7.2); a member is a node of the one graph and is planned once, as every dependency is | +| one full plan per member on every command (2026.9.28.2 and 2026.9.28.3) | the workspace loop plans each member as a root, and the fast-path record belongs to a root | one plan per configuration (§4), and one fast-path record per configuration (§7.1) | +| a shared member compiled once per consumer | each consumer's graph holds its own copy of the member | one node per package per configuration (§2, principle 3) | +| a flag edit in one member rebuilds every member | the directory name hashes every package's flags | the directory name is the configuration key (§3) | + +What exists only for E1 is removed with it: the nested plans, the comparison +of build keys between a member's own build and a consumer's graph +(`packageKeys` and `packageKeyInputs` in the plan, which had no other reader), +the stage edges that compared content for units served from a member's +directory (`servedFromMember`), and the member lock. No depth guard and no +per-process memo is added: both would keep a mechanism the design deletes. +e2e 830, which asserted E1's behaviour, is removed; its criterion (a shared +member compiled once) is the first row of §11. + +Criteria, beside §11: the chain of §1.1 resolves the toolchain once per +configuration for `--workspace` and for `-p app` when something changed; with +nothing changed it prints no toolchain resolution and finishes in well under +a second. The CHANGELOG entry states the regression, its cause and the one +affected version (2026.9.28.3). + +## 15. The virtual root, read by read + +The planner treats its root, `packages[0]`, differently from a dependency at +about fifty sites. Each was classified (the inventory of 2026-09-29) as a value +of the workspace, which the virtual root holds, or a duty of the project being +developed, which every selected member now carries. The rules below state the +result; the sites follow them. + +**Selection.** Every command on a workspace selects members: `--workspace`, and +a virtual root without `-p`, select every member; `-p X`, and a command run in +X's directory, select X; a command at the root of a rooted workspace selects the +root package. The selection is planned by one mechanism in every case: a +virtual root whose `member` edges name the selected members. A member's node is +therefore the same node, with the same commands, whichever selection reaches +it. + +**Configurations.** The selected members are grouped by their effective +root-position values: the toolchain request, `[package] standard`, +`dialect_cxxflags`, `cxx_runtime`, `dependency_linkage`, the target and the +profile, each after workspace inheritance and the command line. Each group is +one plan with one virtual root, whose root-position values are the group's. +Two groups that resolve to one directory (two spellings of one toolchain) are +built one after the other in it. + +**The virtual root** has no sources, targets, hooks or build program. It holds +what is one value per plan: the toolchain, the target, the profile, the +root-position keys, the lock, `[build] device_extensions` and the rule-module +set (the union over the members), and the fast-path record. + +**A member edge** puts a member in the graph and links nothing into the root, +as an `artifacts` edge does (#711). A member is loaded as a `path` dependency, +so its flags, features, build program and target rows follow the path every +dependency already takes. What a member carries beyond a dependency: + +| Duty | Rule | +|---|---| +| targets | each target of a selected member is a link unit of the plan, in the member's product directory (§5); the link unit records its member | +| entry sources | resolved against the member's root, compiled with the member's own flags, as an `artifacts` program's entry is | +| link flags | a member's link unit carries the link flags of the member's closure (the member and what it reaches without an `artifacts` edge), in the order a root build would give them, and not those of other members | +| runtime files | the deploy set of a member's closure is placed in the member's product directory; a shared library of the closure is placed there too, by a link (§5.3) | +| tests | under `mcpp test`, the selected members' `[dev-dependencies]` are loaded as their own, non-transitive, and their test targets are the link units | +| `--features` | applies to each selected member that declares the feature; a feature no selected member declares is refused | +| `[xlings]` entries with `when = "dev"` | installed for a selected member, as for a root | +| `[hooks]` | the hooks of the selected members run around the one build, in member order | +| build program | runs as a dependency's does, with the member's own artifacts directory (`/target/.build-mcpp`, unchanged) and the resolved-graph document a root's program receives | +| graph-wide keys (`cxx_runtime`, `linkage`, `dialect_cxxflags`) | read from the virtual root; a member's value is checked for contradiction, as a dependency's is | + +**The lock** is `/mcpp.lock`, which for a rooted workspace is +the file it already has. `--workspace` writes the complete record; `-p X` +updates the entries of X's closure and keeps the others. Where the workspace +lock is absent, a selected member's own lock supplies the anchors (the git +commits and identities), so the first build after the upgrade does not move a +git dependency. + +**The fast path.** A record is written per selection and configuration. The +header of `build.ninja` names the selection and the requested features, since +neither moves the directory any longer, and the fast path compares it; the +freshness walk covers the selected members' roots and their path dependencies +instead of the whole workspace tree. + +**Commands.** `run`, `test` and `pack` with `-p X` take X's link units by their +member; `pack` reads X's manifest for the package's identity and `[pack]`. + +## 16. Tasks and their dependencies + +All engine work is one pull request in `mcpp-community/mcpp`, released as one +version. The other repositories change only where the release requires it. + +| # | Task | Repository | Depends on | +|---|---|---|---| +| T1 | `modules/graph` (`mcpp.graph`) with unit tests; migrate module order, unit order, host-module order, dependency closure and the build-key fold | mcpp | none | +| T2 | remove E1 (nested plans, member stage edges, member lock, `packageKeys`, e2e 830) | mcpp | none | +| T3 | hard-linked placement of shared libraries, with unit tests | mcpp | none | +| T4 | directory name = configuration key (§3) | mcpp | none | +| T5 | selection and configuration groups; the virtual root and member edges | mcpp | T2 | +| T6 | member duties of §15 (targets, product directories, link flags, runtime files, tests, features, hooks, build programs, lock) | mcpp | T5, T3 | +| T7 | commands: `build`, `run`, `test`, `pack`, `clean --stale`, compile database per configuration, output with one header per configuration | mcpp | T6 | +| T8 | scheduler: groups built concurrently under one `-j` (static division) | mcpp | T5 | +| T9 | fast path per selection and configuration | mcpp | T4, T6 | +| T10 | e2e criteria of §11 and §14; documentation (docs/07 and docs/zh/07, docs/12, the SPECs that name member output paths), CHANGELOG, version | mcpp | T1-T9 | +| T11 | release; GitCode assets mirrored locally as they appear; xim-pkgindex version entry | mcpp, xim-pkgindex | T10 | +| T12 | the package index: CI pin and `latest_mcpp` if required; full sweep | mcpp-index | T11 | +| T13 | the release canaries (xlings, mcppls), both rooted workspaces, and the plugins' CI | xlings, mcppls, mcpp-plugins | T11 | +| T14 | sandbox verification with the CN mirror; issues commented and closed | all | T12, T13 | +| T15 | acceptance on the validation project's pull request: its Windows CI against 1828 s, the core library compiled once, the Release layout | the validation project | T14 | diff --git a/modules/manifest/src/dep_spec.cppm b/modules/manifest/src/dep_spec.cppm index 6a768ccc..cc72fc57 100644 --- a/modules/manifest/src/dep_spec.cppm +++ b/modules/manifest/src/dep_spec.cppm @@ -109,6 +109,11 @@ struct DependencySpec { // are private-only.) bool reexport = false; bool defaultFeatures = true; // consumer opt-out: `default-features = false` + // A member edge (workspace design 2026-09-29 §15): the edge from a + // workspace plan's root to a selected member. It puts the member in the + // graph and links nothing into the root, as an `artifacts` edge does. + // Written by the engine only; no manifest spells it. + bool workspaceMember = false; // suppresses the dep's own [features].default seed // (Cargo parity). Explicit `features = [...]` still apply. // Canonical dependency parsing writes exactly one coordinate. Kept as a diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 7b6efd8e..b89a8b94 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -67,6 +67,11 @@ struct Package { // author asked for it" outside author-owned manifests — see the scope gate // in prepare.cppm. bool standardDeclared = false; + // The root of a workspace plan (workspace design 2026-09-29 §15): a + // manifest the engine synthesises, never parsed, that holds the values + // one plan shares and depends on the selected members. It has no sources, + // targets, hooks or build program of its own. + bool virtualRoot = false; std::string description; std::string license; std::vector authors; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 3289f554..5a7a2d92 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -2283,15 +2283,13 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { auto obj = escape_ninja_path(cu.object); append(std::format("build {} : stage_file {}\n", obj, escape_ninja_path(cu.cachedObject))); - // A member's own build rewrites its outputs in place, and an - // object of equal size is no evidence of equal content: content. - if (!cu.servedFromMember) append(" verify = --verify size\n"); + append(" verify = --verify size\n"); staged.push_back(obj); if (!cu.providesModule.empty() && !cu.cachedBmi.empty()) { auto bmi = bmi_path(cu.providesModule); append(std::format("build {} : stage_file {}\n", bmi, escape_ninja_path(cu.cachedBmi))); - if (!cu.servedFromMember) append(" verify = --verify size\n"); + append(" verify = --verify size\n"); staged.push_back(bmi); } } diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 6536aacf..7231121c 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -71,10 +71,6 @@ struct CompileUnit { bool servedFromCache = false; std::filesystem::path cachedObject; // absolute, inside the cache std::filesystem::path cachedBmi; // absolute; empty if no module - // #734 E1: served from a workspace member's own build directory rather - // than from an immutable cache entry. Its bytes change under an unchanged - // name, so its stage edges compare content, not size. - bool servedFromMember = false; // mcpp#344: this object's address INSIDE a global-cache entry — relative to // `/obj/`, and a pure function of the owning package (its source's // path relative to its own package root). Distinct from `object`, which is @@ -388,12 +384,6 @@ struct BuildPlan { std::vector rcFlags; // -I / -D, target-shaped std::vector compileUnits; // topologically sorted - // Each package's build key (`cache_key::key_hex`), indexed as the graph's - // packages, [0] the root. Set in the global cache mode. #734 E1 compares a - // workspace member's key in a consumer's graph with the member's key as the - // root of its own build: equal keys are equal compile commands. - std::vector packageKeys; - std::vector packageKeyInputs; // each key's inputs, as JSON text std::vector linkUnits; // Build-graph nodes declared by build programs (`mcpp:action=`). Paths are // absolute and engine variables already substituted by the time they get diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 8cbc3a35..6b6242b5 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -606,6 +606,11 @@ export struct BuildOverrides { std::string accel; bool force_static = false; // --static (or implied by musl target) std::string package_filter; // -p : only build this workspace member + // The workspace members this plan builds, as written in `[workspace] + // members` ("." is a rooted workspace's own package), all of one + // configuration group (workspace design 2026-09-29 §15). Empty: the member + // is selected from `package_filter` or the command's directory. + std::vector workspace_members; // --profile . Empty = fall through to `[build] default-profile`, then // to `profile_fallback` below, whose own default is "dev". The comment here // said "release" for as long as `mcpp build --help` did, and neither had diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 575f21ae..332a0708 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -1862,6 +1862,9 @@ static std::expected step6_dependency_build_programs(PrepareS // A package of programs runs its build program in its own tool // sub-build, where its sources are compiled (#649 E6). if (!state.compilesHere(i)) continue; + // A workspace member's program runs where a root's does + // (target_side.cpp, step9_member_build_programs). + if (state.isWorkspaceMemberPackage(i)) continue; std::error_code bpEc; if (!std::filesystem::exists(pkg.root / "build.mcpp", bpEc) && pkg.manifest.buildConfig.ruleModules.empty()) continue; @@ -2033,8 +2036,11 @@ static std::expected step6_dependency_build_programs(PrepareS // A CLAIM THAT ONLY EVER TIGHTENS. if (bcDep.runExclusive && !exclusiveBefore) state.m->buildConfig.runExclusive = true; - state.m->buildConfig.ldflags.insert(state.m->buildConfig.ldflags.end(), - bcDep.ldflags.begin() + ldN, bcDep.ldflags.end()); + // A workspace plan reads each package's own flags when it links + // a member's closure; nothing is pooled in its root (§15). + if (!state.workspacePlan()) + state.m->buildConfig.ldflags.insert(state.m->buildConfig.ldflags.end(), + bcDep.ldflags.begin() + ldN, bcDep.ldflags.end()); } // apply() may have added interface defines to packages' publicUsage diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 1f271c91..df6b277e 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -367,10 +367,14 @@ static std::vector propagateLinkFlags( // exactly that word, so the consumer's renderer reads the dependency's // flags with the same reading its own flags receive, and an element // that packs several tokens is several words on both sides. + // + // A workspace plan links each member's closure with that closure's own + // flags (workspace design 2026-09-29 §15), read from each package when + // the plan is made, so nothing is pooled in the virtual root. std::vector added; for (auto const& word : mcpp::manifest::flag_words(depManifest.buildConfig.ldflags)) { auto normalized = mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word)); - state.m->buildConfig.ldflags.push_back(normalized); + if (!state.workspacePlan()) state.m->buildConfig.ldflags.push_back(normalized); added.push_back(std::move(normalized)); } return added; @@ -1762,8 +1766,23 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { // application `z.o`), compiled its sources in the consumer's build, and // made a tool that depends on the package declaring it a cycle of the // consumer's graph although the two builds never meet. + // A selected workspace member is built in this plan whatever its + // targets are (workspace design 2026-09-29 §15). + const auto selectedMember = state.selectedMemberAt(ctx.dep_root); const bool depProgramOnly = state.isProgramOnlyPackage(*ctx.dep_manifest) - && spec.artifacts.empty(); + && spec.artifacts.empty() && !selectedMember; + // A root receives the profile's own compile flags; in a workspace plan + // each selected member does, as it did when it was the root. The + // tests `mcpp test` discovered are the selected member's targets. + if (selectedMember && state.selectedMembers.size() == 1) + for (auto const& t : state.extraTargets) ctx.dep_manifest->targets.push_back(t); + if (selectedMember) { + auto& mbc = ctx.dep_manifest->buildConfig; + mbc.cflags.insert(mbc.cflags.end(), state.profileCflags.begin(), + state.profileCflags.end()); + mbc.cxxflags.insert(mbc.cxxflags.end(), state.profileCxxflags.begin(), + state.profileCxxflags.end()); + } auto linkFlagsAdded = depProgramOnly ? std::vector{} : propagateLinkFlags(state, ctx.dep_root, *ctx.dep_manifest); @@ -1786,6 +1805,7 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { const auto depPackageIndex = state.packages.size(); auto depPackage = makePackageRoot(state, ctx.dep_root, *state.dep_manifests.back()); if (!depPackage) return std::unexpected(depPackage.error()); + if (selectedMember) depPackage->memberProducts = state.selectedMembers.at(*selectedMember); state.packages.push_back(std::move(*depPackage)); recordDependencyEdge(state, item.consumerDepIndex, depPackageIndex, spec, item.buildOnly, name); @@ -1835,13 +1855,38 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { if (auto fe = validateForwards(state, *state.dep_manifests.back(), depActive, state.dep_manifests.back()->package.name); !fe) return std::unexpected(fe.error()); + // `--features /` for a selected member: a forward + // of that member, applied to its edges as the root's are. + auto injectMemberCliForwards = [&](const std::string& childKey, + mcpp::manifest::DependencySpec& childSpec) { + if (!selectedMember) return; + auto fw = state.memberCliForwards.find(*selectedMember); + if (fw == state.memberCliForwards.end()) return; + for (auto const& [depKey, depFeat] : fw->second) + if (depKey == childKey + && std::ranges::find(childSpec.features, depFeat) == childSpec.features.end()) + childSpec.features.push_back(depFeat); + }; for (auto& [child_name, child_spec] : state.dep_manifests.back()->dependencies) { auto childReq = child_spec; injectForwards(*state.dep_manifests.back(), depActive, child_name, childReq); + injectMemberCliForwards(child_name, childReq); state.worklist.push_back({child_name, childReq, thisDepLabel, childReq.version, selfIdx, ctx.dep_root, item.devOnly, item.buildOnly}); } + // A selected member's `[dev-dependencies]` under `mcpp test`, as a + // root's: its own, and never walked into its dependencies' tests. + if (selectedMember && state.includeDevDeps) { + for (auto& [child_name, child_spec] : state.dep_manifests.back()->devDependencies) { + auto childReq = child_spec; + injectForwards(*state.dep_manifests.back(), depActive, child_name, childReq); + injectMemberCliForwards(child_name, childReq); + state.worklist.push_back({child_name, childReq, thisDepLabel + " (dev-dep)", + childReq.version, selfIdx, ctx.dep_root, + /*devOnly=*/true, item.buildOnly}); + } + } // A dependency's own `[build-dependencies]` — the only channel through // which a package can speak about what IT needs at build time. Both // live channels (`tools`, `host-module`) are written by the CONSUMER @@ -1856,6 +1901,7 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { state.dep_manifests.back()->buildDependencies) { auto childReq = child_spec; injectForwards(*state.dep_manifests.back(), depActive, child_name, childReq); + injectMemberCliForwards(child_name, childReq); state.worklist.push_back({child_name, childReq, thisDepLabel + " (build-dep)", childReq.version, selfIdx, ctx.dep_root, @@ -1973,8 +2019,18 @@ static void step4b_define_lookup_closures(PrepareState& state) { }; // Compiled in this plan: not a package of programs, or one whose programs // this plan ships. + state.isWorkspaceMemberPackage = [&](std::size_t i) { + if (i == 0 || i >= state.packages.size() || !state.workspacePlan()) return false; + if (state.packages[i].memberProducts) return true; + const auto& r = state.packages[i].root; + if (r.lexically_normal() == state.runtimeWorkspaceRoot.lexically_normal()) return true; + return !workspace_member_of(state, r).empty(); + }; + // A selected workspace member is compiled here whatever its targets are + // (workspace design 2026-09-29 §15). state.compilesHere = [&](std::size_t i) { - return i == 0 || !state.isProgramOnlyPackage(state.packages[i].manifest) || state.isArtifactPackage(i); + return i == 0 || !state.isProgramOnlyPackage(state.packages[i].manifest) + || state.isArtifactPackage(i) || state.packages[i].memberProducts.has_value(); }; diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index 08a5532e..d8a1d569 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -109,7 +109,8 @@ static void step4a_define_split_and_identity_closures(PrepareState& state) { const auto feats = i < state.activeFeaturesByPackage.size() ? state.activeFeaturesByPackage[i] : std::vector{}; for (auto const& spec : applicable_xlings_addresses( - man, feats, state.toolPurpose, /*isRoot=*/i == 0)) + man, feats, state.toolPurpose, /*isRoot=*/i == 0 + || state.packages[i].memberProducts.has_value())) claims.push_back({spec, describe(i), i == 0 ? 0 : 1}); } auto unified = addrset::unify(claims); diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index d624670e..a5857989 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -47,16 +47,100 @@ import mcpp.project; namespace mcpp::build { +// The plan's root for a selection of workspace members (workspace design +// 2026-09-29 §15): a virtual root that holds the values the plan shares, and a +// member edge to each selected member, which puts the member in the graph and +// links nothing into the root. A rooted workspace's own package is a member +// like any other ("."), so it is the same node, with the same commands, in +// every selection that reaches it. +static std::expected +select_workspace_members(PrepareState& state, const std::filesystem::path& wsRoot, + mcpp::manifest::Manifest ws, + const std::vector& selection) { + const auto all = mcpp::project::workspace_members(ws, wsRoot); + { + auto first = mcpp::project::load_member_manifest(ws, wsRoot, selection.front()); + if (!first) return std::unexpected(first.error()); + state.m = mcpp::project::virtual_workspace_root(ws, *first, wsRoot); + } + state.runtimeWorkspaceRoot = wsRoot; + state.wsManifest = std::move(ws); + state.root = wsRoot; + state.selectedMemberPaths = selection; + + // `--features` names features of the selected members: each member takes + // the tokens it declares, and a token no selected member declares is + // refused. + std::vector tokens; + for (auto const& t : mcpp::build::feature_request_tokens(state.overrides.features)) + tokens.push_back(t); + std::set claimed; + + for (auto const& mp : selection) { + const auto dir = (wsRoot / mp).lexically_normal(); + std::error_code canonEc; + auto canonical = std::filesystem::weakly_canonical(dir, canonEc); + if (canonEc) canonical = dir; + state.selectedMembers[canonical] = mcpp::project::product_directory_name(all, mp); + auto member = mcpp::project::load_member_manifest(*state.wsManifest, wsRoot, mp); + if (!member) return std::unexpected(member.error()); + std::vector requested; + for (auto const& t : tokens) { + if (auto fwd = mcpp::pm::split_feature_forward_token(t)) { + // `/`: a forward of each selected member + // that declares the dependency key, applied to its edges. + const bool declares = member->dependencies.contains(fwd->first) + || member->devDependencies.contains(fwd->first) + || member->buildDependencies.contains(fwd->first); + if (declares) { + state.memberCliForwards[canonical].push_back(*fwd); + claimed.insert(t); + } + continue; + } + if (member->featuresMap.contains(t) || t == "default") { + requested.push_back(t); + claimed.insert(t); + } + } + mcpp::manifest::DependencySpec spec; + spec.path = dir.string(); + spec.namespace_ = member->package.namespace_; + spec.shortName = member->package.name; + spec.workspaceMember = true; + spec.features = std::move(requested); + spec.declaredIn = "workspace"; + const auto key = member->package.namespace_.empty() + ? member->package.name + : member->package.namespace_ + "." + member->package.name; + state.m->dependencies[key] = std::move(spec); + } + for (auto const& t : tokens) { + if (claimed.contains(t)) continue; + return std::unexpected(std::format( + "--features requests '{}', which no selected workspace member " + "declares", t)); + } + state.overrides.features.clear(); + + std::string shown; + for (auto const& [key, spec] : state.m->dependencies) + shown += (shown.empty() ? "" : ", ") + spec.shortName; + mcpp::ui::status("Workspace", selection.size() == 1 + ? std::format("building member '{}'", shown) + : std::format("building {} members: {}", selection.size(), shown)); + return {}; +} + // STEP FUNCTION (mcpp#722 / T6 follow-on): phase0's own "Workspace -// handling" section, extracted verbatim. +// handling" section. static std::expected step0_workspace_handling(PrepareState& state) { - // ─── Workspace handling ──────────────────────────────────────────── - // If the manifest has [workspace] and is a virtual workspace (no [package]), - // or if -p filter is set, switch to the target member's manifest. + // A command at a workspace's root selects members: the command's own list + // (`--workspace`, one group of it per plan), `-p`, the first member with a + // program at a virtual root, or the workspace's own package. if (state.m->workspace.present) { - std::string targetMember; - - if (!state.overrides.package_filter.empty()) { + std::vector selection = state.overrides.workspace_members; + if (selection.empty() && !state.overrides.package_filter.empty()) { // `-p `: the package identity first, the member's // directory as a fallback -- one resolver shared with every // other `-p`/`--package` command @@ -64,95 +148,81 @@ static std::expected step0_workspace_handling(PrepareState& s auto matched = mcpp::project::resolve_member_dir( *state.m, *state.root, state.overrides.package_filter); if (!matched) return std::unexpected(matched.error()); - targetMember = matched->lexically_relative(*state.root).generic_string(); - } else if (state.m->package.name.empty()) { + auto rel = matched->empty() ? std::string(".") + : matched->lexically_normal().lexically_relative( + state.root->lexically_normal()).generic_string(); + if (rel.empty()) rel = "."; + selection.push_back(rel); + } else if (selection.empty() && state.m->package.name.empty()) { // Virtual workspace: find a member with a program target ("is // this the program", #622 A3's `is_program()`, so a member whose // only target is `kind = "app"` is picked exactly as one whose // target is `bin` is), or use last member. for (auto& mp : state.m->workspace.members) { - auto memberDir = *state.root / mp; - auto mm = mcpp::manifest::load(memberDir / "mcpp.toml", + auto mm = mcpp::manifest::load(*state.root / mp / "mcpp.toml", {.insideWorkspace = true}); if (!mm) continue; - for (auto& t : mm->targets) { - if (t.is_program()) { - targetMember = mp; - break; - } + if (std::ranges::any_of(mm->targets, [](auto const& t) { return t.is_program(); })) { + selection.push_back(mp); + break; } - if (!targetMember.empty()) break; - } - if (targetMember.empty() && !state.m->workspace.members.empty()) { - targetMember = state.m->workspace.members.back(); } + if (selection.empty() && !state.m->workspace.members.empty()) + selection.push_back(state.m->workspace.members.back()); + } else if (selection.empty()) { + selection.push_back("."); } - // else: rooted workspace with [package] — build root normally. Its own - // `x.workspace = true` entries name its own [workspace.dependencies]. - // The workspace context is set here too (#725): it is a property of - // where the manifest lives, not of the branch that was taken, so a - // member this package reaches through its OWN `path` dependencies - // (`depIsMember`, graph.cpp) is recognised as a member and receives - // `[workspace.package]`, `[workspace.build]` and `x.workspace = true` - // the same way a sibling's `path` dependency does. - else if (state.m->workspace.present) { - state.runtimeWorkspaceRoot = *state.root; - state.wsManifest = *state.m; - mcpp::project::merge_workspace_deps(*state.m, *state.m, *state.root); - } - - if (!targetMember.empty()) { - auto memberDir = *state.root / targetMember; - if (!std::filesystem::exists(memberDir / "mcpp.toml")) { - return std::unexpected(std::format( - "workspace member '{}' has no mcpp.toml", targetMember)); - } - state.runtimeWorkspaceRoot = *state.root; - state.wsManifest = std::move(*state.m); // preserve workspace manifest - auto memberManifest = mcpp::manifest::load(memberDir / "mcpp.toml", - {.insideWorkspace = true}); - if (!memberManifest) return std::unexpected(std::format( - "workspace member '{}': {}", targetMember, - memberManifest.error().format())); - state.m = std::move(*memberManifest); - - // ONE call, not a hand-copied list. `*root` is still the WORKSPACE - // root here (the `root = memberDir` reassignment below has not - // happened yet), which is what a relative `[indices].path` or - // `[workspace.dependencies] path` was written against (#224). - mcpp::project::inherit_workspace_config(*state.m, *state.wsManifest, *state.root); - if (auto bad = mcpp::project::workspace_inheritance_error(*state.m, memberDir)) - return std::unexpected(*bad); - - mcpp::ui::status("Workspace", std::format("building member '{}'", targetMember)); - state.root = memberDir; + if (selection.empty()) + return std::unexpected(std::string("the workspace lists no members")); + const auto wsRoot = *state.root; + auto ws = std::move(*state.m); + return select_workspace_members(state, wsRoot, std::move(ws), selection); + } + // Inside a member: the command selects that member, planned from the + // workspace's root like every other selection (§15), so its products and + // its build directory are the ones `-p` and `--workspace` use. + if (state.effective && state.effective->member && state.effective->workspace + && !state.overrides.preloaded_manifest) { + const auto wsRoot = state.effective->workspaceRoot; + auto ws = std::move(*state.effective->workspace); + std::vector selection = state.overrides.workspace_members; + if (selection.empty() && !state.overrides.package_filter.empty()) { + auto matched = mcpp::project::resolve_member_dir( + ws, wsRoot, state.overrides.package_filter); + if (!matched) return std::unexpected(matched.error()); + auto rel = matched->empty() ? std::string(".") + : matched->lexically_normal().lexically_relative( + wsRoot.lexically_normal()).generic_string(); + selection.push_back(rel.empty() ? std::string(".") : rel); + } else if (selection.empty()) { + auto rel = state.root->lexically_normal() + .lexically_relative(wsRoot.lexically_normal()).generic_string(); + // The member as `[workspace] members` spells it, so the + // selection, the product directory and the header agree. + for (auto const& mp : ws.workspace.members) + if (std::filesystem::path(mp).lexically_normal() + == std::filesystem::path(rel).lexically_normal()) { rel = mp; break; } + selection.push_back(rel); } - } else { - // Not at workspace root: inside a member, the loader above has - // already inherited (#224 anchoring included). Only the workspace is - // recorded here, for the membership test of this member's own `path` - // dependencies. - if (state.effective && state.effective->member) { - state.runtimeWorkspaceRoot = state.effective->workspaceRoot; - state.wsManifest = std::move(*state.effective->workspace); - } else if (state.overrides.preloaded_manifest) { - auto wsRoot = mcpp::project::find_workspace_root(*state.root); - if (!wsRoot.empty()) { - if (auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); - wsm && wsm->workspace.present) { - state.runtimeWorkspaceRoot = wsRoot; - state.wsManifest = std::move(*wsm); - } + return select_workspace_members(state, wsRoot, std::move(ws), selection); + } + if (state.overrides.preloaded_manifest) { + auto wsRoot = mcpp::project::find_workspace_root(*state.root); + if (!wsRoot.empty()) { + if (auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); + wsm && wsm->workspace.present) { + state.runtimeWorkspaceRoot = wsRoot; + state.wsManifest = std::move(*wsm); } - // A preloaded manifest was inherited at its dependency load site, - // which gives a member everything but the root-position keys. This - // build IS rooted at it (a host-tool sub-build), so it takes those - // too, from the workspace that lists it (#710). - if (state.wsManifest - && mcpp::project::is_workspace_member(*state.wsManifest, state.runtimeWorkspaceRoot, *state.root)) - mcpp::project::inherit_workspace_root_position( - *state.m, *state.wsManifest, state.runtimeWorkspaceRoot); } + // A preloaded manifest was inherited at its dependency load site, + // which gives a member everything but the root-position keys. This + // build IS rooted at it (a host-tool sub-build), so it takes those + // too, from the workspace that lists it (#710). + if (state.wsManifest + && mcpp::project::is_workspace_member(*state.wsManifest, state.runtimeWorkspaceRoot, *state.root)) + mcpp::project::inherit_workspace_root_position( + *state.m, *state.wsManifest, state.runtimeWorkspaceRoot); } return {}; } @@ -418,8 +488,11 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta } } - // Inject synthetic targets (e.g. test binaries from `mcpp test`). - for (auto& t : state.extraTargets) state.m->targets.push_back(t); + // Inject synthetic targets (e.g. test binaries from `mcpp test`). In a + // workspace plan they are the selected member's tests, and that member + // takes them when it is loaded (graph.cpp). + if (!state.workspacePlan()) + for (auto& t : state.extraTargets) state.m->targets.push_back(t); // #540: a cfg() predicate mcpp cannot evaluate must say so. // diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index a2247317..3b12fbe3 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -235,7 +235,8 @@ static std::expected step13_runner_and_xlings(PrepareState& s const auto feats = i < state.activeFeaturesByPackage.size() ? state.activeFeaturesByPackage[i] : std::vector{}; for (auto const& spec : applicable_xlings_addresses( - man, feats, ToolPurpose::Run, /*isRoot=*/i == 0)) + man, feats, ToolPurpose::Run, /*isRoot=*/i == 0 + || state.packages[i].memberProducts.has_value())) if (std::ranges::find(xlingsSpecs, spec) == xlingsSpecs.end()) { ctx.runTierPending = true; break; } } @@ -1629,7 +1630,7 @@ static std::expected step13_windows_resources(PrepareState& s // The member path (relative to the workspace root) of a package root, when the // root is a member of the workspace this build runs in; empty otherwise. -// Shared by E1 (members built once) and W3 (a member's non-public modules). +// Read by W3 (a member's non-public modules). std::string workspace_member_of(const PrepareState& state, const std::filesystem::path& root) { if (!state.wsManifest || state.runtimeWorkspaceRoot.empty()) return {}; const auto rel = root.lexically_normal() @@ -1645,185 +1646,6 @@ std::string workspace_member_of(const PrepareState& state, const std::filesystem return {}; } -// #734 E1, split out of step13_dependency_cache: the workspace members this -// graph reaches by `path` are built in their own directories and served to it -// through stage edges. `pkgKeys` and `pkgInputs` are this graph's build keys -// and their inputs, index-aligned with `state.packages`. -static std::expected -step13_serve_workspace_members(PrepareState& state, BuildContext& ctx, - const std::vector& pkgKeys, - const std::vector& pkgInputs) { - // ── #734 E1: a workspace member used as a path dependency ───────── - // - // Built once, in its own directory, as the root of its own build, and - // taken from there by every member that consumes it. The member's own - // ninja decides what is stale, so an input outside the member's root - // (a header under `../3rdParty`) counts as it does for the member's - // own build; nothing here stamps files. The member's objects and - // module interfaces reach this graph through the stage edges the - // global cache uses. Equal build keys are the admission rule: the key - // holds everything that decides a unit's compile command, so a member - // whose key differs here (other features, another profile) is compiled - // in this graph as before. A host-tool sub-build and a planning-only - // command (`mcpp emit build-database`) never build a member. - if (state.overrides.tool_depth == 0 && !state.overrides.plan_only - && state.wsManifest && !state.runtimeWorkspaceRoot.empty()) { - auto qualified = [](const mcpp::manifest::Manifest& mm) { - return mm.package.namespace_.empty() ? mm.package.name - : mm.package.namespace_ + "." + mm.package.name; - }; - auto bmiT = mcpp::toolchain::bmi_traits(*state.tc); - for (std::size_t i = 1; i < state.packages.size(); ++i) { - const auto* depIdent = i - 1 < state.dep_cache_identities.size() - ? &state.dep_cache_identities[i - 1] : nullptr; - if (!depIdent || depIdent->sourceKind != "path") continue; - const auto& pkgRoot = state.packages[i]; - const auto member = workspace_member_of(state, pkgRoot.root); - if (member.empty()) continue; - - BuildOverrides sub; - sub.project_root = state.runtimeWorkspaceRoot; - sub.package_filter = member; - sub.target_triple = state.overrides.target_triple; - sub.accel = state.overrides.accel; - sub.force_static = state.overrides.force_static; - sub.profile = state.overrides.profile; - sub.profile_fallback = state.overrides.profile_fallback; - sub.capabilities = state.overrides.capabilities; - sub.toolchain = state.overrides.toolchain; - sub.cache_mode = state.overrides.cache_mode; - if (i < state.activeFeaturesByPackage.size()) - for (auto const& f : state.activeFeaturesByPackage[i]) { - if (!sub.features.empty()) sub.features += ","; - sub.features += f; - } - auto subCtx = prepare_build(/*print_fingerprint=*/false, - /*includeDevDeps=*/false, /*extraTargets=*/{}, sub); - const auto who = qualified(pkgRoot.manifest); - if (!subCtx) { - mcpp::log::verbose("workspace-member", std::format( - "{} is compiled in this graph: its own build could not be " - "planned: {}", who, subCtx.error())); - continue; - } - // The admission compares the key's INPUTS, less `package.index`: - // that field names where a package came from (the namespace a - // dependency is reached under; empty for a root), not how it - // compiles. Every other input must be equal. - auto compile_inputs = [](nlohmann::json j) { - if (j.is_object() && j.contains("package") && j["package"].is_object()) - j["package"].erase("index"); - return j; - }; - const bool sameCompile = !subCtx->plan.packageKeyInputs.empty() - && compile_inputs(nlohmann::json::parse(subCtx->plan.packageKeyInputs[0], nullptr, false)) - == compile_inputs(pkgInputs[i]); - if (!sameCompile) { - // Name the inputs that differ: a mismatch is either a - // real difference (another feature set) or a key input - // that depends on the position, and only the field names - // tell which. - std::string fields; - if (!subCtx->plan.packageKeyInputs.empty()) { - auto own = nlohmann::json::parse(subCtx->plan.packageKeyInputs[0], nullptr, false); - auto here = pkgInputs[i]; - if (own.is_object() && here.is_object()) - for (auto it = here.begin(); it != here.end(); ++it) { - if (own.contains(it.key()) && own[it.key()] == it.value()) continue; - if (it.value().is_object() && own.contains(it.key()) - && own[it.key()].is_object()) { - auto const& o = own[it.key()]; - for (auto jt = it.value().begin(); jt != it.value().end(); ++jt) - if (!o.contains(jt.key()) || o[jt.key()] != jt.value()) { - if (!fields.empty()) fields += ", "; - fields += std::format("{}.{} (here {}, own {})", it.key(), - jt.key(), jt.value().dump(), - o.contains(jt.key()) ? o[jt.key()].dump() : "absent"); - } - continue; - } - if (!fields.empty()) fields += ", "; - fields += it.key(); - } - } - mcpp::log::verbose("workspace-member", std::format( - "{} is compiled in this graph: its build key here ({}) differs " - "from its own ({}); differing inputs: {}", who, pkgKeys[i], - subCtx->plan.packageKeys.empty() ? std::string("none") - : subCtx->plan.packageKeys[0], - fields.empty() ? std::string("(unknown)") : fields)); - continue; - } - // Every unit of the member in this graph must have its - // counterpart in the member's own build, or none is taken: - // a half-served package is the mixed state the global cache - // refuses for the same reason. - std::map own; - for (auto const& scu : subCtx->plan.compileUnits) - own.emplace(scu.source.lexically_normal(), &scu); - std::vector> pairs; - bool complete = true; - for (std::size_t u = 0; u < ctx.plan.compileUnits.size(); ++u) { - auto const& cu = ctx.plan.compileUnits[u]; - if (cu.packageName != who) continue; - auto it = own.find(cu.source.lexically_normal()); - if (it == own.end()) { complete = false; break; } - pairs.push_back({u, it->second}); - } - if (!complete || pairs.empty()) { - mcpp::log::verbose("workspace-member", std::format( - "{} is compiled in this graph: its units here and in its own " - "build are not the same set", who)); - continue; - } - - mcpp::ui::status("Building", std::format( - "workspace member {} in its own directory, once for every " - "member that uses it", who)); - // Two consumers built at once (two `-p` commands) share this - // directory; one ninja runs in it at a time. - std::error_code lockEc; - std::filesystem::create_directories(subCtx->plan.outputDir, lockEc); - std::optional memberLock; - for (int waited = 0; - !(memberLock = mcpp::platform::fs::FileLock::try_acquire(subCtx->plan.outputDir)); - ++waited) { - if (waited == 0) - mcpp::ui::status("Waiting", std::format( - "for another build of workspace member {}", who)); - if (waited >= 3000) return std::unexpected(std::format( - "workspace member {}: another build has held {} for ten minutes", - who, subCtx->plan.outputDir.string())); - std::this_thread::sleep_for(std::chrono::milliseconds(200)); - } - auto be = mcpp::build::make_ninja_backend(); - mcpp::build::BuildOptions bopt; - auto br = be->build(subCtx->plan, bopt); - memberLock.reset(); - if (!br) return std::unexpected(std::format( - "building workspace member {} failed: {}\n{}", who, - br.error().message, br.error().diagnosticOutput)); - if (br->exitCode != 0) return std::unexpected(std::format( - "building workspace member {} failed (exit {})", who, br->exitCode)); - - const auto subOut = subCtx->plan.outputDir; - for (auto const& [u, scu] : pairs) { - auto& cu = ctx.plan.compileUnits[u]; - cu.servedFromCache = true; - cu.servedFromMember = true; - cu.cachedObject = subOut / scu->object; - if (!cu.providesModule.empty()) { - std::string bmi; - for (char c : cu.providesModule) bmi.push_back(c == ':' ? '-' : c); - bmi += std::string(bmiT.bmiExt); - cu.cachedBmi = subOut / std::string(bmiT.bmiDir) / bmi; - } - } - } - } - return {}; -} - static std::expected step13_dependency_cache(PrepareState& state, BuildContext& ctx) { // ─── Global dependency cache: per-package keys, hit → stage edges ── // @@ -1993,12 +1815,6 @@ static std::expected step13_dependency_cache(PrepareState& st for (std::size_t i = 0; i < state.packages.size(); ++i) (void)compute_key(compute_key, i); if (!keyCycleError.empty()) return std::unexpected(keyCycleError); - ctx.plan.packageKeys = pkgKeys; - ctx.plan.packageKeyInputs.clear(); - for (auto const& j : pkgInputs) ctx.plan.packageKeyInputs.push_back(j.dump()); - - if (auto served = step13_serve_workspace_members(state, ctx, pkgKeys, pkgInputs); !served) - return std::unexpected(served.error()); for (std::size_t i = 1; i < state.packages.size(); ++i) { // skip [0] = main const auto& pkgRoot = state.packages[i]; diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 75431c55..db397010 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -706,13 +706,14 @@ static void step11_compute_fingerprint(PrepareState& state) { // `macos_deployment_target` whenever THIS BUILD's resolved toolchain // targets macOS, whether mcpp itself is running on Linux, Windows or // macOS — see the discriminator comment on `min_platform_version` and - // on `canonical_compile_flags`. + // on `canonical_configuration_flags`. const bool fpTargetIsMacos = [&] { auto fpTt = mcpp::toolchain::triple::parse(state.tc->targetTriple); return fpTt && fpTt->os == "macos"; }(); - fpi.compileFlags = canonical_compile_flags(*state.m, fpTargetIsMacos) - + canonical_package_build_metadata(state.packages, fpTargetIsMacos); + // The configuration alone (workspace design 2026-09-29 §3): a package's own + // flags are attributes of its node and reach its commands, not this name. + fpi.compileFlags = canonical_configuration_flags(*state.m, fpTargetIsMacos); // [c-abi] REALISATION AND `__OPENKAL__` PARTICIPATE IN THE FINGERPRINT // (design 2026-09-18 §3.4, gap #4 of the design's own self-review). Two // builds whose C library declares `data-model = "lp64"` and `"llp64"` @@ -725,32 +726,10 @@ static void step11_compute_fingerprint(PrepareState& state) { if (state.tc->kernelAbiIsOpenkal) fpi.compileFlags += " openkal-kernel-abi"; for (auto& t : state.tc->cEnvTokens) fpi.compileFlags += " cenv:" + t; for (auto& t : state.tc->cEnvBuiltinsTokens) fpi.compileFlags += " cenv:" + t; - // A package opting OUT via `c-environment = "platform"` (§3.4) still - // changes what ITS OWN objects contain, relative to a graph where it - // did not opt out — so the opt-out is folded in too, named by the - // package rather than by its flags, since the flags it now keeps are - // simply the ones already covered above. - // - // GATED ON THE REALISATION ACTUALLY BEING ACTIVE (`cEnvTokens` or - // `cEnvBuiltinsTokens` non-empty) — NOT unconditional. `cEnvironment == - // "platform"` is true for every `mcpp:kernel-abi=` provider now - // (it is INFERRED, this same revision), in every graph that uses one, - // whether or not that graph's C library declares `[c-abi]` at all. An - // unconditional loop here folded `cenv-platform:` into the - // fingerprint of EVERY project using openkal-windows (say) even when - // nothing about the realised environment was active — moving every - // such project's output directory on upgrade for a string that - // describes an opt-out from a realisation that never ran. There is - // nothing to opt OUT of when there is nothing being realised, so the - // opt-out changes nothing about that package's own objects and must - // not move the fingerprint either — the same "declares nothing, byte - // identical" guarantee the rest of this block already gives, which this - // loop had broken on its own. - if (!state.tc->cEnvTokens.empty() || !state.tc->cEnvBuiltinsTokens.empty()) { - for (auto& pkg : state.packages) - if (pkg.manifest.cEnvironment == "platform") - fpi.compileFlags += " cenv-platform:" + pkg.manifest.package.name; - } + // A package opting OUT via `c-environment = "platform"` (§3.4) is an + // attribute of that package's node: it changes that package's commands, + // which ninja rebuilds, and not the configuration (workspace design + // 2026-09-29 §3). // The module-edge schedule changes the SHAPE of build.ninja, and the fast // path replays that file without a plan to compare against. Folding the // switch into the fingerprint puts a differently-scheduled build in a diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index d615cbbc..0f444874 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -290,6 +290,35 @@ struct PrepareState { std::optional effective; std::expected m = std::unexpected(std::string{}); std::optional wsManifest; // keep workspace manifest alive + // The members this plan builds (workspace design 2026-09-29 §15), by + // normalised directory, each with the directory below `bin/` its products + // are placed in. Empty outside a workspace plan. The members as written in + // `[workspace] members`, in selection order, beside it. + std::map selectedMembers; + std::vector selectedMemberPaths; + // `--features /` tokens, by the selected member that + // declares the dependency key: the forwards its edges receive. + std::map>> + memberCliForwards; + // The profile's own compile flags (`[profile.] cflags`/`cxxflags`), + // which a root receives; in a workspace plan every selected member does. + std::vector profileCflags, profileCxxflags; + bool workspacePlan() const { return !selectedMemberPaths.empty(); } + // A package of a workspace plan that is a member of the workspace, selected + // or reached as another member's dependency (the workspace's own package + // included): its build program runs where a root's does (§15). + std::function isWorkspaceMemberPackage; + // The selected member whose directory is `dir`, when there is one: the + // key into `selectedMembers` and `memberCliForwards` (weakly canonical, + // so a symbolic link in the path names the same member). + std::optional selectedMemberAt(const std::filesystem::path& dir) const { + if (selectedMembers.empty()) return std::nullopt; + std::error_code ec; + auto key = std::filesystem::weakly_canonical(dir, ec); + if (ec) key = dir.lexically_normal(); + if (selectedMembers.contains(key)) return key; + return std::nullopt; + } std::filesystem::path runtimeWorkspaceRoot; mcpp::xlings::runtime::RuntimeSelection runtimeSelection; std::filesystem::path workRoot; @@ -504,7 +533,7 @@ struct PrepareState { std::expected phase0_manifest_and_workspace(PrepareState& state); // plan.cpp: the member path of a package root within the workspace this build -// runs in, or empty (#734 E1, W3). +// runs in, or empty (W3). std::string workspace_member_of(const PrepareState& state, const std::filesystem::path& root); std::expected phase1_toolchain_spec_and_axes(PrepareState& state); std::expected phase2_define_toolchain_resolver(PrepareState& state); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index d156d7bc..dd5348f7 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -1242,9 +1242,9 @@ static std::expected step9_layer_conditional_config(PrepareSt // runtime over several C libraries — so patching only the root would leave // the one package this feature exists for unserved. // - // `*m` as well as the snapshots: `canonical_compile_flags(*m)` feeds the - // fingerprint, so a contribution reaching the snapshots and not the - // manifest would compile with flags the fingerprint does not describe. + // `*m` as well as the snapshots: the plan reads the root's flags from + // both, so a contribution reaching only one of them would reach some of + // the root's commands and not others. // MUTATING `pkg.manifest` IS NOT ENOUGH, AND THAT IS THE WHOLE // DIFFICULTY OF A LATE PRODUCER. `makePackageRoot` snapshots the manifest's // build inputs into `privateBuild` / `linkUsage`, and the compile and link @@ -1261,10 +1261,9 @@ static std::expected step9_layer_conditional_config(PrepareSt layerCtx.kernelAbi = state.resolvedTargetSide.kernelAbi.interfaceName; layerCtx.cAbi = state.resolvedTargetSide.cAbi.interfaceName; layerCtx.cxxAbi = state.resolvedTargetSide.cxx.interfaceName; - // The root manifest feeds `canonical_compile_flags`, and therefore the - // fingerprint: a contribution reaching the snapshots but not `*m` would - // compile with flags the fingerprint does not describe, and the next - // build would call that a cache hit. + // The root manifest and its snapshots are both read by the plan, so a + // contribution reaching the snapshots but not `*m` would reach some of + // the root's commands and not others. merge_layer_conditional_config(*state.m, layerCtx); for (auto& pkg : state.packages) { const auto mark = state.markDirectiveTail(pkg.manifest); @@ -1522,6 +1521,9 @@ static std::expected step9_root_build_program(PrepareState& s // and a guard that asked only about the file left the synthesis unreachable // -- the shaders went uncompiled and the refusal named the missing program // rather than the guard. Measured. + // A workspace plan's virtual root has no program; its members' programs + // run in `step9_member_build_programs`, below. + if (state.m->package.virtualRoot) return {}; if (std::filesystem::exists(*state.root / "build.mcpp") || !state.m->buildConfig.ruleModules.empty()) { auto host = state.host_tc_for_build_program(); @@ -1756,9 +1758,9 @@ static std::expected step9_root_build_program(PrepareState& s pkg0.manifest.modules.sources.insert( pkg0.manifest.modules.sources.end(), state.m->modules.sources.begin() + rmodN, state.m->modules.sources.end()); - // Fingerprint metadata (canonical_package_build_metadata folds - // packages[].manifest.buildConfig) — mirror the flag/include tails, - // as the old pre-snapshot ordering implicitly did. + // The snapshot's manifest is what later readers of the root's + // flags see -- mirror the flag/include tails, as the old + // pre-snapshot ordering implicitly did. pkg0.manifest.buildConfig.cflags.insert( pkg0.manifest.buildConfig.cflags.end(), bcRoot.cflags.begin() + static_cast(mark.cflags), @@ -1806,6 +1808,196 @@ static std::expected step9_root_build_program(PrepareState& s return {}; } +// The graph document a build program reads (#647 E1): every package the +// program's own package reaches, itself included, dependencies before the +// packages that request them (ties in discovery order), with `root` marking +// the package whose program reads it. For a root that is the whole graph. +static std::expected, std::string> +write_graph_document(PrepareState& state, std::size_t subject, + const std::filesystem::path& artifactsDir) { + std::vector reached(state.packages.size(), subject == 0); + reached[subject] = true; + for (bool grew = true; grew;) { + grew = false; + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex < reached.size() && reached[r.consumerPackageIndex] + && r.dependencyPackageIndex < reached.size() + && !reached[r.dependencyPackageIndex]) { + reached[r.dependencyPackageIndex] = true; + grew = true; + } + } + std::vector order; + std::vector placed(state.packages.size(), false); + std::size_t wanted = 0; + for (bool b : reached) wanted += b ? 1 : 0; + while (order.size() < wanted) { + std::size_t pick = state.packages.size(); + for (std::size_t i = 0; i < state.packages.size() && pick == state.packages.size(); ++i) { + if (placed[i] || !reached[i]) continue; + bool ready = true; + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i + && r.dependencyPackageIndex < state.packages.size() + && reached[r.dependencyPackageIndex] + && !placed[r.dependencyPackageIndex]) { ready = false; break; } + if (ready) pick = i; + } + // A cycle leaves nothing ready; its first member in discovery + // order is taken so the document is still complete. + if (pick == state.packages.size()) + for (std::size_t i = 0; i < state.packages.size(); ++i) + if (!placed[i] && reached[i]) { pick = i; break; } + placed[pick] = true; + order.push_back(pick); + } + nlohmann::json doc; + doc["kind"] = "mcpp.graph"; + doc["version"] = 1; + nlohmann::json list = nlohmann::json::array(); + for (auto i : order) { + auto entry = state.graph_package_entry(i, true); + entry["root"] = i == subject; + list.push_back(std::move(entry)); + } + doc["packages"] = std::move(list); + const auto text = doc.dump(2) + "\n"; + const auto graphPath = artifactsDir / "graph.json"; + std::error_code gec; + std::filesystem::create_directories(graphPath.parent_path(), gec); + const auto tmp = graphPath.string() + ".tmp"; + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << text; + } + std::filesystem::rename(tmp, graphPath, gec); + if (gec) + return std::unexpected(std::format( + "cannot write the graph document '{}': {}", + graphPath.string(), gec.message())); + return std::pair{graphPath, mcpp::toolchain::hash_string(text)}; +} + +// The build programs of a workspace plan's members (workspace design +// 2026-09-29 §15). A member is the project being developed, so its program +// runs where a root's does: after every dependency's program, with the link +// form of each dependency and the graph document of what it reaches, and with +// its own artifacts directory, `/target/.build-mcpp`, where it was +// when the member was the root of its own build. A workspace member reached as +// a dependency of another member runs here too, so its program sees the same +// environment in every selection. +static std::expected step9_member_build_programs(PrepareState& state) { + if (!state.m->package.virtualRoot) return {}; + for (std::size_t i = 1; i < state.packages.size(); ++i) { + if (!state.isWorkspaceMemberPackage(i)) continue; + if (!state.compilesHere(i)) continue; + auto& pkg = state.packages[i]; + std::error_code bpEc; + if (!std::filesystem::exists(pkg.root / "build.mcpp", bpEc) + && pkg.manifest.buildConfig.ruleModules.empty()) continue; + auto host = state.host_tc_for_build_program(); + if (!host) return std::unexpected(host.error()); + mcpp::build::BuildProgramEnv bpEnv; + bpEnv.targetTriple = state.resolvedTargetCanonical; + fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, + state.cfg_opt ? &*state.cfg_opt : nullptr); + bpEnv.toolsBin = state.projectSubosBin; + bpEnv.profile = state.effectiveProfile; + bpEnv.accel = state.resolvedAccel(); + fill_package_build_env(bpEnv, pkg.manifest); + bpEnv.packFormat = state.overrides.pack_format; + bpEnv.packStageDir = state.overrides.pack_stage_dir; + bpEnv.packStrip = state.overrides.pack_strip; + bpEnv.packDebugSymbolsDir = state.overrides.pack_debug_symbols_dir; + bpEnv.languageModules = pkg.manifest.language.modules; + bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; + if (auto dit = state.deviceSourcesByPackage.find(pkg.root.string()); + dit != state.deviceSourcesByPackage.end()) + bpEnv.deviceSources = dit->second; + bpEnv.artifactsDir = pkg.root / "target" / ".build-mcpp"; + if (i < state.activeFeaturesByPackage.size()) + bpEnv.features = state.activeFeaturesByPackage[i]; + { + std::map linkForms; + for (auto const& [idx, form] : state.dependencyLinkForms) + if (form.recorded) + linkForms.emplace(idx, std::string( + mcpp::build::linkage_form::to_string(form.answer.linkage))); + state.fillDepDirs(bpEnv, i, &linkForms); + } + state.fillXpkgDirs(bpEnv, pkg.manifest, i); + if (auto tit = state.toolEnvByConsumer.find(i); tit != state.toolEnvByConsumer.end()) + bpEnv.toolPaths = tit->second; + bpEnv.hostModules = state.hostModulesByConsumer.count(i) + ? state.hostModulesByConsumer.at(i) : decltype(bpEnv.hostModules){}; + bpEnv.dormantFeatures = state.dormantFeaturesByConsumer.count(i) + ? state.dormantFeaturesByConsumer.at(i) : decltype(bpEnv.dormantFeatures){}; + auto graph = write_graph_document(state, i, bpEnv.artifactsDir); + if (!graph) return std::unexpected(graph.error()); + bpEnv.graphFile = graph->first; + bpEnv.graphDigest = graph->second; + + auto& bc = pkg.manifest.buildConfig; + const auto mark = state.markDirectiveTail(pkg.manifest); + const auto actN = bc.actions.size(); + const auto runnerN = bc.runner.size(); + auto namedBefore = bc.namedRunners; + const bool exclusiveBefore = bc.runExclusive; + auto bp = mcpp::build::run_build_program( + pkg.manifest, pkg.root, host->first, host->second, + pkg.manifest.cppStandard, bpEnv); + if (!bp) { + if (!state.overrides.plan_only) + return std::unexpected(std::format( + "workspace member '{}': {}", pkg.manifest.package.name, bp.error())); + state.planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + std::format("workspace member '{}': {}", pkg.manifest.package.name, bp.error()), + mcpp::wire::Severity::Error, (pkg.root / "build.mcpp").string()}); + state.programFailedPackages.insert(pkg.root.string()); + continue; + } + state.foldDirectiveTailIntoPrivateBuild(pkg, pkg.manifest, mark); + state.adoptActionOutputs(pkg.manifest, pkg.root, actN); + // How the artifact is executed reaches the plan's root, where `mcpp + // run` reads it, by the rule the dependencies' runners follow: one + // supplier per runner. + auto& rootBc = state.m->buildConfig; + if (bc.runner.size() > runnerN) { + if (!rootBc.runner.empty() && !state.runnerProvider.empty()) + return std::unexpected(std::format( + "'{}' and '{}' both supply a runner for this target.\n" + " Drop one of them, or state the runner in " + "[target.].runner.", + state.runnerProvider, pkg.manifest.package.name)); + rootBc.runner.assign(bc.runner.begin() + static_cast(runnerN), + bc.runner.end()); + state.runnerProvider = pkg.manifest.package.name; + } + for (auto const& [name, nr] : bc.namedRunners) { + auto before = namedBefore.find(name); + const bool grew = before == namedBefore.end() + || nr.argv.size() > before->second.argv.size() + || (nr.longLived && !before->second.longLived); + if (!grew) continue; + auto& slot = rootBc.namedRunners[name]; + auto& who = state.namedRunnerProvider[name]; + if (!slot.argv.empty() && !who.empty()) + return std::unexpected(std::format( + "'{}' and '{}' both supply a runner named '{}' for this target.", + who, pkg.manifest.package.name, name)); + slot = nr; + who = pkg.manifest.package.name; + } + if (bc.runExclusive && !exclusiveBefore) rootBc.runExclusive = true; + } + if (auto err = state.checkVersionFloors(); err) return std::unexpected(*err); + // What the programs published (an include directory, an interface + // define) reaches the members' consumers, as after the dependencies' + // programs. + state.computeUsageRequirements(); + return {}; +} + static std::expected step9_device_sources_reach_an_action(PrepareState& state) { // ── Every device source must reach some action ───────────────────────── // @@ -2029,6 +2221,7 @@ std::expected phase9_target_side(PrepareState& state) { if (auto r = step9_dependency_link_forms(state); !r) return std::unexpected(r.error()); step9_define_graph_package_entry_closure(state); if (auto r = step9_root_build_program(state); !r) return std::unexpected(r.error()); + if (auto r = step9_member_build_programs(state); !r) return std::unexpected(r.error()); if (auto r = step9_device_sources_reach_an_action(state); !r) return std::unexpected(r.error()); if (auto r = step9_rerun_input_prepare_dir(state); !r) return std::unexpected(r.error()); diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 4c05b59c..8f587677 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -254,6 +254,10 @@ static std::expected step1_define_early_toolchain_closures(Pr pr.cflags.begin(), pr.cflags.end()); state.m->buildConfig.cxxflags.insert(state.m->buildConfig.cxxflags.end(), pr.cxxflags.begin(), pr.cxxflags.end()); + // A workspace plan's root compiles nothing; its selected members take + // these as they are loaded (graph.cpp). + state.profileCflags = pr.cflags; + state.profileCxxflags = pr.cxxflags; state.m->buildConfig.ldflags.insert(state.m->buildConfig.ldflags.end(), pr.ldflags.begin(), pr.ldflags.end()); } diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index 2d055ba2..dfc1f815 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -508,315 +508,47 @@ std::filesystem::path target_dir(const mcpp::toolchain::Toolchain& tc, } -// Compose a stable canonical compile-flags string for fingerprinting. -// Exported so the "every build-variant knob is in here" invariant is machine- -// checkable: the profile knobs were absent for a long time precisely because -// nothing could assert on this string. -// `includeDialectFlags`: false when this call serves the PER-PACKAGE -// fingerprint loop (`canonical_package_build_metadata` below) for a package -// that is not necessarily this build's root. `dialectCxxflags` is graph-wide -// (types.cppm's BuildConfig::dialectCxxflags): only the root's value reaches -// any command, so only the root's value may enter the fingerprint, and only -// once (#717 design 2026-09-27 §6.2). The default keeps this the SAME call -// the direct root-only call site below already makes. -std::string canonical_compile_flags(const mcpp::manifest::Manifest& m, - bool targetIsMacos = false, - bool includeDialectFlags = true) { +// The configuration's part of the build directory's name (workspace design +// 2026-09-29 §3). A build directory is one configuration: the inputs every +// node of one graph shares because objects compiled under two values of them +// cannot be linked or imported together. They are the C++ standard, the +// macOS deployment target, the dialect flags, the profile knobs and the +// dependency linkage, read from the root, which is the virtual root of a +// workspace plan; the toolchain, the target, the standard library, the runtime +// contract and the mcpp version enter the fingerprint beside this string. +// +// A package's own flags, sources, include directories, per-glob flags, +// C standard, module extensions and runtime needs are ATTRIBUTES of its node, +// not inputs of the configuration. They reach that package's command lines, +// and ninja, which records every edge's command, rebuilds exactly the edges +// whose commands changed. Hashing them into the directory's name instead moved +// every package of the graph to a new directory, and rebuilt all of them, +// when one package's `cxxflags` changed. +std::string canonical_configuration_flags(const mcpp::manifest::Manifest& root, + bool targetIsMacos = false) { std::string s; - s += "-std="; s += m.package.standard; + s += "-std="; s += root.package.standard; s += " -fmodules"; - // macOS deployment target changes the effective compile triple - // (arm64-apple-macosxNN) — a std.pcm built for one target cannot be - // loaded by a TU compiled for another. Fold the resolved value - // (env override > [build] macos_deployment_target manifest default) - // into the fingerprint so switching targets rebuilds the BMI cache - // instead of dying with a module config mismatch. - // - // TARGET-KEYED, NOT HOST-KEYED (#685). This used to read - // `if constexpr (mcpp::platform::is_macos)`, i.e. the platform mcpp - // itself was BUILT for, which left the fingerprint blind to - // `macos_deployment_target` whenever the BUILD ran on a non-Apple host — - // so `mcpp build --target aarch64-macos` on Linux kept the same output - // directory no matter what the manifest key was edited to. The caller - // answers whether THIS build's target is macOS (the same discriminator - // `min_platform_version` uses); it defaults to false so a caller that - // has no target in hand (a manifest-only unit test) gets today's - // non-macOS behaviour rather than silently guessing. - // - // The built-in default floor (rustc-style) lives in the single - // resolver (platform::macos::deployment_target), so this rule, the - // flags and the std-module prebuild always agree — the 0.0.50-era - // attempt to inject a default here alone left the test build's - // std.pcm unstaged (import std failed wholesale on macos CI). if (auto dtv = mcpp::platform::macos::deployment_target( - targetIsMacos, m.buildConfig.macosDeploymentTarget); + targetIsMacos, root.buildConfig.macosDeploymentTarget); !dtv.empty()) { s += " macos_deployment_target="; s += dtv; } - if (!m.buildConfig.cStandard.empty()) { - s += " c_standard="; - s += m.buildConfig.cStandard; - } - for (auto const& flag : m.buildConfig.cflags) { - s += " cflag:"; - s += flag; - } - for (auto const& flag : m.buildConfig.cxxflags) { - s += " cxxflag:"; + for (auto const& flag : root.buildConfig.dialectCxxflags) { + s += " dialect:"; s += flag; } - // Explicit [build] dialect_cxxflags (auto-promoted ones are already in - // cxxflags above) — they change every BMI in the graph. - // - // GATED: this is graph-wide (only the root's value reaches a command, - // BuildConfig::dialectCxxflags's own comment), so it belongs in the - // fingerprint only where `m` is known to be the root -- the direct call - // below, not the per-package loop of `canonical_package_build_metadata`, - // which calls this for every dependency too (#717 design §6.2, finding 7: - // a dependency's own value used to enter ITS fingerprint although it - // reaches no command). - if (includeDialectFlags) { - for (auto const& flag : m.buildConfig.dialectCxxflags) { - s += " dialect:"; - s += flag; - } - } - for (auto const& flag : m.buildConfig.ldflags) { - s += " ldflag:"; - s += flag; - } - // Per-glob flags (G4): full ordered serialization — glob + every list — - // so editing any entry (or reordering) re-fingerprints the output dir. - for (auto const& gf : m.buildConfig.globFlags) { - s += " globflags:"; s += gf.glob; - for (auto const& f : gf.cflags) { s += " gc:"; s += f; } - for (auto const& f : gf.cxxflags) { s += " gxx:"; s += f; } - for (auto const& f : gf.asmflags) { s += " gas:"; s += f; } - for (auto const& f : gf.defines) { s += " gd:"; s += f; } - } - // [build] module_extensions changes WHICH FILES ARE MODULE INTERFACES, - // i.e. the shape of the graph: which units emit a BMI, which objects link - // unconditionally, which ninja rule each unit gets. That is a build - // variant, so it belongs in the fingerprint — mcpp.toml's mtime alone only - // protects the fast path within one output dir, not the BMI cache. - // - // Contrast [build] build_program_timeout, which is deliberately absent: - // it changes no edge. See BuildConfig::buildProgramTimeoutSecs. - for (auto const& e : m.buildConfig.moduleExtensions) { - s += " modext:"; - s += e; - } - // The resolved [profile] knobs. These are NOT in cflags/cxxflags: the - // profile block (see the profile resolution below) lands them in - // buildConfig.optLevel/debug/lto/strip and flags.cppm turns them into - // -O/-g/-flto at command-construction time. Leaving them out made - // `--dev`, `--release` and `--profile dist` share ONE fingerprint, hence - // one target/// directory AND one global cache entry — so a - // release build could be served -O0 -g dependency objects. They are - // build-variant by definition; they belong here. - s += " opt="; s += m.buildConfig.optLevel; - s += " debug="; s += m.buildConfig.debug ? "1" : "0"; - s += " lto="; s += m.buildConfig.lto ? "1" : "0"; - s += " strip="; s += m.buildConfig.strip ? "1" : "0"; - // #519 — the same reasoning as the profile knobs above, one axis later. - // The REQUEST is folded in rather than the derived `-fPIC`, because this - // string is built before the plan exists; the request is what a user - // edits and the flag is a function of it. Without this, flipping - // `dependency_linkage` reuses the previous configuration's output - // directory — measured on a two-package fixture, where both builds landed - // in `target/x86_64-linux-gnu/5d4a4a8a584ba471/` and the shared build's - // `libcore.so` was left sitting in the static build's `bin/`. - // - // Only appended when non-empty, so every existing build directory keeps - // its identity and this release rebuilds nothing. - if (!m.buildConfig.dependencyLinkage.empty()) { + s += " opt="; s += root.buildConfig.optLevel; + s += " debug="; s += root.buildConfig.debug ? "1" : "0"; + s += " lto="; s += root.buildConfig.lto ? "1" : "0"; + s += " strip="; s += root.buildConfig.strip ? "1" : "0"; + if (!root.buildConfig.dependencyLinkage.empty()) { s += " deplinkage="; - s += m.buildConfig.dependencyLinkage; + s += root.buildConfig.dependencyLinkage; } return s; } -std::string canonical_package_build_metadata( - const std::vector& packages, - bool targetIsMacos = false) -{ - std::string s; - for (auto const& pkg : packages) { - s += "\npackage:"; - s += pkg.manifest.package.namespace_; - s += "/"; - s += pkg.manifest.package.name; - s += "@"; - s += pkg.manifest.package.version; - s += " source="; - s += pkg.manifest.package.sourceProvenance; - // WHAT THIS PACKAGE IS BUILT WITH, AND NOT ONLY WHAT IT ASKS THE - // RUNTIME FOR. - // - // Only the root's compile inputs used to reach the fingerprint, through - // `canonical_compile_flags` on the root manifest. A DEPENDENCY's - // `[build] cflags` / `defines` / `sources` / per-glob flags reached - // nothing — so editing one left the fingerprint unchanged, the consumer - // kept the same output directory, and the fast path replayed a - // build.ninja generated before the edit. - // - // AND THE WAY THAT SHOWS IS THAT THE EDIT APPEARS TO HAVE HAD NO - // EFFECT. Measured 2026-08-23 on a path dependency: a flag added to - // `[build] cflags` was absent from the generated `unit_cflags` after a - // rebuild, absent after touching the sources, and present the moment - // `target/` was removed. The first two observations are what a reader - // uses to conclude the flag is being filtered, and one was concluded - // and written down before the third measurement was taken. - // - // The comment beside the root-flag tail merge in prepare.cppm has said - // "canonical_package_build_metadata folds packages[].manifest. - // buildConfig" since before this fix. It now does. - // - // packages[0] is the root, whose flags `canonical_compile_flags` - // already folds; serialising it twice is harmless and keeps this loop - // one rule rather than one rule and an exception. - // - // EXCEPT for `dialect_cxxflags` (#717 design §6.2, finding 7): that - // key is graph-wide, so a dependency's own value must not enter ITS - // fingerprint contribution, and the root's must enter the fingerprint - // exactly once -- through the DIRECT root-only call this function's - // caller already makes on the root manifest (`canonical_compile_flags - // (*state.m, ...)`, scan.cpp), not through this per-package loop, - // where `includeDialectFlags = false` for every entry including the - // root. - s += ' '; - s += canonical_compile_flags(pkg.manifest, targetIsMacos, - /*includeDialectFlags=*/false); - // The level a C++-layer provider compiles its implementation units at - // (`make_plan`). Appended only when there is one, so every other - // output directory keeps its identity. - if (auto own = mcpp::manifest::cxx_layer_implementation_standard(pkg.manifest)) { - s += " implementation-standard="; - s += own->canonical; - } - for (auto const& src : pkg.manifest.buildConfig.sources) { - s += " src:"; - s += src; - } - for (auto const& dir : pkg.manifest.buildConfig.includeDirs) { - s += " inc:"; - s += dir.generic_string(); - } - for (auto const& dir : pkg.manifest.buildConfig.includeDirsAfter) { - s += " inca:"; - s += dir.generic_string(); - } - auto const& runtime = pkg.manifest.runtimeConfig; - for (auto const& requirement : runtime.requirements) { - s += " runtime-need:"; - s += requirement.kind; - s += ':'; - s += requirement.value; - s += ':'; - s += requirement.phase; - s += requirement.required ? ":required" : ":optional"; - } - for (auto const& artifact : runtime.artifacts) { - s += " runtime-artifact:"; - s += artifact.role; - s += ':'; - s += artifact.path.generic_string(); - s += ':'; - s += artifact.provenance; - s += ':'; - s += artifact.abi; - s += ':'; - s += artifact.digest; - s += ':'; - s += artifact.hostFingerprint; - } - for (auto const& value : runtime.linkIntent.libraries) - s += " link-library:" + value; - for (auto const& value : runtime.linkIntent.linkLibraryDirs) - s += " link-dir:" + value.generic_string(); - for (auto const& value : runtime.linkIntent.transitiveNeededDirs) - s += " needed-dir:" + value.generic_string(); - for (auto const& value : runtime.linkIntent.runtimeSearchDirs) - s += " runtime-dir:" + value.generic_string(); - for (auto const& value : runtime.linkIntent.frameworks) - s += " framework:" + value; - for (auto const& value : runtime.linkIntent.deployFiles) - s += " deploy:" + value.generic_string(); - // Legacy fields remain fingerprinted while they are readable. - for (auto const& value : runtime.libraryDirs) - s += " legacy-runtime-dir:" + value.generic_string(); - for (auto const& value : runtime.dlopenLibs) - s += " legacy-soname:" + value; - for (auto const& value : runtime.capabilities) - s += " legacy-capability:" + value; - for (auto const& value : runtime.provides) - s += " legacy-provides:" + value; - for (auto const& [capability, provider] : runtime.providerOverrides) - s += " provider-override:" + capability + '=' + provider; - if (!pkg.manifest.buildConfig.cStandard.empty()) { - s += " c_standard="; - s += pkg.manifest.buildConfig.cStandard; - } - for (auto const& flag : pkg.manifest.buildConfig.cflags) { - s += " cflag:"; - s += flag; - } - for (auto const& flag : pkg.manifest.buildConfig.cxxflags) { - s += " cxxflag:"; - s += flag; - } - for (auto const& flag : pkg.manifest.buildConfig.ldflags) { - s += " ldflag:"; - s += flag; - } - // Per-glob flags — same full ordered serialization as the root-side - // block above. Until #253 dependency globFlags were unfingerprinted - // (held only by "descriptor frozen per version" + "feature toggles - // always change cflags via -DMCPP_FEATURE_*"); feature-folded entries - // make the vector build-variant, so fingerprint it directly. - // featureOrigin is diagnostic-only and deliberately NOT serialized - // (the active feature set is already in cflags above). - for (auto const& gf : pkg.manifest.buildConfig.globFlags) { - s += " globflags:"; s += gf.glob; - for (auto const& f : gf.cflags) { s += " gc:"; s += f; } - for (auto const& f : gf.cxxflags) { s += " gxx:"; s += f; } - for (auto const& f : gf.asmflags) { s += " gas:"; s += f; } - for (auto const& f : gf.defines) { s += " gd:"; s += f; } - } - // Same reason as the root block, and it cannot be skipped on the - // grounds that "a descriptor is frozen per version": path and git - // dependencies are not frozen, and this key changes their products. - for (auto const& e : pkg.manifest.buildConfig.moduleExtensions) { - s += " modext:"; - s += e; - } - if (pkg.usageResolved) { - for (auto const& dir : pkg.privateBuild.includeDirs) { - s += " private_include:"; - s += dir.generic_string(); - } - for (auto const& dir : pkg.publicUsage.includeDirs) { - s += " public_include:"; - s += dir.generic_string(); - } - for (auto const& dir : pkg.privateBuild.includeDirsAfter) { - s += " private_include_after:"; - s += dir.generic_string(); - } - for (auto const& dir : pkg.publicUsage.includeDirsAfter) { - s += " public_include_after:"; - s += dir.generic_string(); - } - } - for (auto const& [path, content] : pkg.manifest.buildConfig.generatedFiles) { - s += " genfile:"; - s += path.generic_string(); - s += "="; - s += content; - } - } - return s; -} } // namespace mcpp::build diff --git a/src/build/stage.cppm b/src/build/stage.cppm index 7ee0b56e..fec199bc 100644 --- a/src/build/stage.cppm +++ b/src/build/stage.cppm @@ -154,14 +154,38 @@ std::filesystem::path temp_sibling(const std::filesystem::path& dst) { return dst.parent_path() / name; } +// A shared library is placed by a hard link where the file system allows one +// (workspace design 2026-09-29 §5.3): several product directories that load +// the same DLL or shared object then hold one copy of its bytes. Only shared +// libraries are linked. A program may write a data file that lies beside it, +// and through a link that write would reach the file's source, a package's own +// tree or a payload store; a loader never writes the libraries it loads. +bool links_on_placement(const std::filesystem::path& p) { + const auto name = p.filename().generic_string(); + auto lower = name; + for (auto& c : lower) c = static_cast(std::tolower(static_cast(c))); + return lower.ends_with(".dll") || lower.ends_with(".dylib") + || lower.ends_with(".so") || lower.find(".so.") != std::string::npos; +} + // One full write attempt: temp+rename first, in-place overwrite as fallback. +// The temporary is a hard link to the source when `links_on_placement` admits +// it and the link can be made, and a copy otherwise (another volume, FAT or +// exFAT, a permission), so the fallback is today's behaviour. // Returns an empty error_code on success; otherwise the most informative error // (the in-place one — that's where 1224 / 32 shows up). std::error_code write_once(const std::filesystem::path& src, const std::filesystem::path& dst) { auto tmp = temp_sibling(dst); std::error_code ec; - std::filesystem::copy_file(src, tmp, std::filesystem::copy_options::overwrite_existing, ec); + bool linked = false; + if (links_on_placement(src)) { + std::error_code lec; + std::filesystem::create_hard_link(src, tmp, lec); + linked = !lec; + } + if (!linked) + std::filesystem::copy_file(src, tmp, std::filesystem::copy_options::overwrite_existing, ec); if (!ec) { std::error_code rec; std::filesystem::rename(tmp, dst, rec); @@ -172,11 +196,18 @@ std::error_code write_once(const std::filesystem::path& src, // deletes. std::error_code rmec; std::filesystem::remove(tmp, rmec); + ec = rec; } else { std::error_code rmec; std::filesystem::remove(tmp, rmec); } + // A destination placed earlier by a link shares its bytes with that + // placement's source, so it is never written in place. + std::error_code nec; + if (const auto links = std::filesystem::hard_link_count(dst, nec); !nec && links > 1) + return ec; + std::error_code cec; std::filesystem::copy_file(src, dst, std::filesystem::copy_options::overwrite_existing, cec); if (!cec) return {}; diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 704aa3af..34c1633b 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -148,6 +148,10 @@ struct PackageRoot { UsageRequirements publicUsage; UsageRequirements linkUsage; bool usageResolved = false; + // Set on a selected workspace member that the plan builds (workspace + // design 2026-09-29 §15): the directory below `bin/` its products are + // placed in. Absent on the plan's root and on every dependency. + std::optional memberProducts; }; ScanResult scan_packages(const std::vector& packages); diff --git a/src/project.cppm b/src/project.cppm index 8f4163df..122729b6 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -649,4 +649,187 @@ resolve_member_dir(const mcpp::manifest::Manifest& rootManifest, package_filter, list)); } + +// ─── Workspace plans (workspace design 2026-09-29 §15) ────────────────────── +// +// A command on a workspace selects members and plans them under one root: the +// root package of a rooted workspace when it is selected, and otherwise a +// virtual root that holds the values one plan shares. The functions below are +// shared by the command layer, which groups the selected members into plans, +// and by the planner, which builds each plan's root; both read the same list +// of root-position values, so a group and its root cannot disagree about it. + +// A selected member's effective manifest: loaded as a member, with the +// workspace's configuration inherited, exactly as the planner loads the member +// it builds. +export std::expected +load_member_manifest(const mcpp::manifest::Manifest& workspace, + const std::filesystem::path& wsRoot, + std::string_view memberPath) { + const auto dir = wsRoot / std::string(memberPath); + if (memberPath == ".") { + auto m = workspace; + merge_workspace_deps(m, workspace, wsRoot); + return m; + } + if (!std::filesystem::exists(dir / "mcpp.toml")) + return std::unexpected(std::format( + "workspace member '{}' has no mcpp.toml", memberPath)); + auto mm = mcpp::manifest::load(dir / "mcpp.toml", {.insideWorkspace = true}); + if (!mm) return std::unexpected(std::format( + "workspace member '{}': {}", memberPath, mm.error().format())); + inherit_workspace_config(*mm, workspace, wsRoot); + if (auto bad = workspace_inheritance_error(*mm, dir)) return std::unexpected(*bad); + return std::move(*mm); +} + +// The values of a member's manifest that are one value per plan: the +// toolchain request and the target rows, the C++ standard, the graph-wide +// `[build]` keys, the profiles and the indices. Members whose keys are equal +// are planned together; the key is a canonical string of those values and +// nothing else, so a member's own flags, sources and dependencies never +// separate it from another member. +export std::string root_position_key(const mcpp::manifest::Manifest& m) { + std::string s; + auto field = [&](std::string_view name, std::string_view value) { + s += name; s += '='; s += value; s += '\x1f'; + }; + auto list = [&](std::string_view name, const std::vector& values) { + s += name; s += '='; + for (auto const& v : values) { s += v; s += '\x1e'; } + s += '\x1f'; + }; + for (auto const& [platform, spec] : m.toolchain.byPlatform) + field("toolchain." + platform, spec); + for (auto const& [triple, e] : m.targetOverrides) { + field("target." + triple + ".toolchain", e.toolchain); + field("target." + triple + ".linkage", e.linkage); + field("target." + triple + ".cxx_runtime", e.cxxRuntime); + field("target." + triple + ".sysroot", e.sysroot); + field("target." + triple + ".min_api", std::to_string(e.minApiLevel)); + list("target." + triple + ".runner", e.runner); + for (auto const& [name, argv] : e.namedRunners) + list("target." + triple + ".runners." + name, argv); + } + field("standard", m.package.standard); + auto const& b = m.buildConfig; + list("dialect_cxxflags", b.dialectCxxflags); + field("cxx_runtime", b.cxxRuntime); + field("cxx_runtime_tests", b.cxxRuntimeTests); + field("cxx_runtime_shared", b.cxxRuntimeShared); + field("static_stdlib", b.staticStdlib ? "1" : "0"); + field("dependency_linkage", b.dependencyLinkage); + field("target", b.target); + field("macos_deployment_target", b.macosDeploymentTarget); + field("ios_deployment_target", b.iosDeploymentTarget); + field("abi_threads", b.abiThreadsDeclared ? (b.abiThreads ? "1" : "0") : "-"); + field("abi_exceptions", b.abiExceptionsDeclared ? (b.abiExceptions ? "1" : "0") : "-"); + field("accel", b.accel); + field("bmi_schedule", b.bmiSchedule); + field("jobs", b.jobs); + field("cache", b.cacheMode); + field("allow_host_libs", b.allowHostLibs ? "1" : "0"); + field("platform_dependencies", b.platformDependencies); + field("default_profile", b.defaultProfile); + list("runner", b.runner); + for (auto const& [name, r] : b.namedRunners) + list("runners." + name + (r.longLived ? ".long" : ""), r.argv); + field("run_exclusive", b.runExclusive ? "1" : "0"); + for (auto const& [name, pr] : m.profiles) { + field("profile." + name, std::format("{}/{}/{}/{}/{}", pr.optLevel, + pr.debug ? 1 : 0, pr.lto ? 1 : 0, pr.strip ? 1 : 0, + pr.dependencyLinkageDeclared ? pr.dependencyLinkage : std::string("-"))); + list("profile." + name + ".cflags", pr.cflags); + list("profile." + name + ".cxxflags", pr.cxxflags); + list("profile." + name + ".ldflags", pr.ldflags); + } + for (auto const& [name, idx] : m.indices) + field("index." + name, std::format("{}#{}#{}#{}#{}", idx.url, idx.rev, idx.tag, + idx.branch, idx.path.generic_string())); + for (auto const& [cap, pin] : m.capabilityPins) field("cap." + cap, pin); + for (auto const& [tool, pin] : m.toolOverrides) field("tool." + tool, pin); + return s; +} + +// The root of a plan whose selected members do not include a rooted +// workspace's own package. It carries the root-position values of the first +// selected member, which every member of the plan shares (`root_position_key`), +// the workspace's own tables, and nothing a package owns: no sources, targets, +// dependencies, flags, hooks or build program. Its dependencies are the member +// edges the planner adds. +export mcpp::manifest::Manifest +virtual_workspace_root(const mcpp::manifest::Manifest& workspace, + const mcpp::manifest::Manifest& first, + const std::filesystem::path& wsRoot) { + mcpp::manifest::Manifest v; + v.sourcePath = wsRoot / "mcpp.toml"; + v.package.name = "workspace"; + v.package.version = "0.0.0"; + v.package.virtualRoot = true; + v.package.standard = first.package.standard; + v.package.standardDeclared = first.package.standardDeclared; + v.package.mcppFloor = workspace.workspace.inherited.mcppFloor; + v.language.standard = first.language.standard; + v.cppStandard = first.cppStandard; + v.toolchain = first.toolchain; + v.targetOverrides = first.targetOverrides; + v.indices = first.indices; + v.profiles = first.profiles; + v.capabilityPins = first.capabilityPins; + v.toolOverrides = first.toolOverrides; + v.workspace = workspace.workspace; + v.xlings.subos = workspace.xlings.subos; + v.xlings.subosDeclared = workspace.xlings.subosDeclared; + auto& b = v.buildConfig; + auto const& f = first.buildConfig; + b.sourcesDeclared = true; + b.runner = f.runner; + b.namedRunners = f.namedRunners; + b.runExclusive = f.runExclusive; + b.jobs = f.jobs; + b.bmiSchedule = f.bmiSchedule; + b.accel = f.accel; + b.staticStdlib = f.staticStdlib; + b.cxxRuntime = f.cxxRuntime; + b.cxxRuntimeTests = f.cxxRuntimeTests; + b.cxxRuntimeShared = f.cxxRuntimeShared; + b.target = f.target; + b.dialectCxxflags = f.dialectCxxflags; + b.abiThreads = f.abiThreads; + b.abiThreadsDeclared = f.abiThreadsDeclared; + b.abiExceptions = f.abiExceptions; + b.abiExceptionsDeclared = f.abiExceptionsDeclared; + b.allowHostLibs = f.allowHostLibs; + b.macosDeploymentTarget = f.macosDeploymentTarget; + b.iosDeploymentTarget = f.iosDeploymentTarget; + b.defaultProfile = f.defaultProfile; + b.dependencyLinkage = f.dependencyLinkage; + b.cacheMode = f.cacheMode; + b.platformDependencies = f.platformDependencies; + return v; +} + +// The directory a member's products are placed in, below the configuration's +// `bin/` (workspace design 2026-09-29 §5.2): the member's package name, or +// `.` when another member of the workspace has the same +// name. The workspace's own package, `"."`, keeps `bin/` itself (empty). +// Decided over every member of the workspace, not over one selection, so a +// member's directory is the same whichever command built it. +export std::string product_directory_name(const std::vector& all, + std::string_view memberPath) { + if (memberPath == ".") return {}; + auto it = std::ranges::find_if(all, [&](const WorkspaceMember& m) { + return m.memberPath == memberPath; + }); + if (it == all.end() || it->name.empty()) { + std::string s(memberPath); + for (auto& c : s) if (c == '/' || c == '\\') c = '.'; + return s; + } + const auto shared = std::ranges::count_if(all, [&](const WorkspaceMember& m) { + return m.name == it->name; + }); + return shared > 1 ? qualified_member_name(*it) : it->name; +} + } // namespace mcpp::project diff --git a/tests/e2e/830_a_shared_member_is_compiled_once.sh b/tests/e2e/830_a_shared_member_is_compiled_once.sh deleted file mode 100755 index 506865ff..00000000 --- a/tests/e2e/830_a_shared_member_is_compiled_once.sh +++ /dev/null @@ -1,118 +0,0 @@ -#!/usr/bin/env bash -# requires: unix-shell -# 830_a_shared_member_is_compiled_once.sh -- #734 E1. -# -# A workspace member that other members use as a path dependency is built once, -# in its own directory, and every consumer takes its objects and module -# interfaces from there. Its own ninja decides what is stale, including inputs -# outside its root. The fixture has the shape that measured the defect: a -# library member with module units, one unit outside its root, and a header -# outside its root; two program members use it. -# -# W1 `mcpp build --workspace` compiles each library unit once in total -# (counted from every build directory's .ninja_log); -# W2 a following `mcpp build -p app2` compiles no library unit; -# W3 a header outside the library's root changes: the next build compiles -# its includer once and no other library unit, and both programs run the -# new value; -# W4 after another change, `-p app1` and `-p app2` run at the same time: -# both succeed, the includer is compiled once, and both run the new -# value (one build of the member directory at a time). -set -e - -TMP=$(mktemp -d) -trap "rm -rf $TMP" EXIT -cd "$TMP" -fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } -MCPP="${MCPP:-mcpp}" - -mkdir -p ws/lib/src ws/shared ws/app1/src ws/app2/src -cd ws -cat > mcpp.toml <<'EOF' -[workspace] -members = ["lib", "app1", "app2"] - -[workspace.package] -version = "0.1.0" -EOF -cat > lib/mcpp.toml <<'EOF' -[package] -namespace = "probe" -name = "core830" - -[build] -sources = ["src/*.cppm", "src/*.cpp", "../shared/extra.cpp"] - -[targets.core830] -kind = "lib" -EOF -printf '#pragma once\ninline int shared_value() { return 7; }\n' > shared/value.h -printf 'export module probe.core830;\nexport int core_one();\nexport int core_extra();\n' > lib/src/core830.cppm -printf 'module probe.core830;\nint core_one() { return 1; }\n' > lib/src/one.cpp -printf 'module;\n#include "value.h"\nmodule probe.core830;\nint core_extra() { return shared_value(); }\n' > shared/extra.cpp -for a in app1 app2; do - printf '[package]\nname = "%s"\n\n[dependencies]\n"probe.core830" = { path = "../lib" }\n' "$a" > $a/mcpp.toml - printf 'import probe.core830;\n#include \nint main() { std::printf("%%d\\n", core_one() + core_extra()); return 0; }\n' > $a/src/main.cpp -done - -lib_compiles() { # compile edges that produced a library object, in every build directory - local n=0 - for log in $(find . -name .ninja_log -path '*target*' 2>/dev/null); do - local dir; dir=$(dirname "$log") - # The rule of each output the log records, read from that directory's - # build.ninja: a library object staged from the member's directory is - # a `stage_file` edge, a compile is a `cxx_module` / `cxx_object` edge. - c=$(awk -F'\t' 'NR>1 {print $4}' "$log" | grep -E '(core830|one|extra)\.(m\.)?o$' | - while read -r out; do - grep -E "^build [^:]*${out//./\\.}[ |:]" "$dir/build.ninja" | head -1 | - sed -E 's/^build [^:]*: ([a-z_]+).*/\1/' - done | grep -cE '^(cxx_module|cxx_object|c_object)$' || true) - n=$((n + c)) - done - echo "$n" -} - -# W1 -"$MCPP" build --workspace > w1.log 2>&1 || fail "W1: the workspace build failed" w1.log -units=3 # core830.cppm, one.cpp, ../shared/extra.cpp -got=$(lib_compiles) -[ "$got" -eq "$units" ] || { - for log in $(find . -name .ninja_log -path '*target*'); do echo "== $log"; awk -F'\t' 'NR>1 {print $4}' "$log" | grep -E "core830|one\.o|extra\.o"; done > w1.edges - fail "W1: the library's $units units were compiled $got times" w1.log w1.edges -} -grep -q "workspace member probe.core830 in its own directory" w1.log || fail "W1: the shared build was not taken" w1.log - -# W2 -before=$(lib_compiles) -"$MCPP" build -p app2 > w2.log 2>&1 || fail "W2: the build of app2 failed" w2.log -after=$(lib_compiles) -[ "$before" = "$after" ] || fail "W2: building app2 compiled library units again ($before -> $after)" w2.log - -# W3 -sleep 1.1 -printf '#pragma once\ninline int shared_value() { return 40; }\n' > shared/value.h -before=$(lib_compiles) -"$MCPP" build --workspace > w3.log 2>&1 || fail "W3: the rebuild failed" w3.log -after=$(lib_compiles) -[ $((after - before)) -eq 1 ] || fail "W3: a header outside the root caused $((after - before)) library compiles, not 1" w3.log -for a in app1 app2; do - "$MCPP" run -p $a > run-$a.log 2>&1 || fail "W3: $a did not run" run-$a.log - grep -qx 41 run-$a.log || fail "W3: $a did not take the changed header" run-$a.log -done - -# W4 -sleep 1.1 -printf '#pragma once\ninline int shared_value() { return 90; }\n' > shared/value.h -before=$(lib_compiles) -"$MCPP" build -p app1 > w4a.log 2>&1 & p1=$! -"$MCPP" build -p app2 > w4b.log 2>&1 & p2=$! -wait $p1 || fail "W4: the concurrent build of app1 failed" w4a.log -wait $p2 || fail "W4: the concurrent build of app2 failed" w4b.log -after=$(lib_compiles) -[ $((after - before)) -eq 1 ] || fail "W4: two concurrent builds compiled the includer $((after - before)) times" w4a.log w4b.log -for a in app1 app2; do - "$MCPP" run -p $a > run4-$a.log 2>&1 || fail "W4: $a did not run" run4-$a.log - grep -qx 91 run4-$a.log || fail "W4: $a did not take the changed header" run4-$a.log -done - -echo "OK" diff --git a/tests/unit/test_build_profile.cpp b/tests/unit/test_build_profile.cpp index 229c3f83..cf7899b9 100644 --- a/tests/unit/test_build_profile.cpp +++ b/tests/unit/test_build_profile.cpp @@ -82,18 +82,18 @@ TEST(BuildProfile, CustomProfileNamePassesThrough) { TEST(BuildProfile, OptLevelIsInTheCanonicalFlags) { auto a = base(); auto b = base(); b.buildConfig.optLevel = "0"; - EXPECT_NE(mcpp::build::canonical_compile_flags(a), - mcpp::build::canonical_compile_flags(b)); + EXPECT_NE(mcpp::build::canonical_configuration_flags(a), + mcpp::build::canonical_configuration_flags(b)); } TEST(BuildProfile, DebugLtoStripAreInTheCanonicalFlags) { - auto ref = mcpp::build::canonical_compile_flags(base()); + auto ref = mcpp::build::canonical_configuration_flags(base()); { auto m = base(); m.buildConfig.debug = true; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_NE(mcpp::build::canonical_configuration_flags(m), ref); } { auto m = base(); m.buildConfig.lto = true; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_NE(mcpp::build::canonical_configuration_flags(m), ref); } { auto m = base(); m.buildConfig.strip = true; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_NE(mcpp::build::canonical_configuration_flags(m), ref); } } // The exact shape the three built-in profiles resolve to (prepare_build's @@ -111,9 +111,9 @@ TEST(BuildProfile, BuiltInProfilesProduceDistinctCanonicalFlags) { dist.buildConfig.optLevel = "3"; dist.buildConfig.strip = true; - auto fdev = mcpp::build::canonical_compile_flags(dev); - auto frelease = mcpp::build::canonical_compile_flags(release); - auto fdist = mcpp::build::canonical_compile_flags(dist); + auto fdev = mcpp::build::canonical_configuration_flags(dev); + auto frelease = mcpp::build::canonical_configuration_flags(release); + auto fdist = mcpp::build::canonical_configuration_flags(dist); EXPECT_NE(fdev, frelease); EXPECT_NE(frelease, fdist); @@ -121,24 +121,33 @@ TEST(BuildProfile, BuiltInProfilesProduceDistinctCanonicalFlags) { } TEST(BuildProfile, CanonicalFlagsStillCoverTheNonProfileKnobs) { - auto ref = mcpp::build::canonical_compile_flags(base()); + auto ref = mcpp::build::canonical_configuration_flags(base()); { auto m = base(); m.package.standard = "c++26"; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_NE(mcpp::build::canonical_configuration_flags(m), ref); } + { auto m = base(); m.buildConfig.dialectCxxflags = {"-freflection"}; + EXPECT_NE(mcpp::build::canonical_configuration_flags(m), ref); } + { auto m = base(); m.buildConfig.dependencyLinkage = "shared"; + EXPECT_NE(mcpp::build::canonical_configuration_flags(m), ref); } +} + +// Workspace design 2026-09-29 §3: a package's own flags are attributes of its +// node, not inputs of the configuration, so editing them keeps the build +// directory and ninja rebuilds the edges whose commands changed. +TEST(BuildProfile, PackageFlagsAreNotInTheConfiguration) { + auto ref = mcpp::build::canonical_configuration_flags(base()); { auto m = base(); m.buildConfig.cxxflags = {"-DFOO"}; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_EQ(mcpp::build::canonical_configuration_flags(m), ref); } { auto m = base(); m.buildConfig.cflags = {"-DBAR"}; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_EQ(mcpp::build::canonical_configuration_flags(m), ref); } { auto m = base(); m.buildConfig.ldflags = {"-lm"}; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } - { auto m = base(); m.buildConfig.dialectCxxflags = {"-freflection"}; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_EQ(mcpp::build::canonical_configuration_flags(m), ref); } { auto m = base(); m.buildConfig.cStandard = "c17"; - EXPECT_NE(mcpp::build::canonical_compile_flags(m), ref); } + EXPECT_EQ(mcpp::build::canonical_configuration_flags(m), ref); } } // ── macos_deployment_target: TARGET-keyed, not HOST-keyed (mcpp#685) ──────── // -// `canonical_compile_flags` used to fold this value in only +// The fingerprint's flag string used to fold this value in only // `if constexpr (mcpp::platform::is_macos)` -- the platform mcpp ITSELF was // built for -- so `mcpp build --target aarch64-macos` on a Linux host kept // the exact same fingerprint (hence the exact same `target///` @@ -158,18 +167,18 @@ TEST(BuildProfile, NonMacosTargetIgnoresDeploymentTargetInTheFingerprint) { // one building for something other than macOS) must not see the // fingerprint move just because the manifest names a macOS floor it will // never apply. - EXPECT_EQ(mcpp::build::canonical_compile_flags(ref), - mcpp::build::canonical_compile_flags(changed)); - EXPECT_EQ(mcpp::build::canonical_compile_flags(ref, /*targetIsMacos=*/false), - mcpp::build::canonical_compile_flags(changed, /*targetIsMacos=*/false)); + EXPECT_EQ(mcpp::build::canonical_configuration_flags(ref), + mcpp::build::canonical_configuration_flags(changed)); + EXPECT_EQ(mcpp::build::canonical_configuration_flags(ref, /*targetIsMacos=*/false), + mcpp::build::canonical_configuration_flags(changed, /*targetIsMacos=*/false)); } TEST(BuildProfile, TargetIsMacosFoldsDeploymentTargetIntoTheFingerprint) { ScopedDeploymentTargetEnv noEnv(nullptr); auto ref = base(); auto changed = base(); changed.buildConfig.macosDeploymentTarget = "11.0"; - EXPECT_NE(mcpp::build::canonical_compile_flags(ref, /*targetIsMacos=*/true), - mcpp::build::canonical_compile_flags(changed, /*targetIsMacos=*/true)); + EXPECT_NE(mcpp::build::canonical_configuration_flags(ref, /*targetIsMacos=*/true), + mcpp::build::canonical_configuration_flags(changed, /*targetIsMacos=*/true)); // AND THE ENV OVERRIDE, folded the same way `deployment_target` resolves // it: with MACOSX_DEPLOYMENT_TARGET set, the manifest value stops @@ -178,8 +187,8 @@ TEST(BuildProfile, TargetIsMacosFoldsDeploymentTargetIntoTheFingerprint) { ScopedDeploymentTargetEnv env("12.3"); auto manifestA = base(); manifestA.buildConfig.macosDeploymentTarget = "11.0"; auto manifestB = base(); manifestB.buildConfig.macosDeploymentTarget = "13.0"; - EXPECT_EQ(mcpp::build::canonical_compile_flags(manifestA, /*targetIsMacos=*/true), - mcpp::build::canonical_compile_flags(manifestB, /*targetIsMacos=*/true)); + EXPECT_EQ(mcpp::build::canonical_configuration_flags(manifestA, /*targetIsMacos=*/true), + mcpp::build::canonical_configuration_flags(manifestB, /*targetIsMacos=*/true)); } // ── cache-mode parsing ────────────────────────────────────────────────────── diff --git a/tests/unit/test_build_stage.cpp b/tests/unit/test_build_stage.cpp index 98ada988..c253bbcf 100644 --- a/tests/unit/test_build_stage.cpp +++ b/tests/unit/test_build_stage.cpp @@ -120,6 +120,48 @@ TEST(BuildStage, DifferentSizeIsAlwaysCopied) { EXPECT_EQ(read_file(dst), "longer-payload"); } +// Workspace design 2026-09-29 §5.3: a shared library placed in two product +// directories holds one copy of its bytes where the file system allows a link; +// any other file is copied, so a program that writes a file beside itself +// never writes through to its source. +TEST(BuildStage, ASharedLibraryIsPlacedByALinkAndADataFileByACopy) { + Tmp tmp; + auto lib = tmp.path / "store" / "libfoo.so.1"; + auto data = tmp.path / "store" / "table.dat"; + write_file(lib, "library"); + write_file(data, "data"); + for (auto const& dir : {"cli", "gui"}) { + ASSERT_TRUE(stage_file(lib, tmp.path / "bin" / dir / "libfoo.so.1", no_retry())); + ASSERT_TRUE(stage_file(data, tmp.path / "bin" / dir / "table.dat", no_retry())); + } + std::error_code ec; + const auto libLinks = std::filesystem::hard_link_count(lib, ec); + ASSERT_FALSE(ec); + // Three names on a file system with links (the source and two + // placements), one where the link fell back to a copy. + EXPECT_TRUE(libLinks == 3 || libLinks == 1) << libLinks; + if (libLinks == 3) + EXPECT_TRUE(std::filesystem::equivalent(tmp.path / "bin" / "cli" / "libfoo.so.1", + tmp.path / "bin" / "gui" / "libfoo.so.1")); + EXPECT_EQ(std::filesystem::hard_link_count(data), 1u); + EXPECT_EQ(read_file(tmp.path / "bin" / "gui" / "table.dat"), "data"); +} + +// A linked placement is replaced out of place: a new source never rewrites the +// bytes the earlier source shares with the placement. +TEST(BuildStage, ReplacingALinkedPlacementLeavesTheEarlierSourceIntact) { + Tmp tmp; + auto first = tmp.path / "v1" / "foo.dll"; + auto second = tmp.path / "v2" / "foo.dll"; + auto dst = tmp.path / "bin" / "app" / "foo.dll"; + write_file(first, "version one"); + write_file(second, "version two!"); + ASSERT_TRUE(stage_file(first, dst, no_retry())); + ASSERT_TRUE(stage_file(second, dst, no_retry())); + EXPECT_EQ(read_file(dst), "version two!"); + EXPECT_EQ(read_file(first), "version one"); +} + TEST(BuildStage, MissingSourceIsAnError) { Tmp tmp; auto r = stage_file(tmp.path / "nope.bin", tmp.path / "dst.bin", no_retry()); From ee46723587eb09eed4ef42dd68c614f1952fd457 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 04:16:31 +0800 Subject: [PATCH 02/11] feat: add mcpp.graph and migrate hand-written graph algorithms onto it modules/graph provides topological_order, levels and closure over plain index-addressed adjacency lists (std::vector>, edge u -> v meaning "u depends on v"). Kahn's algorithm keeps its ready set ordered, so ties break by ascending node index; a cycle is reported as the ordered ring that walks it rather than the set of nodes left over. Five hand-written variants are migrated onto it, each previously a private DFS or Kahn's-algorithm implementation with its own tie-break and cycle-reporting shape: - src/modgraph/graph.cppm: topo_sort's Kahn's algorithm. The cycle diagnostic in validate.cppm now names the ring ("a -> b -> c -> a") instead of an unordered list of suspects. - src/build/prepare/features.cpp: the within-package unit order by imports (falls back to declaration order on a cycle, preserving the existing "leave it to the compiler" behaviour), and the cross-package host-module provider order (restricted to each consumer's own reachable closure before calling into the module, so a cycle outside that closure cannot fail an unrelated consumer's build). - src/build/prepare/graph.cpp: step4b_cycle_check's package-cycle DFS. - src/build/dep_graph.cppm: transitive_dependencies, on mcpp::graph::closure. - src/build/prepare/plan.cpp: the build-cache key fold walks topological_order instead of recursing, turning the memoized recursion into a loop; the per-node fold and taint stay local to the caller. modules/graph/tests covers ties, diamonds, disconnected components, self-loops, a longer cycle (path checked exactly), levels and closure, including the include/exclude-roots and cycle-tolerant cases. --- mcpp.toml | 2 + modules/graph/mcpp.toml | 20 +++ modules/graph/src/graph.cppm | 220 ++++++++++++++++++++++++++ modules/graph/tests/test_graph.cpp | 245 +++++++++++++++++++++++++++++ src/build/dep_graph.cppm | 54 ++++--- src/build/prepare/features.cpp | 146 ++++++++--------- src/build/prepare/graph.cpp | 35 ++--- src/build/prepare/plan.cpp | 50 +++--- src/modgraph/graph.cppm | 43 ++--- src/modgraph/validate.cppm | 15 +- 10 files changed, 649 insertions(+), 181 deletions(-) create mode 100644 modules/graph/mcpp.toml create mode 100644 modules/graph/src/graph.cppm create mode 100644 modules/graph/tests/test_graph.cpp diff --git a/mcpp.toml b/mcpp.toml index e75f4d68..838e1ad8 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -92,6 +92,7 @@ libs = { path = "modules/libs" } log = { path = "modules/log" } versioning = { path = "modules/versioning" } source-kind = { path = "modules/source-kind" } +graph = { path = "modules/graph" } dyndep = { path = "modules/dyndep" } platform = { path = "modules/platform" } manifest = { path = "modules/manifest" } @@ -104,6 +105,7 @@ members = [ "modules/log", "modules/versioning", "modules/source-kind", + "modules/graph", "modules/dyndep", "modules/platform", "modules/manifest", diff --git a/modules/graph/mcpp.toml b/modules/graph/mcpp.toml new file mode 100644 index 00000000..da2ce4e5 --- /dev/null +++ b/modules/graph/mcpp.toml @@ -0,0 +1,20 @@ +# mcpp.graph — algorithms over index-addressed directed graphs. +# +# A leaf by construction: ordering a set of nodes by their dependency edges +# needs no knowledge of packages, modules or manifests, and every caller +# already reduces its own graph (module imports, package dependencies, build +# rules) to node indices 0..n-1 before it asks the question. This package is +# that reduction's common answer, factored out after the same three +# algorithms — Kahn's algorithm with a cycle path, dependency levels, and a +# closure walk — had been hand-written at four call sites with four different +# tie-break and cycle-reporting behaviours. + +[package] +name = "graph" +namespace = "mcpp" +version = "0.1.0" +description = "Algorithms over index-addressed directed graphs: topological order, levels, closure" +license = "Apache-2.0" + +[dev-dependencies.compat] +gtest = "1.15.2" diff --git a/modules/graph/src/graph.cppm b/modules/graph/src/graph.cppm new file mode 100644 index 00000000..0300b23e --- /dev/null +++ b/modules/graph/src/graph.cppm @@ -0,0 +1,220 @@ +// mcpp.graph — Kahn's algorithm, dependency levels and closure, over +// index-addressed directed graphs. +// +// THE SHAPE EVERY CALLER ALREADY HAS +// +// A module-import graph, a package-dependency graph and a build-rule graph +// are unrelated domains, but by the time any of them asks "what order", "what +// depth" or "what does this reach", the caller has already reduced its own +// nodes to a dense index range 0..n-1 and its edges to an adjacency list. +// This package is that reduced question's one answer, factored out after +// `src/modgraph/graph.cppm`, `src/build/prepare/features.cpp` (twice), +// `src/build/dep_graph.cppm` and `src/build/prepare/plan.cpp` had each +// hand-written a variant of Kahn's algorithm or a DFS cycle check, with four +// different tie-break rules and three different cycle-reporting shapes. +// +// EDGE ORIENTATION +// +// `deps[u]` lists the nodes `u` depends on: an edge `u -> v` means "u depends +// on v", so a leaf with no dependencies is a node with an empty adjacency +// list. This is the direction every migrated call site already used for its +// own hand-rolled adjacency (a consumer's edge list, an importer's imports, +// a package's `[dependencies]`), so no call site has to invert its data to +// call in. +// +// STABILITY +// +// `topological_order` and `levels` are Kahn's algorithm with the ready set +// kept as an ordered set rather than an arbitrary queue: whenever more than +// one node has no unmet dependency, the LOWEST INDEX is emitted next. Two +// graphs that differ only in an unconstrained sibling order therefore +// disagree nowhere it can be observed by anything downstream, and the answer +// for a fully-ordered chain (0 depends on nothing, 1 depends on 0, ...) is +// always the identity permutation. +// +// CYCLES +// +// `topological_order` and `levels` report a cycle as the ordered PATH that +// walks it — `cycle = {a, b, c, a}`, read "a depends on b depends on c +// depends on a" — not merely the set of nodes that Kahn's algorithm could not +// retire. A path names the actionable edges; a set makes the reader +// reconstruct them. `closure` asks a different question (reachability, not +// order) and is defined to tolerate a cycle rather than reject it: a visited +// set already terminates the walk, and rejecting would turn every caller that +// does not care about order into one that has to. + +export module mcpp.graph; + +import std; + +export namespace mcpp::graph { + +// `deps[u]` = the node ids `u` depends on, `u` and each entry of `deps[u]` +// being indices into `deps` itself (0..deps.size()-1). Duplicate entries +// within one node's list are tolerated (treated as one edge). +using AdjacencyList = std::vector>; + +// A cycle, reported as the ring it walks: `cycle.front() == cycle.back()`, +// and consecutive entries are a real edge (`cycle[i]` depends on +// `cycle[i + 1]`). Never empty when returned as an error — a ring has at +// least the two entries closing a self-loop (`{a, a}`). +struct CycleError { + std::vector cycle; +}; + +// Dependency-first order: every node comes after every node it (transitively) +// depends on. See the module header for the stability and cycle contract. +std::expected, CycleError> +topological_order(const AdjacencyList& deps); + +// `result[u]` = 0 when `u` has no dependencies, otherwise +// `1 + max(result[v] for v in deps[u])`. Same stability and cycle contract as +// `topological_order`, computed by walking that order once — every node's +// dependencies are already leveled by the time the node itself is reached. +std::expected, CycleError> +levels(const AdjacencyList& deps); + +// Every node reachable from `roots` by following dependency edges +// transitively, returned sorted and deduplicated. `include_roots` selects +// whether the roots themselves are in the result (default: excluded, i.e. the +// STRICT closure — what the roots depend on, not the roots' own identity). +// +// Tolerant of cycles: a visited set already terminates the walk, so a cycle +// reachable from a root is silently absorbed into the answer rather than +// rejected. Closure is a reachability question; a caller that also needs to +// know a cycle is not well-formed asks `topological_order` or `levels` +// instead — that is exactly the caller (`plan.cpp`'s cache-key fold) that +// must reject one. +std::vector +closure(const AdjacencyList& deps, std::span roots, + bool include_roots = false); + +} // namespace mcpp::graph + +namespace mcpp::graph { + +namespace { + +// Shared Kahn's-algorithm walk. Returns the dependency-first order, or (when +// a cycle remains) the CycleError both public functions report verbatim. +std::expected, CycleError> +kahn_order(const AdjacencyList& deps) { + const std::size_t n = deps.size(); + std::vector> dependents(n); // reverse edges + std::vector remaining(n, 0); // unmet deps left + for (std::size_t u = 0; u < n; ++u) { + for (auto v : deps[u]) { + remaining[u]++; + dependents[v].push_back(u); + } + } + + std::set ready; // ordered: smallest index served first + for (std::size_t u = 0; u < n; ++u) + if (remaining[u] == 0) ready.insert(u); + + std::vector order; + order.reserve(n); + while (!ready.empty()) { + auto it = ready.begin(); + std::size_t u = *it; + ready.erase(it); + order.push_back(u); + for (auto w : dependents[u]) + if (--remaining[w] == 0) ready.insert(w); + } + + if (order.size() == n) return order; + + // A cycle remains among the nodes Kahn's algorithm could not retire + // (`remaining[u] > 0`). Walk it with an explicit-stack DFS restricted to + // those nodes to recover the actual ring, rather than reporting the + // leftover set: the set says something is wrong, the ring says what. + std::vector stuck(n); + for (std::size_t u = 0; u < n; ++u) stuck[u] = remaining[u] > 0; + + std::vector state(n, 0); // 0 unseen, 1 on the current path, 2 done + std::vector path; + for (std::size_t start = 0; start < n; ++start) { + if (!stuck[start] || state[start] != 0) continue; + std::vector> frame{{start, 0}}; + while (!frame.empty()) { + auto& [u, k] = frame.back(); + if (k == 0) { + state[u] = 1; + path.push_back(u); + } + bool descended = false; + while (k < deps[u].size()) { + auto v = deps[u][k++]; + if (!stuck[v]) continue; + if (state[v] == 1) { + auto ring_start = std::ranges::find(path, v); + CycleError err; + err.cycle.assign(ring_start, path.end()); + err.cycle.push_back(v); + return std::unexpected(std::move(err)); + } + if (state[v] == 0) { + frame.push_back({v, 0}); + descended = true; + break; + } + } + if (descended) continue; + state[u] = 2; + path.pop_back(); + frame.pop_back(); + } + } + // Unreachable in theory: Kahn's algorithm left at least one stuck node, + // and every stuck node's dependencies are themselves stuck (otherwise + // Kahn's algorithm would have retired it), so the DFS above always + // closes a ring before running out of stuck, unseen starting points. + // Reported defensively rather than assumed, as an unclosed path rather + // than a valid ring, so a defect here is legible instead of undefined. + CycleError err; + err.cycle = path; + return std::unexpected(std::move(err)); +} + +} // namespace + +std::expected, CycleError> +topological_order(const AdjacencyList& deps) { + return kahn_order(deps); +} + +std::expected, CycleError> +levels(const AdjacencyList& deps) { + auto order = kahn_order(deps); + if (!order) return std::unexpected(order.error()); + + std::vector level(deps.size(), 0); + for (auto u : *order) { + std::size_t lv = 0; + for (auto v : deps[u]) lv = std::max(lv, level[v] + 1); + level[u] = lv; + } + return level; +} + +std::vector +closure(const AdjacencyList& deps, std::span roots, + bool include_roots) { + std::set seen; + std::vector stack(roots.begin(), roots.end()); + for (auto r : roots) seen.insert(r); + while (!stack.empty()) { + auto u = stack.back(); + stack.pop_back(); + if (u >= deps.size()) continue; + for (auto v : deps[u]) + if (seen.insert(v).second) stack.push_back(v); + } + if (!include_roots) + for (auto r : roots) seen.erase(r); + return {seen.begin(), seen.end()}; +} + +} // namespace mcpp::graph diff --git a/modules/graph/tests/test_graph.cpp b/modules/graph/tests/test_graph.cpp new file mode 100644 index 00000000..c7eb00b3 --- /dev/null +++ b/modules/graph/tests/test_graph.cpp @@ -0,0 +1,245 @@ +#include + +import std; +import mcpp.graph; + +// SUBSYSTEM-LEVEL: this package answers "what order / what depth / what does +// this reach" for an index-addressed directed graph, and nothing else. The +// migrated callers (src/modgraph/graph.cppm, src/build/prepare/features.cpp, +// src/build/prepare/graph.cpp, src/build/dep_graph.cppm, +// src/build/prepare/plan.cpp) restate a subset of these facts through their +// own edge shape in tests/unit/test_modgraph.cpp and in the e2e suite. + +namespace g = mcpp::graph; + +namespace { +g::AdjacencyList chain(std::size_t n) { + // 0 <- 1 <- 2 <- ... <- n-1 (node i depends on i-1). + g::AdjacencyList deps(n); + for (std::size_t i = 1; i < n; ++i) deps[i] = {i - 1}; + return deps; +} +} + +TEST(TopologicalOrder, EmptyGraphOrdersNothing) { + auto r = g::topological_order({}); + ASSERT_TRUE(r.has_value()); + EXPECT_TRUE(r->empty()); +} + +TEST(TopologicalOrder, SingleNodeNoDeps) { + auto r = g::topological_order({{}}); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0})); +} + +TEST(TopologicalOrder, UnconstrainedNodesComeOutInInputIndexOrder) { + // No edges at all: every node is ready from the start, and ties are + // broken by index, so the order is the identity permutation. + g::AdjacencyList deps(5); + auto r = g::topological_order(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 1, 2, 3, 4})); +} + +TEST(TopologicalOrder, TiesAreBrokenByInputIndexEvenAgainstDeclarationOrder) { + // Node 2 depends on nothing but is declared last among the roots; nodes + // 0 and 1 depend on nothing either. All three become ready at once, so + // the tie-break (lowest index first) — not the order dependents were + // declared in — decides: 0, 1, 2, then 3 (which depends on all three). + g::AdjacencyList deps(4); + deps[3] = {2, 0, 1}; // declared in a scrambled order on purpose + auto r = g::topological_order(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 1, 2, 3})); +} + +TEST(TopologicalOrder, Diamond) { + // 3 depends on {1, 2}; 1 and 2 both depend on {0}. + g::AdjacencyList deps(4); + deps[1] = {0}; + deps[2] = {0}; + deps[3] = {1, 2}; + auto r = g::topological_order(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 1, 2, 3})); +} + +TEST(TopologicalOrder, DisconnectedComponentsOrderByIndexNotByComponent) { + // Two unrelated chains, 0 -> 2 and 1 -> 3 (node 2 depends on node 0, + // node 3 depends on node 1). Both roots are ready at once, so the + // tie-break processes 0 then 1 before either child — the two + // components' roots interleave by index rather than one component + // finishing before the other starts. + g::AdjacencyList deps(4); + deps[2] = {0}; + deps[3] = {1}; + auto r = g::topological_order(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 1, 2, 3})); +} + +TEST(TopologicalOrder, LongChainIsTheIdentityPermutation) { + auto deps = chain(6); + auto r = g::topological_order(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 1, 2, 3, 4, 5})); +} + +TEST(TopologicalOrder, SelfLoopIsACycleOfLengthOne) { + g::AdjacencyList deps(1); + deps[0] = {0}; + auto r = g::topological_order(deps); + ASSERT_FALSE(r.has_value()); + EXPECT_EQ(r.error().cycle, (std::vector{0, 0})); +} + +TEST(TopologicalOrder, LongerCycleReportsTheExactPath) { + // a(0) -> b(1) -> c(2) -> a(0), plus an unrelated node 3 that depends on + // nothing and must not appear in the ring. + g::AdjacencyList deps(4); + deps[0] = {1}; // a depends on b + deps[1] = {2}; // b depends on c + deps[2] = {0}; // c depends on a + auto r = g::topological_order(deps); + ASSERT_FALSE(r.has_value()); + // The path starts wherever the DFS first re-enters the ring; what is + // pinned down is that it IS the ring a -> b -> c -> a, walked in one + // direction, closed at both ends by the same node. + const auto& cycle = r.error().cycle; + ASSERT_GE(cycle.size(), 2u); + EXPECT_EQ(cycle.front(), cycle.back()); + for (std::size_t i = 0; i + 1 < cycle.size(); ++i) { + auto u = cycle[i], v = cycle[i + 1]; + EXPECT_TRUE(std::ranges::find(deps[u], v) != deps[u].end()) + << u << " does not declare a dependency on " << v; + } + // Exactly the three ring members, not the unrelated fourth node. + std::set members(cycle.begin(), cycle.end()); + EXPECT_EQ(members, (std::set{0, 1, 2})); +} + +TEST(TopologicalOrder, CycleAmongOtherwiseOrderableNodes) { + // 0 is a normal leaf; 1 and 2 cycle on each other; 3 depends on 1. + g::AdjacencyList deps(4); + deps[1] = {2}; + deps[2] = {1}; + deps[3] = {1}; + auto r = g::topological_order(deps); + ASSERT_FALSE(r.has_value()); + std::set members(r.error().cycle.begin(), r.error().cycle.end()); + EXPECT_EQ(members, (std::set{1, 2})); +} + +TEST(Levels, LeavesAreLevelZero) { + g::AdjacencyList deps(3); + auto r = g::levels(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 0, 0})); +} + +TEST(Levels, ChainCountsUp) { + auto deps = chain(4); + auto r = g::levels(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ(*r, (std::vector{0, 1, 2, 3})); +} + +TEST(Levels, DiamondTakesTheLongerArm) { + // 4 depends on {1, 3}; 1 depends on {0}; 3 depends on {2}, {2} depends + // on {0}. The two arms from 0 to 4 have different lengths (2 and 3), and + // the level is the LONGER one. + g::AdjacencyList deps(5); + deps[1] = {0}; + deps[2] = {0}; + deps[3] = {2}; + deps[4] = {1, 3}; + auto r = g::levels(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ((*r)[0], 0u); + EXPECT_EQ((*r)[1], 1u); + EXPECT_EQ((*r)[2], 1u); + EXPECT_EQ((*r)[3], 2u); + EXPECT_EQ((*r)[4], 3u); +} + +TEST(Levels, DisconnectedNodesAreIndependent) { + g::AdjacencyList deps(3); + deps[2] = {1}; + // node 0 is isolated and must not affect node 2's level. + auto r = g::levels(deps); + ASSERT_TRUE(r.has_value()); + EXPECT_EQ((*r)[0], 0u); + EXPECT_EQ((*r)[1], 0u); + EXPECT_EQ((*r)[2], 1u); +} + +TEST(Levels, SameCycleErrorAsTopologicalOrder) { + g::AdjacencyList deps(2); + deps[0] = {1}; + deps[1] = {0}; + auto r = g::levels(deps); + ASSERT_FALSE(r.has_value()); + std::set members(r.error().cycle.begin(), r.error().cycle.end()); + EXPECT_EQ(members, (std::set{0, 1})); +} + +TEST(Closure, DirectNeighboursAreJustAdjacency) { + g::AdjacencyList deps(3); + deps[0] = {1, 2}; + EXPECT_EQ(deps[0], (std::vector{1, 2})); +} + +TEST(Closure, ExcludesRootsByDefault) { + // 0 -> 1 -> 2 -> 3 (a chain of dependencies). + g::AdjacencyList deps(4); + deps[0] = {1}; + deps[1] = {2}; + deps[2] = {3}; + auto c = g::closure(deps, std::array{std::size_t{0}}); + EXPECT_EQ(c, (std::vector{1, 2, 3})); +} + +TEST(Closure, IncludeRootsAddsThemBack) { + g::AdjacencyList deps(4); + deps[0] = {1}; + deps[1] = {2}; + deps[2] = {3}; + auto c = g::closure(deps, std::array{std::size_t{0}}, /*include_roots=*/true); + EXPECT_EQ(c, (std::vector{0, 1, 2, 3})); +} + +TEST(Closure, MultipleRootsUnionAndDeduplicate) { + // 0 -> 2, 1 -> 2 -> 3: both roots reach 2, so it appears once. + g::AdjacencyList deps(4); + deps[0] = {2}; + deps[1] = {2}; + deps[2] = {3}; + auto c = g::closure(deps, std::array{std::size_t{0}, std::size_t{1}}); + EXPECT_EQ(c, (std::vector{2, 3})); +} + +TEST(Closure, DisconnectedNodeIsNeverReached) { + g::AdjacencyList deps(3); + deps[0] = {1}; + // node 2 is isolated: no root can reach it. + auto c = g::closure(deps, std::array{std::size_t{0}}); + EXPECT_EQ(c, (std::vector{1})); +} + +TEST(Closure, TolerantOfACycleReachableFromARoot) { + // 0 -> 1 -> 2 -> 1 (a cycle downstream of the root). Closure does not + // reject it; it just returns the reachable set once. + g::AdjacencyList deps(3); + deps[0] = {1}; + deps[1] = {2}; + deps[2] = {1}; + auto c = g::closure(deps, std::array{std::size_t{0}}); + EXPECT_EQ(c, (std::vector{1, 2})); +} + +TEST(Closure, RootWithNoEdgesHasAnEmptyClosure) { + g::AdjacencyList deps(2); + auto c = g::closure(deps, std::array{std::size_t{0}}); + EXPECT_TRUE(c.empty()); +} diff --git a/src/build/dep_graph.cppm b/src/build/dep_graph.cppm index 452476a4..40d1178f 100644 --- a/src/build/dep_graph.cppm +++ b/src/build/dep_graph.cppm @@ -23,17 +23,22 @@ // // WHAT IS DELIBERATELY *NOT* HERE // -// The build cache's per-package key walk (prepare.cppm, `self(self, …)`) is -// NOT a closure query and is not migrated. It is a memoized fold that computes -// a value per node (that package's full cache key), treats a cycle as a hard -// ERROR rather than something to skip, and threads a taint flag alongside. -// Folding it into a generic traversal would either lose those properties or -// force the abstraction to grow until it described exactly one caller. Two -// walks that answer genuinely different questions are not duplication. +// The build cache's per-package key walk (plan.cpp) is NOT a closure query +// and its fold is not expressed through this module. It computes a value per +// node (that package's full cache key) FROM its dependencies' already-computed +// values, treats a cycle as a hard ERROR rather than something to skip, and +// threads a taint flag alongside. It does use mcpp.graph — `topological_order` +// turns what used to be a memoized recursion into a loop over that order — but +// the per-node fold and the taint are still the caller's, because folding them +// into a generic traversal would either lose those properties or force the +// abstraction to grow until it described exactly one caller. `closure` and +// `topological_order` ask two genuinely different questions (reachability, +// order), and neither is "compute this fold". export module mcpp.build.dep_graph; import std; +import mcpp.graph; export namespace mcpp::build::dep_graph { @@ -55,29 +60,28 @@ direct_dependencies(const std::vector& edges, std::size_t consumer) { // Every package reachable from `from`, excluding `from` itself. Sorted and // deduplicated, so a caller folding it into a cache key gets a stable answer -// without re-sorting. +// without re-sorting. Implemented on mcpp.graph's closure: the edges are +// reduced to an adjacency list once and mcpp::graph::closure walks it, which +// is the same "consumer depends on dependency" direction the edge already +// carries (`u -> v` in mcpp.graph means "u depends on v"). // -// A cycle is TOLERATED here (the visited set terminates it) rather than -// reported. This is a reachability question, and the callers that must reject -// a cycle — the build-cache key walk — detect it where they can say which -// package the cycle runs through, which is the only form of that message worth -// printing. +// A cycle is TOLERATED here (mcpp::graph::closure's visited set terminates +// it) rather than reported. This is a reachability question, and the callers +// that must reject a cycle — the build-cache key walk — detect it where they +// can say which package the cycle runs through, which is the only form of +// that message worth printing. template std::vector transitive_dependencies(const std::vector& edges, std::size_t from) { - std::set seen; - std::vector stack{from}; - while (!stack.empty()) { - auto cur = stack.back(); - stack.pop_back(); - for (auto const& e : edges) { - if (e.consumerPackageIndex != cur) continue; - if (!seen.insert(e.dependencyPackageIndex).second) continue; - stack.push_back(e.dependencyPackageIndex); - } + std::size_t n = from + 1; + for (auto const& e : edges) { + n = std::max(n, e.consumerPackageIndex + 1); + n = std::max(n, e.dependencyPackageIndex + 1); } - seen.erase(from); - return {seen.begin(), seen.end()}; + mcpp::graph::AdjacencyList deps(n); + for (auto const& e : edges) + deps[e.consumerPackageIndex].push_back(e.dependencyPackageIndex); + return mcpp::graph::closure(deps, std::array{from}, /*include_roots=*/false); } // The two spellings a dependency is addressable by: its canonical package name diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 332a0708..1599d467 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -18,6 +18,7 @@ import mcpp.modgraph.glob; import mcpp.modgraph.graph; import mcpp.modgraph.scanner; import mcpp.modgraph.validate; +import mcpp.graph; import mcpp.toolchain.hostflags; // the compile-token producer the package std module reuses import mcpp.toolchain.detect; import mcpp.toolchain.dialect; @@ -1014,37 +1015,29 @@ static std::vector host_module_units(const PrepareState& st, s for (std::size_t i = 0; i < pending.size(); ++i) byName.emplace(pending[i].name, i); - std::vector state(pending.size(), 0); // 0 new, 1 open, 2 done + // deps[u] = the pending-list indices `u` imports BY NAME within this + // same package. mcpp.graph's stable tie-break (lowest index first among + // unconstrained nodes) is what preserves path order in the + // unconstrained case; the old hand-written DFS achieved the same thing + // by always trying the path-sorted list from the front. + mcpp::graph::AdjacencyList deps(pending.size()); + for (std::size_t u = 0; u < pending.size(); ++u) + for (auto const& want : pending[u].imports) + if (auto it = byName.find(want); it != byName.end()) + deps[u].push_back(it->second); + + // A CYCLE IS LEFT TO THE COMPILER, ON PURPOSE. It is ill-formed C++ and + // the compiler says so with the two units named; refusing here would + // report the same fact in a worse place, and getting the ordering wrong + // is no longer possible either way -- so on a cycle this falls back to + // declaration order instead of surfacing mcpp.graph's CycleError. std::vector order; - order.reserve(pending.size()); - // Iterative post-order DFS over the path-sorted list: the first - // unit that can be emitted is emitted, which is what preserves - // path order in the unconstrained case. - const auto visit = [&](std::size_t start) { - std::vector> stack{{start, 0}}; - while (!stack.empty()) { - auto& [u, k] = stack.back(); - if (state[u] == 2) { stack.pop_back(); continue; } - state[u] = 1; - if (k < pending[u].imports.size()) { - auto const& want = pending[u].imports[k++]; - auto it = byName.find(want); - // A CYCLE IS LEFT TO THE COMPILER, ON PURPOSE. It - // is ill-formed C++ and the compiler says so with - // the two units named; refusing here would report - // the same fact in a worse place, and getting the - // ordering wrong is no longer possible either way. - if (it != byName.end() && state[it->second] == 0) - stack.push_back({it->second, 0}); - continue; - } - state[u] = 2; - order.push_back(u); - stack.pop_back(); - } - }; - for (std::size_t i = 0; i < pending.size(); ++i) - if (state[i] == 0) visit(i); + if (auto ordered = mcpp::graph::topological_order(deps)) { + order = std::move(*ordered); + } else { + order.resize(pending.size()); + std::iota(order.begin(), order.end(), std::size_t{0}); + } for (auto i : order) push(pending[i].path, std::move(pending[i].name)); return out; @@ -1135,6 +1128,12 @@ step6_host_module_registration(PrepareState& state) { ? pkg.name : pkg.namespace_ + "." + pkg.name; }; auto units = [&](std::size_t p) { return host_module_units(state, p); }; + // Host-module provider edges over EVERY package, built once: + // pkgHostDeps[p] is what directHostProviders(p) computes, reused + // below instead of re-querying provisionGraph per consumer. + mcpp::graph::AdjacencyList pkgHostDeps(state.packages.size()); + for (std::size_t p = 0; p < state.packages.size(); ++p) + pkgHostDeps[p] = directHostProviders(p); for (std::size_t c = 0; c < state.provisionGraph.visible.size(); ++c) { const auto direct = directHostProviders(c); if (direct.empty()) continue; @@ -1161,55 +1160,60 @@ step6_host_module_registration(PrepareState& state) { } } - // Post-order DFS, so a rule's own host modules are compiled - // BEFORE it. That ordering is the entire mechanism: the - // compile loop in build_program.cppm accumulates the module - // flags as it goes, so each entry sees the BMIs of everything - // ahead of it, and "a rule may import another rule" needs no - // second machinery — only this sort. - std::vector ordered; - std::set done; - std::vector path; // for the cycle diagnostic - auto visit = [&](auto&& self, std::size_t p) -> std::expected { - if (done.contains(p)) return {}; - if (std::ranges::find(path, p) != path.end()) { - // A cycle, reported AS a cycle and naming the packages - // on it. A depth limit would answer a different - // question and would answer it later. - std::string ring; - bool started = false; - for (auto q : path) { - if (q == p) started = true; - if (!started) continue; - ring += identity(q); - ring += " -> "; - } - ring += identity(p); - return std::unexpected(std::format( - "build rules form an import cycle: {}\n" - " A rule's host modules are compiled before " - "it, so a cycle has no order that could satisfy " - "all of them.", ring)); + // Dependency-first order, so a rule's own host modules are + // compiled BEFORE it. That ordering is the entire mechanism: + // the compile loop in build_program.cppm accumulates the + // module flags as it goes, so each entry sees the BMIs of + // everything ahead of it, and "a rule may import another + // rule" needs no second machinery — only this sort. + // + // Restricted to `c`'s own reachable closure (not the whole + // project) and reindexed to a dense local graph before + // calling mcpp.graph: a cycle elsewhere among packages `c` + // never reaches must not fail `c`'s build. + auto reach = mcpp::graph::closure(pkgHostDeps, direct, + /*include_roots=*/true); + std::map local; + for (std::size_t i = 0; i < reach.size(); ++i) + local.emplace(reach[i], i); + mcpp::graph::AdjacencyList sub(reach.size()); + for (std::size_t i = 0; i < reach.size(); ++i) + for (auto q : pkgHostDeps[reach[i]]) + if (auto it = local.find(q); it != local.end()) + sub[i].push_back(it->second); + + auto topo = mcpp::graph::topological_order(sub); + if (!topo) { + // A cycle, reported AS a cycle and naming the packages + // on it. A depth limit would answer a different + // question and would answer it later. + std::string ring; + bool first = true; + for (auto li : topo.error().cycle) { + if (!first) ring += " -> "; + first = false; + ring += identity(reach[li]); } - path.push_back(p); - for (auto q : directHostProviders(p)) - if (auto r = self(self, q); !r) return r; - path.pop_back(); - done.insert(p); + return std::unexpected(std::format( + "build rules form an import cycle: {}\n" + " A rule's host modules are compiled before " + "it, so a cycle has no order that could satisfy " + "all of them.", ring)); + } + + std::vector ordered; + for (auto li : *topo) { + auto p = reach[li]; for (auto& hm : units(p)) { hm.importable = isDirect.contains(p); ordered.push_back(std::move(hm)); } - return {}; - }; - for (auto p : direct) - if (auto r = visit(visit, p); !r) - return std::unexpected(r.error()); + } // Every provider on this consumer's rule closure, transitive - // ones included -- `done` is exactly that set, and a rule + // ones included -- `reach` is exactly that set, and a rule // imported by another rule declares payloads just as directly. - state.hostModuleProvidersByConsumer[c].assign(done.begin(), done.end()); + state.hostModuleProvidersByConsumer[c].assign(reach.begin(), reach.end()); if (auto clash = prov::host_module_collision(ordered)) return std::unexpected(*clash); diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index df6b277e..4e833561 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -19,6 +19,7 @@ import mcpp.modgraph.glob; import mcpp.modgraph.graph; import mcpp.modgraph.scanner; import mcpp.modgraph.validate; +import mcpp.graph; import mcpp.toolchain.hostflags; // the compile-token producer the package std module reuses import mcpp.toolchain.detect; import mcpp.toolchain.dialect; @@ -1917,33 +1918,15 @@ static std::expected step4b_cycle_check(PrepareState& state) // refused by default and built under `--cache=local`. Every edge counts, // build-only ones included, as the key walk counts them. { - // Named visitState, not state: `state` is this function's PrepareState - // parameter, which the loop below also reads. - std::vector visitState(state.packages.size(), 0); // 0 new / 1 on stack / 2 done - std::vector stack, cycle; - auto visit = [&](auto&& self, std::size_t u) -> bool { - visitState[u] = 1; - stack.push_back(u); - for (auto const& e : state.dependencyEdges) { - if (e.consumerPackageIndex != u) continue; - const auto v = e.dependencyPackageIndex; - if (v >= state.packages.size()) continue; - if (visitState[v] == 1) { - cycle.assign(std::ranges::find(stack, v), stack.end()); - cycle.push_back(v); - return true; - } - if (visitState[v] == 0 && self(self, v)) return true; - } - stack.pop_back(); - visitState[u] = 2; - return false; - }; - for (std::size_t i = 0; i < state.packages.size() && cycle.empty(); ++i) - if (visitState[i] == 0) (void)visit(visit, i); - if (!cycle.empty()) { + mcpp::graph::AdjacencyList deps(state.packages.size()); + for (auto const& e : state.dependencyEdges) { + if (e.consumerPackageIndex >= deps.size()) continue; + if (e.dependencyPackageIndex >= state.packages.size()) continue; + deps[e.consumerPackageIndex].push_back(e.dependencyPackageIndex); + } + if (auto order = mcpp::graph::topological_order(deps); !order) { std::string path; - for (auto p : cycle) { + for (auto p : order.error().cycle) { if (!path.empty()) path += " -> "; path += std::format("'{}'", mcpp::build::qualified_package_name(state.packages[p].manifest)); diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 3b12fbe3..271da635 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -44,6 +44,7 @@ import mcpp.platform.capacity; // the host fallback handed to schedule::decide import mcpp.build.graph_shape; // #407: the graph says which mode wrote it import mcpp.build.runtime_validation; // declared artifact -> identity verdict import mcpp.build.cache_key; +import mcpp.graph; import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against import mcpp.pack.prebuilt; // …and the check itself import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest @@ -1736,7 +1737,6 @@ static std::expected step13_dependency_cache(PrepareState& st std::vector pkgKeys(state.packages.size()); std::vector pkgInputs(state.packages.size(), nlohmann::json::object()); - std::vector keyState(state.packages.size(), 0); // 0 new/1 busy/2 done std::string keyCycleError; // Does this package's own transitive upstream contain anything that is // not an immutable index payload? If so it cannot be cached either, even @@ -1751,19 +1751,28 @@ static std::expected step13_dependency_cache(PrepareState& st // enforced structurally anyway, because "unreachable today" is how the // transitive path-dep leak got in. std::vector localTaint(state.packages.size(), 0); - auto compute_key = [&](auto&& self, std::size_t idx) -> const std::string& { - static const std::string kEmpty; - if (keyState[idx] == 2) return pkgKeys[idx]; - if (keyState[idx] == 1) { - if (keyCycleError.empty()) { - keyCycleError = std::format( - "dependency cycle through package '{}' while computing " - "its build-cache key", state.packages[idx].manifest.package.name); - } - return kEmpty; - } - keyState[idx] = 1; + // Axis F is each direct dependency's OWN key, which forces a + // bottom-up order: `dependencyEdges` is a DAG (the modgraph + // validator rejects cycles among module imports, but this is the + // package graph, which has no such upstream guard), so the fold + // below walks `mcpp::graph::topological_order` — dependencies before + // dependents — instead of recursing, with an explicit cycle check up + // front standing in for the old in-progress guard. + mcpp::graph::AdjacencyList pkgDeps(state.packages.size()); + for (auto& e : state.dependencyEdges) + if (e.consumerPackageIndex < pkgDeps.size()) + pkgDeps[e.consumerPackageIndex].push_back(e.dependencyPackageIndex); + auto keyOrder = mcpp::graph::topological_order(pkgDeps); + if (!keyOrder) { + keyCycleError = std::format( + "dependency cycle through package '{}' while computing " + "its build-cache key", + state.packages[keyOrder.error().cycle.front()].manifest.package.name); + return std::unexpected(keyCycleError); + } + + for (auto idx : *keyOrder) { ck::PackageAxes pa; if (idx > 0 && idx - 1 < state.dep_cache_identities.size()) { pa.indexName = state.dep_cache_identities[idx - 1].indexName; @@ -1794,7 +1803,9 @@ static std::expected step13_dependency_cache(PrepareState& st if (!selfIsIndex) localTaint[idx] = 1; for (auto& e : state.dependencyEdges) { if (e.consumerPackageIndex != idx) continue; - auto& up = self(self, e.dependencyPackageIndex); + // `idx`'s dependencies precede it in `keyOrder`, so their + // keys and taint are already folded in below. + auto const& up = pkgKeys[e.dependencyPackageIndex]; if (!up.empty()) pa.upstreamKeys.push_back(up); if (localTaint[e.dependencyPackageIndex]) localTaint[idx] = 1; for (auto& f : e.requestedFeatures) pa.features.push_back(f); @@ -1807,14 +1818,9 @@ static std::expected step13_dependency_cache(PrepareState& st pa.features.erase(std::unique(pa.features.begin(), pa.features.end()), pa.features.end()); - pkgKeys[idx] = ck::key_hex(axes, pa); - pkgInputs[idx] = ck::to_json(axes, pa); - keyState[idx] = 2; - return pkgKeys[idx]; - }; - for (std::size_t i = 0; i < state.packages.size(); ++i) - (void)compute_key(compute_key, i); - if (!keyCycleError.empty()) return std::unexpected(keyCycleError); + pkgKeys[idx] = ck::key_hex(axes, pa); + pkgInputs[idx] = ck::to_json(axes, pa); + } for (std::size_t i = 1; i < state.packages.size(); ++i) { // skip [0] = main const auto& pkgRoot = state.packages[i]; diff --git a/src/modgraph/graph.cppm b/src/modgraph/graph.cppm index 4c391333..4944c863 100644 --- a/src/modgraph/graph.cppm +++ b/src/modgraph/graph.cppm @@ -4,6 +4,7 @@ export module mcpp.modgraph.graph; import std; import mcpp.source_kind; +import mcpp.graph; export namespace mcpp::modgraph { @@ -121,7 +122,8 @@ struct Graph { }; // Topological order: returns indices of units in producer-before-consumer order. -// Returns std::unexpected with the cycle if any. +// Returns std::unexpected with the cycle if any, as the ordered path that +// walks it (see mcpp.graph) rather than merely the units left over. struct CycleError { std::vector cycle; }; @@ -132,38 +134,15 @@ std::expected, CycleError> topo_sort(const Graph& g); namespace mcpp::modgraph { std::expected, CycleError> topo_sort(const Graph& g) { - std::vector indeg(g.units.size(), 0); - std::vector> adj(g.units.size()); - for (auto [c, p] : g.edges) { - // edge means: consumer depends on producer. So producer must come first. - // indegree of consumer counts unmet producer dependencies. - indeg[c]++; - adj[p].push_back(c); - } + // g.edges: (consumer, producer) pairs, "consumer depends on producer". + // mcpp.graph's adjacency direction is the same: deps[u] lists what u + // depends on, so an edge translates straight across with no inversion. + mcpp::graph::AdjacencyList deps(g.units.size()); + for (auto [c, p] : g.edges) deps[c].push_back(p); - std::vector order; - order.reserve(g.units.size()); - std::vector queue; - for (std::size_t i = 0; i < indeg.size(); ++i) { - if (indeg[i] == 0) queue.push_back(i); - } - while (!queue.empty()) { - std::size_t u = queue.back(); - queue.pop_back(); - order.push_back(u); - for (auto v : adj[u]) { - if (--indeg[v] == 0) queue.push_back(v); - } - } - if (order.size() != g.units.size()) { - // Cycle remains. Report units still with positive indegree. - CycleError err; - for (std::size_t i = 0; i < indeg.size(); ++i) { - if (indeg[i] > 0) err.cycle.push_back(i); - } - return std::unexpected(err); - } - return order; + auto order = mcpp::graph::topological_order(deps); + if (!order) return std::unexpected(CycleError{std::move(order.error().cycle)}); + return std::move(*order); } } // namespace mcpp::modgraph diff --git a/src/modgraph/validate.cppm b/src/modgraph/validate.cppm index ae604c38..358fada8 100644 --- a/src/modgraph/validate.cppm +++ b/src/modgraph/validate.cppm @@ -217,14 +217,19 @@ ValidateReport validate(const Graph& g, // 3. Topology auto topo = topo_sort(g); if (!topo) { - std::string names; + // The cycle is now the actual ring `topo_sort` walked (mcpp.graph), + // not merely the units it could not order, so the message can name + // the path rather than an unordered set of suspects. + std::string ring; + bool first = true; for (auto i : topo.error().cycle) { - if (i < g.units.size()) { - names += g.units[i].path.string() + " "; - } + if (i >= g.units.size()) continue; + if (!first) ring += " -> "; + first = false; + ring += g.units[i].path.string(); } r.errors.push_back({{}, - std::format("circular module dependency among: {}", names)}); + std::format("circular module dependency: {}", ring)}); } else { r.topoOrder = std::move(*topo); } From 796dbda5825420d1e1799437a8b7384bf4811494 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 05:07:16 +0800 Subject: [PATCH 03/11] Workspace build graph, second part: member link groups and product directories, the workspace fast path, the scheduler, the lock, clean --stale, tests, documentation and the 2026.9.29.1 version --- .github/actions/bootstrap-mcpp/action.yml | 2 +- .github/actions/setup-macos-llvm/action.yml | 2 +- .github/workflows/bootstrap-macos.yml | 2 +- .github/workflows/ci-fresh-install.yml | 6 +- .github/workflows/ci-linux-e2e.yml | 2 +- .github/workflows/cross-build-test.yml | 4 +- .github/workflows/release.yml | 14 +- CHANGELOG.md | 82 +++ docs/07-workspace.md | 84 ++- docs/zh/07-workspace.md | 63 +- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/execute.cppm | 248 +++++++- src/build/graph_shape.cppm | 59 +- src/build/ninja_backend.cppm | 58 +- src/build/plan.cppm | 573 +++++++++++++----- src/build/prepare.cppm | 30 + src/build/prepare/graph.cpp | 25 +- src/build/prepare/manifest.cpp | 12 +- src/build/prepare/plan.cpp | 70 ++- src/build/prepare/records.cpp | 49 +- src/build/prepare/state.cppm | 8 + src/cli/cmd_build.cppm | 226 +++++-- src/pack/library_pipeline.cppm | 12 +- src/pack/pipeline.cppm | 10 +- src/project.cppm | 10 +- src/xlings/xlings.cppm | 2 +- tests/e2e/120_ws_root_indices.sh | 10 +- tests/e2e/211_configure_only_cdb.sh | 27 +- .../e2e/218_module_extensions_graph_shape.sh | 31 +- tests/e2e/321_workspace_inheritance.sh | 22 +- tests/e2e/35_workspace.sh | 11 +- .../e2e/770_workspace_member_as_dependency.sh | 10 +- .../806_dash_p_resolves_the_package_first.sh | 24 +- ...orkspace_is_one_graph_per_configuration.sh | 150 +++++ ...ace_plans_once_and_replays_in_one_check.sh | 87 +++ ...35_a_shared_library_is_placed_by_a_link.sh | 74 +++ tests/unit/test_graph_shape.cpp | 21 +- tests/unit/test_workspace_plan.cpp | 103 ++++ 39 files changed, 1902 insertions(+), 325 deletions(-) create mode 100755 tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh create mode 100755 tests/e2e/834_a_workspace_plans_once_and_replays_in_one_check.sh create mode 100755 tests/e2e/835_a_shared_library_is_placed_by_a_link.sh create mode 100644 tests/unit/test_workspace_plan.cpp diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 366a9efb..5174cce2 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.9.28.2' + default: '2026.9.29.1' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index 6246af6f..3bc4d66e 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.9.28.2' + default: '2026.9.29.1' image: description: > The runner label the job runs on (macos-15, xcode-27). It is part of the diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index e5c423aa..fdb2d72a 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index b129839f..9e572f87 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.29.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -315,7 +315,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.29.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -416,7 +416,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.29.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 21492aa8..e0ad446f 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -384,7 +384,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.28.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.29.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 38078cd8..16e253b6 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -135,7 +135,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -289,7 +289,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d84401a5..68710368 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -104,7 +104,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -322,7 +322,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -393,7 +393,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.9.28.2-linux-aarch64.tar.gz" + XLA="xlings-2026.9.29.1-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -402,9 +402,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.9.28.2/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.9.29.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.9.28.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.9.29.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -482,7 +482,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -665,7 +665,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.28.2' + XLINGS_VERSION: '2026.9.29.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f74104c..c842f9a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,88 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.9.29.1] - 2026-09-29 + +This release builds a workspace as one graph per configuration. The selected +members and everything they depend on are planned together under a virtual +root, so a member that several members use is compiled once; each member's +products are placed in its own product directory. It also fixes the planning +regression of 2026.9.28.3. The design, its readings and the task plan are in +`.agents/docs/2026-09-29-workspace-build-graph-design.md`. The xlings pin moves +to 2026.9.29.1. + +### Fixed + +- **Planning a workspace no longer grows with the length of its dependency + chains (2026.9.28.3 only).** E1 planned a shared member as the root of a + nested build, and the nested build met the same condition again, so a chain + of n members cost 2^n plans: a nine-member workspace with nothing to build + took 79 s instead of 6.3 s, and a chain of five libraries and a program took + 36.8 s for `--workspace`. E1 is removed. The same chain now plans once + (0.5 s), and with nothing changed `--workspace`, `-p app` and a build inside + the member are each answered by the fast path in a few milliseconds (e2e + 834). + +### Behaviour changes + +- **A workspace is one graph per configuration.** A command on a workspace + plans its selected members together: `--workspace`, and a virtual root + without `-p`, select every member (a rooted workspace's own package + included); `-p X`, and a command in X's directory, select X; a command at a + rooted workspace's root selects its own package. Members that share their + toolchain request, target, C++ standard and the other values that apply to a + whole graph are one plan, with one `build.ninja`; members that differ are + separate plans, built at the same time under a static share of the jobs. A + member used by several members is compiled once (e2e 833). +- **Build directories are at the workspace root.** A member builds in + `/target///` instead of its own `target/`. + Its programs and shared libraries are in its product directory, + `bin//` (`bin/./` when two members share a + name; a rooted workspace's own package keeps `bin/`), with the shared + libraries and runtime files its programs load beside them. The first build + after the upgrade is a full build; `mcpp clean --stale` removes the build + directories members held under their own `target/`. A member's build program + still writes to `/target/.build-mcpp/`. +- **`-p X` plans X and what X reaches, in the shared build directory.** + `mcpp build --workspace` followed by `mcpp build -p X` compiles nothing; a + package is compiled again only when its active features differ between the + two commands. +- **The directory name is the configuration.** It hashes the toolchain, the + target, the standard library, the runtime contract, the C++ standard, the + dialect flags, the profile and the other values every node of one graph + shares. A package's own `cflags`, `cxxflags`, `ldflags`, defines, sources and + include directories reach its commands instead: editing them recompiles that + package and what imports it, in the same directory, rather than moving the + whole graph to a new one. This holds for single-package projects as well. +- **One lock and one compile database per workspace.** `mcpp.lock` is at the + workspace root, as it already was for a rooted workspace; `--workspace` + writes the whole record and `-p X` updates the entries of X's graph. Where + the workspace has no lock yet, a selected member's own lock supplies the + git commits. The workspace root's `compile_commands.json` covers every member + that has been built or configured. +- **A member keeps its duties as the project being developed.** Its targets + are built, its `[dev-dependencies]` are loaded under `mcpp test`, its + `[hooks]` run around the build, its `[xlings]` entries with `when = "dev"` are + installed, and its build program runs after every dependency's program with + the graph document of what it reaches (in which it is the `root`) and the + dependencies' link forms. `--features f` activates `f` in each selected + member that declares it and is refused when none does. +- **Each member links its own closure.** A member's programs link the flags of + the packages they reach and place the runtime files of those packages, never + another member's. +- **Shared libraries are placed by a hard link.** A shared library placed + beside several programs is one file with several names where the file system + supports links, and a copy elsewhere. Other deployed files are copied, since + a program may write a file beside itself (e2e 835). +- **The fast path records the request.** `build.ninja` states the workspace + members and the features it was planned for, and a fast path replays it only + for the same request; one record is kept per selection and configuration. +- **`mcpp.graph`.** One module holds the engine's graph algorithms: a stable + topological order with the cycle reported as its path, dependency levels and + the transitive closure. Module order, host-module order, unit order by + imports, the dependency closure, the package-cycle check and the build-key + fold use it; a cycle of modules is now reported as `a -> b -> a`. + ## [2026.9.28.3] - 2026-09-28 This release implements mcpp's part of the #734 design: the three layers of the diff --git a/docs/07-workspace.md b/docs/07-workspace.md index db87a023..83cbf601 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -334,8 +334,8 @@ mcpp test --workspace # test every member (one report per member; continue At a **virtual** workspace root (only `[workspace]`, no `[package]`), bare `mcpp build` / `mcpp test` act on **all** members. At a **rooted** workspace -(`[package]` + `[workspace]`), they act on the root package; use `--workspace` to -include all members. `mcpp test --workspace` builds + runs each member's +(`[package]` + `[workspace]`), they act on the root package; `--workspace` +acts on the root package and every member. `mcpp test --workspace` builds + runs each member's `tests/**/*.cpp` independently — discovery is scoped per member, so two members may each have a `tests/main.cpp` without colliding. @@ -346,7 +346,7 @@ cd libs/http mcpp build # auto-detects the workspace and builds the current member ``` -mcpp searches upward from the current directory; if it finds an `mcpp.toml` containing `[workspace]` and the current directory is listed in `members`, it automatically enters workspace mode and inherits the workspace configuration. +mcpp searches upward from the current directory; if it finds an `mcpp.toml` containing `[workspace]` and the current directory is listed in `members`, it automatically enters workspace mode and inherits the workspace configuration. The command then acts as `mcpp build -p ` at the workspace root: it builds in the workspace's build directory (§6). ### 5.3 The `-p, --package` Option @@ -413,27 +413,31 @@ fan-out continues; a timed-out build fails that member; `--workspace-timeout` st the fan-out and lists what did not run instead of leaving the CI job to kill the process (which discards everything it had to say). -### 5.4 A member used by other members is built once (mcpp 2026.9.28.3+) - -A member that other members use as a path dependency is built once, in its own -directory, and every member that uses it takes its objects and module -interfaces from there. A `--workspace` build and separate `-p` builds of two -programs therefore compile a shared library member once, where each program -used to compile it again in its own directory. - -- **Staleness is the member's own.** Before a consumer builds, the member's - own build runs, and its ninja decides what is stale, including an input - outside the member's root (a header under `../3rdParty`). -- **Equal build keys are the condition.** The member's build key in the - consumer's graph must equal its key as the root of its own build; the key - covers the toolchain, the flags, the profile and the features. A member that - a consumer builds differently (another feature set, for example) is compiled - in that consumer's graph, as before; `-v` states the input that differs. -- **One build at a time.** Two consumers built at once take turns on the - member's directory. -- **Scope.** The rule applies in the default cache mode (`--cache off` compiles - everything in the graph that asks for it) and to members only; a path - dependency outside the workspace has one consumer and keeps its behaviour. +### 5.4 One graph per configuration (mcpp 2026.9.29.1+) + +A command on a workspace plans its members together: the selected members and +everything they depend on are one build graph, with one `build.ninja`, and a +member that several members use is compiled once. + +- **Configurations.** Members are built in one graph when they share their + toolchain request, target, C++ standard, `dialect_cxxflags`, C++ runtime, + profile and the other `[build]` values that apply to a whole graph. Members + that differ in one of them are built in separate graphs, at the same time, + sharing the command's jobs. +- **Selection.** `--workspace`, and a virtual root without `-p`, select every + member. `-p X`, and a command run in X's directory, plan X and what X + reaches. The two share the build directory, so `mcpp build --workspace` + followed by `mcpp build -p X` compiles nothing, and a package is compiled + again only when its active features differ between the two commands. +- **Flags.** A member's `cflags`, `cxxflags`, `ldflags` and defines apply to + that member's commands. Editing them recompiles that member and what + imports it; the build directory stays the same. +- **Features.** `--features f` activates `f` in each selected member that + declares it, and is refused when no selected member declares it. +- **Hooks.** The `[hooks]` of every selected member run around the build, in + member order. +- **No-op builds.** A command repeated with nothing changed is answered by one + check per configuration, without planning. ## 6. Directory Layout @@ -458,7 +462,37 @@ myproject/ └── main.cpp # import myproject.http; ``` -Each member's build artifacts live under its own `target/` subdirectory. +A workspace builds at its root (2026.9.29.1+): + +``` +myproject/ +├── mcpp.lock # one lock for the workspace +└── target/// + ├── build.ninja, compile_commands.json # one graph and one database per configuration + ├── obj// # intermediate objects of every package + └── bin/ + ├── server/ # a member's products: bin// + │ ├── server + │ └── libfoo.so # shared libraries and runtime files it loads + └── ... +``` + +- A member's programs and shared libraries are in its **product directory**, + `bin//`, with the shared libraries, DLLs and deployed files + its programs load placed beside them. Two members with the same package + name use `bin/./`. A rooted workspace's own package keeps + `bin/`. +- A shared library placed in several product directories is one file with + several names where the file system supports hard links; elsewhere it is + copied. +- `compile_commands.json` at the workspace root covers every member that has + been built or configured. +- `mcpp.lock` at the workspace root records the resolution of every member. + `mcpp build --workspace` writes the whole record; `mcpp build -p X` updates + the entries of X's graph. +- A member's build program writes to `/target/.build-mcpp/`. +- Build directories a member held under its own `target/` with an earlier mcpp + are not read; `mcpp clean --stale` removes them. A project outside the workspace reaches a member of a git-hosted workspace by the member's identity: `myproject.http = { git = "...", rev = "..." }` selects diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index 0f1ecf65..933b7021 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -316,7 +316,7 @@ mcpp test --workspace # test every member (one report per member; continue 在**虚拟**工作空间根（只有 `[workspace]`、没有 `[package]`）下，裸 `mcpp build` / `mcpp test` 作用于**全体**成员；在**带根包**的工作空间（`[package]` + -`[workspace]`）下，两者作用于根包，用 `--workspace` 才纳入全体成员。 +`[workspace]`）下，两者作用于根包；`--workspace` 作用于根包与全体成员。 `mcpp test --workspace` 独立构建并运行每个成员的 `tests/**/*.cpp`——发现按成员 隔离，因此两个成员各有一个 `tests/main.cpp` 也不冲突。 @@ -328,7 +328,8 @@ mcpp build # auto-detects the workspace and builds the current ``` mcpp 从当前目录向上搜索；若发现某个 `mcpp.toml` 含 `[workspace]` 且当前目录在其 -`members` 列表中，则自动进入工作空间模式并继承工作空间配置。 +`members` 列表中，则自动进入工作空间模式并继承工作空间配置。此时命令等同于在工作空间根 +执行 `mcpp build -p <当前成员>`，在工作空间的构建目录中构建（§6）。 ### 5.3 `-p, --package` 选项 @@ -386,20 +387,24 @@ mcpp test --workspace --workspace-timeout 1800 # whole fan-out (default 0 = no `--workspace-timeout` 停止扇出并列出未运行的成员，而不是把进程留给 CI 去 kill—— 那样会把进程本该说出的话一并丢掉。 -### 5.4 被其他成员使用的成员只构建一次（mcpp 2026.9.28.3+） - -被其他成员以 path 依赖使用的成员只构建一次，在它自己的目录中构建，每个使用它的成员从那里取得 -其对象与模块接口。因此一次 `--workspace` 构建，或分别对两个程序执行的 `-p` 构建，只编译共享的 -库成员一次；此前每个程序都在自己的目录中再编译一遍。 - -- **是否过期由成员自己判断。** 消费方构建之前，成员自己的构建先运行，由它的 ninja 判断哪些 - 产物过期，包括成员根目录之外的输入（例如 `../3rdParty` 下的头文件）。 -- **条件是构建键相等。** 成员在消费方图中的构建键必须等于它作为自身构建的根时的键；键覆盖 - 工具链、编译参数、配置档与 feature。消费方以不同方式构建的成员（例如启用了另一组 feature） - 仍在该消费方的图中编译，与此前相同；`-v` 会写出不同的那一项输入。 -- **同一时间只有一个构建。** 同时构建的两个消费方轮流使用成员的目录。 -- **适用范围。** 该规则在默认缓存模式下生效（`--cache off` 在发起构建的图中编译全部内容）， - 且只适用于成员；工作区之外的 path 依赖只有一个消费方，行为不变。 +### 5.4 每个配置一张构建图（mcpp 2026.9.29.1+） + +对工作空间的命令把成员放在一起规划：被选中的成员及其全部依赖构成一张构建图，只有一个 +`build.ninja`，被多个成员使用的成员只编译一次。 + +- **配置。** 工具链请求、目标、C++ 标准、`dialect_cxxflags`、C++ 运行时、配置档，以及其他 + 作用于整张图的 `[build]` 值都相同的成员在同一张图中构建。其中任一项不同的成员在各自的图中 + 构建，这些图同时进行，共享命令的并行任务数。 +- **选择。** `--workspace`，以及虚拟工作空间根下不带 `-p` 的命令，选中全体成员；`-p X` 与在 + X 的目录中执行的命令规划 X 及其所依赖的一切。两者共用构建目录，因此先执行 + `mcpp build --workspace` 再执行 `mcpp build -p X` 不编译任何内容；只有当某个包在两条命令中 + 启用的 feature 不同时，它才会被重新编译。 +- **编译参数。** 成员的 `cflags`、`cxxflags`、`ldflags` 与 defines 作用于该成员自己的命令。 + 修改它们会重新编译该成员以及导入它的单元，构建目录保持不变。 +- **Feature。** `--features f` 在每个声明了 `f` 的被选成员中启用它；没有被选成员声明它时， + 命令被拒绝。 +- **Hooks。** 每个被选成员的 `[hooks]` 按成员顺序在构建前后运行。 +- **无事可做的构建。** 在没有任何改动时重复执行的命令，每个配置只做一次检查，不重新规划。 ## 6. 目录布局 @@ -424,7 +429,31 @@ myproject/ └── main.cpp # import myproject.http; ``` -各成员的构建产物存在各自的 `target/` 子目录下。 +工作空间在其根目录构建（2026.9.29.1+）： + +``` +myproject/ +├── mcpp.lock # 整个工作空间一份锁文件 +└── target/// + ├── build.ninja, compile_commands.json # 每个配置一张图、一份数据库 + ├── obj// # 各包的中间产物 + └── bin/ + ├── server/ # 成员的产物：bin/<包名>/ + │ ├── server + │ └── libfoo.so # 其程序加载的共享库与运行时文件 + └── ... +``` + +- 成员的程序与共享库位于它的**产物目录** `bin/<包名>/`，其程序加载的共享库、DLL 与部署文件 + 放在旁边。两个成员包名相同时使用 `bin/./`。带根包的工作空间中，根包仍使用 + `bin/`。 +- 放入多个产物目录的共享库，在支持硬链接的文件系统上是同一个文件的多个名字，其他情况下被复制。 +- 工作空间根的 `compile_commands.json` 覆盖所有已构建或已配置的成员。 +- 工作空间根的 `mcpp.lock` 记录全体成员的解析结果。`mcpp build --workspace` 写入完整记录； + `mcpp build -p X` 更新 X 所在图中的条目。 +- 成员的构建程序写入 `/target/.build-mcpp/`。 +- 早期版本的 mcpp 在成员自己的 `target/` 下留下的构建目录不再被读取；`mcpp clean --stale` + 会删除它们。 工作空间之外的工程，以成员身份引用托管在 git 上的工作空间中的一个成员： `myproject.http = { git = "...", rev = "..." }` 会在根 manifest 的 `members` diff --git a/mcpp.toml b/mcpp.toml index 838e1ad8..3c737776 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.28.3" +version = "2026.9.29.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index d8849330..40e106fb 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.28.3"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.29.1"; } // namespace mcpp diff --git a/src/build/execute.cppm b/src/build/execute.cppm index ef934fca..7fc35ea7 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -96,7 +96,7 @@ constexpr std::string_view kBuildCacheFile = "target/.build_cache"; // working set is now targets × profiles rather than targets alone — 4 was // enough for one profile, not for a dev/release/dist rotation across a host // and a cross target. -constexpr int kBuildCacheMaxEntries = 8; +constexpr int kBuildCacheMaxEntries = 32; // P3: one entry per (target, fingerprint) pair. // THE INPUTS THAT CHOOSE A TOOLCHAIN AND ARE NOT IN THE MANIFEST. @@ -246,6 +246,13 @@ struct BuildCacheEntry { // the list for the reason `depSourceRootsRecorded` is. std::vector xlingsPayloads; bool xlingsPayloadsRecorded = false; + // The workspace selection of the command that wrote the entry, and the + // members of the entry's own plan, each joined by a unit separator + // (workspace design 2026-09-29 §15). Empty outside a workspace. One + // command on a workspace writes one entry per configuration group, and a + // fast path matches every entry of its selection. + std::string selection; + std::string group; }; std::vector read_build_cache(const std::filesystem::path& projectRoot) { @@ -383,6 +390,15 @@ std::vector read_build_cache(const std::filesystem::path& proje e.xlingsPayloadsRecorded = true; haveNextLine = static_cast(std::getline(f, line)); } + // Optional `selection=` and `group=`; absent outside a workspace. + if (haveNextLine && line.starts_with("selection=")) { + e.selection = line.substr(10); + haveNextLine = static_cast(std::getline(f, line)); + } + if (haveNextLine && line.starts_with("group=")) { + e.group = line.substr(6); + haveNextLine = static_cast(std::getline(f, line)); + } entries.push_back(std::move(e)); if (!haveNextLine || line.empty()) break; } @@ -431,7 +447,14 @@ void write_build_cache(const std::filesystem::path& projectRoot, bool runTierPending = false, const std::string& features = {}, const std::string& toolchainRequest = {}, - std::vector xlingsPayloads = {}) { + std::vector xlingsPayloads = {}, + const std::string& selection = {}, + const std::string& group = {}) { + // The configuration groups of one workspace command build at the same + // time (workspace design 2026-09-29 §6) and record into one file; the + // read, the edit and the write are one step. + static std::mutex cacheWrite; + std::lock_guard lock(cacheWrite); auto path = projectRoot / kBuildCacheFile; auto entries = read_build_cache(projectRoot); @@ -440,7 +463,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, // switching profiles back and forth could never be incremental AND the // surviving entry pointed at the other profile's build dir. std::erase_if(entries, [&](const BuildCacheEntry& e) { - return e.targetTriple == targetTriple && e.profile == profile; + return e.targetTriple == targetTriple && e.profile == profile + && e.selection == selection && e.group == group; }); // Insert at front (MRU). @@ -459,6 +483,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, newEntry.toolchainRecorded = true; newEntry.xlingsPayloads = std::move(xlingsPayloads); newEntry.xlingsPayloadsRecorded = true; + newEntry.selection = selection; + newEntry.group = group; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -505,6 +531,10 @@ void write_build_cache_entries(const std::filesystem::path& path, f << "toolchain=" << e.toolchainRequest << '\n'; f << "xlingsPayloads=" << e.xlingsPayloads.size() << '\n'; for (auto& p : e.xlingsPayloads) f << p << '\n'; + if (!e.selection.empty() || !e.group.empty()) { + f << "selection=" << e.selection << '\n'; + f << "group=" << e.group << '\n'; + } } } @@ -889,6 +919,11 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, for (auto& note : ctx.manifest.inferredNotes) { mcpp::ui::status("Inferred", note); } + // A plan of one workspace member reports what was inferred about the + // member, as its own build did (§15). + if (ctx.workspaceMembers.size() == 1) + for (auto& note : ctx.workspaceMembers.front().manifest.inferredNotes) + mcpp::ui::status("Inferred", note); // Announce the package being built (and any deps). A dep served from the // global cache says "Cached" and HOW MANY translation units that saved. The @@ -900,9 +935,12 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, for (auto& dep : ctx.cachedDeps) cachedUnits[dep.name] = dep.units; std::set announced; announced.insert(ctx.manifest.package.name); - mcpp::ui::status("Compiling", - std::format("{} v{} (.)", - ctx.manifest.package.name, ctx.manifest.package.version)); + // A workspace plan's virtual root compiles nothing; its members are + // announced as its dependencies, by their directories (§15). + if (!ctx.manifest.package.virtualRoot) + mcpp::ui::status("Compiling", + std::format("{} v{} (.)", + ctx.manifest.package.name, ctx.manifest.package.version)); for (auto& [name, spec] : ctx.manifest.dependencies) { if (announced.contains(name)) continue; announced.insert(name); @@ -928,7 +966,14 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, return std::format("(git {} {})", spec.gitRefKind.empty() ? "rev" : spec.gitRefKind, ref); }; - std::string ver = spec.isPath() + auto memberDir = [&] { + auto rel = std::filesystem::path(spec.path).lexically_normal() + .lexically_relative(ctx.projectRoot.lexically_normal()).generic_string(); + return std::format("({})", rel.empty() ? std::string(".") : rel); + }; + std::string ver = spec.workspaceMember + ? memberDir() + : spec.isPath() ? "(path)" : spec.isGit() ? gitReference() @@ -1007,6 +1052,7 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, for (auto& e : entries) { if (e.targetTriple == targetOverride && e.profile == ctx.profile && e.cacheMode == cache_mode_name(ctx.cacheMode) + && e.selection == ctx.workspaceRequest && e.group == ctx.workspaceGroup && !e.fingerprint.empty()) { auto newFp = ctx.outputDir.filename().string(); if (e.fingerprint != newFp) { @@ -1060,7 +1106,8 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, for (auto const& p : ctx.xlingsPayloads) v.push_back(p.generic_string()); return v; - }()); + }(), + ctx.workspaceRequest, ctx.workspaceGroup); } // The one place the --strict policy is settled. Degradations reported by @@ -1556,7 +1603,8 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo for (auto& e : entries) { if (e.targetTriple == currentTarget && e.profile == want->profile && e.cacheMode == want->cacheMode && e.features == want->features - && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest) { + && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest + && e.selection.empty()) { match = &e; break; } @@ -1599,6 +1647,10 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo // printed `Finished`; and a broken file under tests/ (never scanned here) // failed a plain `mcpp build` outright. if (!mcpp::build::is_plain_build_graph(ninjaPath)) return fast_path_declined("build", "build.ninja was written by another mode (test, pack or a named target)"); + // What the graph was planned for (workspace design 2026-09-29 §3): the + // features no longer name the directory, so the graph says which it has. + if (mcpp::build::read_request(ninjaPath) != mcpp::build::request_tag({}, want->features)) + return fast_path_declined("build", "build.ninja was written for another request (features or workspace members)"); auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); if (ec) return fast_path_declined("build", "the time of build.ninja cannot be read"); @@ -1650,6 +1702,111 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo return 0; } +// The fast path of a command on a workspace (workspace design 2026-09-29 +// §15): one record per configuration group of the command's selection, each +// checked the way `try_fast_build` checks a project's record, and the groups' +// graphs replayed when every one is current. A build with nothing to do then +// costs one check per group instead of a plan. The freshness sweep covers the +// trees of the members and of the `path` dependencies each record names, not +// the workspace's whole tree. +export std::optional try_fast_workspace_build( + const std::filesystem::path& wsRoot, + const std::vector>& groups, + bool verbose, bool no_cache) { + if (no_cache) return fast_path_declined("workspace", "the build cache is off (--no-cache)"); + if (mcpp::platform::env::get("MCPP_LOCKED").value_or("") == "1") + return fast_path_declined("workspace", "MCPP_LOCKED=1 asks for a locked resolution"); + auto join = [](const std::vector& v) { + std::string out; + for (auto const& x : v) { if (!out.empty()) out += '\x1e'; out += x; } + return out; + }; + std::vector request; + for (auto const& g : groups) request.insert(request.end(), g.begin(), g.end()); + const auto selection = join(request); + const auto entries = read_build_cache(wsRoot); + + struct Ready { + std::filesystem::path outputDir; + std::string ninjaProgram, runtimeEnvKey, runtimeEnvValue; + mcpp::build::runtime_validation::ArtifactSnapshot validated; + }; + std::vector ready; + std::string profile; + std::error_code ec; + const auto wsToml = std::filesystem::last_write_time(wsRoot / "mcpp.toml", ec); + if (ec) return fast_path_declined("workspace", "the workspace's mcpp.toml cannot be read"); + for (auto const& g : groups) { + auto want = fast_path_identity(wsRoot / g.front()); + if (!want) return fast_path_declined("workspace", "a member's manifest could not be read"); + for (auto const& mp : g) { + auto w = mp == g.front() ? want : fast_path_identity(wsRoot / mp); + if (!w) return fast_path_declined("workspace", "a member's manifest could not be read"); + if (w->hooksActive) + return fast_path_declined("workspace", "a member declares [hooks], which run on every build"); + } + const auto group = join(g); + const BuildCacheEntry* match = nullptr; + for (auto const& e : entries) + if (e.targetTriple.empty() && e.profile == want->profile + && e.cacheMode == want->cacheMode && e.features.empty() + && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest + && e.selection == selection && e.group == group) { match = &e; break; } + if (!match) return fast_path_declined("workspace", "no recorded build matches this selection"); + if (!match->runtimeBinding || match->runtimeEnvKey.empty()) + return fast_path_declined("workspace", "the recorded build predates the runtime binding"); + const std::filesystem::path outputDir(match->outputDir); + if (!match->fingerprint.empty() && outputDir.filename().string() != match->fingerprint) + return fast_path_declined("workspace", "the recorded build directory is not the one for this fingerprint"); + const auto ninjaPath = outputDir / "build.ninja"; + if (!std::filesystem::exists(ninjaPath, ec)) + return fast_path_declined("workspace", "build.ninja does not exist"); + if (!mcpp::build::is_plain_build_graph(ninjaPath)) + return fast_path_declined("workspace", "build.ninja was written by another mode (test, pack or a named target)"); + if (mcpp::build::read_request(ninjaPath) != mcpp::build::request_tag(group, {})) + return fast_path_declined("workspace", "build.ninja was written for another request (features or workspace members)"); + const auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); + if (ec) return fast_path_declined("workspace", "the time of build.ninja cannot be read"); + const auto runtimeTime = std::filesystem::last_write_time( + match->runtimeBinding->subosDir / ".xlings.json", ec); + if (ec || runtimeTime > ninjaTime) + return fast_path_declined("workspace", "the runtime's .xlings.json is newer than build.ninja"); + if (wsToml > ninjaTime) + return fast_path_declined("workspace", "the workspace's mcpp.toml is newer than build.ninja"); + if (!match->depSourceRootsRecorded + || dep_sources_newer_than(match->depSourceRoots, ninjaTime, want->extTable)) + return fast_path_declined("workspace", "a member's or a path dependency's manifest or source is newer than build.ninja"); + if (!xlings_payloads_present(*match)) + return fast_path_declined("workspace", "a recorded xlings payload is missing"); + auto validated = mcpp::build::runtime_validation::validated_artifact_snapshot( + outputDir, *match->runtimeBinding); + if (!validated) + return fast_path_declined("workspace", "no validated artifact snapshot is recorded for this build"); + auto ninjaProgram = match->ninjaProgram; + if (ninjaProgram.size() >= 2 && ninjaProgram.front() == '\'' + && ninjaProgram.back() == '\'') + ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); + ready.push_back({outputDir, ninjaProgram, match->runtimeEnvKey, + match->runtimeEnvValue, std::move(*validated)}); + profile = want->profile; + } + + std::chrono::milliseconds total{}; + for (auto& r : ready) { + restore_root_compile_commands(wsRoot, r.outputDir); + std::chrono::milliseconds elapsed{}; + auto rc = run_ninja_fast(r.ninjaProgram, r.outputDir, r.outputDir / "build.ninja", + verbose, r.runtimeEnvKey, r.runtimeEnvValue, &elapsed); + if (!rc) return fast_path_declined("workspace", "ninja reported a stale graph"); + if (*rc != 0) return rc; + if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged(r.validated)) + return fast_path_declined("workspace", "ninja relinked an artifact, whose closure the full path validates"); + total += elapsed; + } + mcpp::ui::finished(profile, total); + return 0; +} + // mcpp#225 (E2): `mcpp run`'s fast path. Mirrors try_fast_build's // fingerprint/freshness gate against the SAME cache entry `mcpp build` // wrote (targetTriple == "" — a HOST build; see the precondition below), then @@ -1695,7 +1852,8 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, for (auto& e : entries) { if (e.targetTriple.empty() && e.profile == want->profile && e.cacheMode == want->cacheMode && e.features == want->features - && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest) { + && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest + && e.selection.empty()) { match = &e; break; } @@ -1759,6 +1917,10 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, // the run target at all, so running ninja against it would report success // and then exec a stale (or absent) binary. if (!mcpp::build::is_plain_build_graph(ninjaPath)) return fast_path_declined("run", "build.ninja was written by another mode (test, pack or a named target)"); + // What the graph was planned for (workspace design 2026-09-29 §3): the + // features no longer name the directory, so the graph says which it has. + if (mcpp::build::read_request(ninjaPath) != mcpp::build::request_tag({}, want->features)) + return fast_path_declined("run", "build.ninja was written for another request (features or workspace members)"); auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); if (ec) return fast_path_declined("run", "the time of build.ninja cannot be read"); @@ -2256,6 +2418,9 @@ export int build_run_target(const std::optional& targetName, if (auto rc = run_build_plan(*ctx, /*verbose=*/false, no_cache, target_triple); rc != 0) return rc; + // The program run is the selected member's (workspace design 2026-09-29 + // §15), with its closure's runtime. + focus_on_member(*ctx); // Find binary target const mcpp::build::LinkUnit* chosen = nullptr; @@ -3203,10 +3368,29 @@ export int clean_project(bool wipe_bmi) { // // With no record at all there is nothing to compare against, and the command // refuses rather than guess. +// +// A WORKSPACE builds at its root (workspace design 2026-09-29 §15), so the +// command acts on the workspace's `target/` from anywhere inside it, and it +// also removes the build directories each member held under its own +// `target/` before that: none of them is read by a workspace build. A +// member's `target/.build-mcpp/`, where its build program's outputs are, is +// kept. export int clean_stale(bool dryRun, std::int64_t keepWithinSecs) { namespace fs = std::filesystem; auto root = mcpp::project::find_manifest_root(fs::current_path()); if (!root) { std::println(stderr, "error: not in an mcpp package"); return 2; } + std::vector memberDirs; + { + auto m = mcpp::manifest::load(*root / "mcpp.toml", {.insideWorkspace = true}); + if (m && !m->workspace.present) { + if (auto ws = mcpp::project::find_workspace_root(*root); !ws.empty()) { + root = ws; + m = mcpp::manifest::load(*root / "mcpp.toml"); + } + } + if (m && m->workspace.present) + for (auto const& mp : m->workspace.members) memberDirs.push_back(*root / mp); + } const fs::path target = *root / "target"; // Triple directory -> the fingerprints recorded under it. One container @@ -3289,6 +3473,50 @@ export int clean_stale(bool dryRun, std::int64_t keepWithinSecs) { return 1; } + // The members' own build directories from before the workspace was the + // unit of build: `/target//<16 hex digits>/`. + auto isFingerprint = [](const std::string& name) { + return name.size() == 16 && std::ranges::all_of(name, [](char c) { + return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f'); + }); + }; + for (auto const& memberDir : memberDirs) { + std::error_code mec; + const auto memberTarget = memberDir / "target"; + if (!fs::is_directory(memberTarget, mec)) continue; + for (fs::directory_iterator tripleIt(memberTarget, mec), end; tripleIt != end; + tripleIt.increment(mec)) { + std::error_code tec; + if (!tripleIt->is_directory(tec) || tec) continue; + if (tripleIt->path().filename().string().starts_with(".")) continue; + for (fs::directory_iterator fpIt(tripleIt->path(), tec), fend; fpIt != fend; + fpIt.increment(tec)) { + std::error_code fec; + if (!fpIt->is_directory(fec) || fec) continue; + if (!isFingerprint(fpIt->path().filename().string())) continue; + const auto dir = fpIt->path(); + const std::string shown = dir.lexically_relative(*root).generic_string(); + const auto size = mcpp::bmi_cache::dir_size(dir); + if (dryRun) { + std::println("would remove {} ({}; the workspace builds in {})", shown, + mcpp::bmi_cache::human_bytes(size), target.string()); + } else { + std::error_code rec; + fs::remove_all(dir, rec); + if (rec) { + std::println(stderr, "error: cannot remove {}: {}", shown, rec.message()); + ++failed; + continue; + } + std::println("removed {} ({}; the workspace builds in {})", shown, + mcpp::bmi_cache::human_bytes(size), target.string()); + } + bytes += size; + ++removed; + } + } + } + if (removed == 0 && failed == 0) { std::println("Nothing stale under {}: every fingerprint directory is recorded as current", target.string()); diff --git a/src/build/graph_shape.cppm b/src/build/graph_shape.cppm index 679d2d04..16a0be93 100644 --- a/src/build/graph_shape.cppm +++ b/src/build/graph_shape.cppm @@ -71,13 +71,66 @@ std::string_view to_string(GraphShape shape) { // (putting it there would cost a full recompile to package an already-built // tree). So `pack --format X` then `build` would replay a graph carrying a dist // edge, which is the `A then B then A` shape both other fields exist to stop. +// `requestTag` records what the graph was planned for beyond its directory's +// name: a workspace plan's members and the requested features (`request_tag`). +// A build directory is one configuration (workspace design 2026-09-29 §3), so +// `-p a`, `-p b`, `--workspace` and a `--features` build all write their +// graphs into one directory, and the fast paths replay a graph only when its +// tag is the one the request computes. std::string header_line(GraphShape shape, std::string_view scheduleTag, bool accelOverridden = false, - std::string_view packFormat = {}) { - return std::format("# mcpp:graph={};schedule={};accel={};dist={}", + std::string_view packFormat = {}, + std::string_view requestTag = {}) { + return std::format("# mcpp:graph={};schedule={};accel={};dist={};request={}", to_string(shape), scheduleTag, accelOverridden ? "override" : "default", - packFormat.empty() ? std::string_view("none") : packFormat); + packFormat.empty() ? std::string_view("none") : packFormat, + requestTag); +} + +// The tag of a request: the workspace members of the plan (their paths joined +// by a unit separator; empty outside a workspace) and the requested features, +// normalised so that spelling and order do not change it. +std::string request_tag(std::string_view group, std::string_view features) { + std::vector toks; + for (std::size_t i = 0; i < features.size();) { + auto c = features.find_first_of(", ", i); + auto t = features.substr(i, c == std::string_view::npos ? c : c - i); + if (!t.empty()) toks.emplace_back(t); + if (c == std::string_view::npos) break; + i = c + 1; + } + std::ranges::sort(toks); + toks.erase(std::unique(toks.begin(), toks.end()), toks.end()); + std::string key(group); + key += '\x1f'; + for (auto const& t : toks) { key += t; key += ','; } + std::uint64_t h = 0xcbf29ce484222325ull; + for (unsigned char c : key) { h ^= c; h *= 0x100000001b3ull; } + return std::format("{:016x}", h); +} + +// The request tag this graph was written for; empty when the file predates +// the field, which the fast paths treat as a miss. +std::string read_request(const std::filesystem::path& ninjaPath) { + std::ifstream input(ninjaPath); + if (!input) return {}; + std::string line; + for (int i = 0; i < 8 && std::getline(input, line); ++i) { + constexpr std::string_view prefix = "# mcpp:graph="; + if (!line.starts_with(prefix)) continue; + auto value = std::string_view(line).substr(prefix.size()); + while (!value.empty() && (value.back() == '\r' || value.back() == ' ')) + value.remove_suffix(1); + constexpr std::string_view key = ";request="; + const auto at = value.find(key); + if (at == std::string_view::npos) return {}; + auto rest = value.substr(at + key.size()); + if (const auto semi = rest.find(';'); semi != std::string_view::npos) + rest = rest.substr(0, semi); + return std::string(rest); + } + return {}; } // Read the shape back. `nullopt` means "this file does not say" — a build.ninja diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 5a7a2d92..b7fb1aae 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1184,7 +1184,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // replay. Must stay within the first few lines — see read_shape. append(mcpp::build::header_line(plan.graphShape, plan.scheduleTag, plan.accelOverridden, - plan.packFormat) + "\n"); + plan.packFormat, plan.requestTag) + "\n"); append("ninja_required_version = 1.11\n\n"); // All compile/link flags are computed once via flags.cppm. @@ -1200,6 +1200,30 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // else happens to ask. const auto& deployFiles = flags.runtimeDeploy; + // A workspace plan's link groups (workspace design 2026-09-29 §15): a + // member's units link with the flags of the member's closure and take the + // runtime files of that closure, computed by the same `compute_flags` from + // a copy of the plan with the group's fields in place of the plan's. + std::vector groupFlags; + if (!plan.linkGroups.empty()) { + BuildPlan view = plan; + for (auto& g : view.linkGroups) { + swap_link_group(view, g); + groupFlags.push_back(compute_flags(view)); + swap_link_group(view, g); + } + } + // Every file placed beside an artifact: the plan's own deploy set (a + // workspace plan's virtual root has no program, so none), each group's, + // and the graph-built shared libraries each group's units load. + std::vector placedFiles; + if (!plan.manifest.package.virtualRoot) placedFiles = deployFiles; + for (std::size_t g = 0; g < groupFlags.size(); ++g) { + for (auto const& d : groupFlags[g].runtimeDeploy) placedFiles.push_back(d); + for (auto const& pl : plan.linkGroups[g].placements) + placedFiles.push_back({{pl.source}, pl.dest}); + } + // ── The raw image a flasher takes ────────────────────────────────────── // // Only freestanding targets get this; a hosted binary is loaded by a @@ -2741,6 +2765,10 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // Link units for (auto& lu : plan.linkUnits) { + // A member's unit reads its link group's flags (§15); every other + // unit reads the plan's. + const auto& uflags = lu.linkGroup >= 0 + ? groupFlags[static_cast(lu.linkGroup)] : flags; std::string ins; // FIRST, before every other object. Mach-O runs __init_offsets in // LINK order and has no priority-ordered init section, so "runs @@ -2806,8 +2834,11 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // deps), so other targets are unaffected. std::string orderOnly; if (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary) { - for (auto const& d : deployFiles) + for (auto const& d : uflags.runtimeDeploy) orderOnly += " " + escape_ninja_path(d.dest); + if (lu.linkGroup >= 0) + for (auto const& pl : plan.linkGroups[static_cast(lu.linkGroup)].placements) + orderOnly += " " + escape_ninja_path(pl.dest); } // The import library is a SECOND output of this edge, declared as an @@ -2945,13 +2976,13 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // Swapping the driver is not sufficient by itself — this slot names // `libc++.a` by path on macOS and `-static-libstdc++` on MinGW. tail.cxxRuntime = cxxUnit - ? flags.ldStdlibFor(role_of(lu.kind)) - : flags.ldStdlibCFor(role_of(lu.kind)); + ? uflags.ldStdlibFor(role_of(lu.kind)) + : uflags.ldStdlibCFor(role_of(lu.kind)); // An archive has no run-time search path of its own: `ar` never // reads `$unit_ldflags`, so rpath flags there would be dead bytes // in every graph that builds a static library. if (lu.kind != LinkUnit::StaticLibrary) - tail.runtimeFallback = flags.ldRuntimeFallback; + tail.runtimeFallback = uflags.ldRuntimeFallback; // NOT ON A DIRECT LINK. The tag selects between `DT_RPATH` // and `DT_RUNPATH`, entries of a dynamic section; an image with no // loader has neither, and `ld.lld` rejects the flag's `-Wl,` form @@ -2975,6 +3006,11 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { if (auto unit = tail.render(); !unit.empty()) out_line += " unit_ldflags =" + unit + "\n"; } + // The group's link line, in place of the plan's (§15). + if (lu.linkGroup >= 0 && lu.kind != LinkUnit::StaticLibrary) { + out_line += " ldflags =" + uflags.ld + "\n"; + out_line += " c_ldflags =" + uflags.ldC + "\n"; + } append(std::move(out_line)); // ── Freestanding artifact set ────────────────────────────────────── @@ -3035,11 +3071,11 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // exactly one source (every project before this feature, and most // packages after it) emits the exact same line as always: the loop below // reduces to the one-word case with no change in spelling. - if (deployFiles.size() >= 2) { + if (placedFiles.size() >= 2) { // #734 E4: one edge for the whole list. One entry keeps the per-file // edge below, byte for byte. std::string outs, ins, list; - for (auto const& d : deployFiles) { + for (auto const& d : placedFiles) { outs += " " + escape_ninja_path(d.dest); for (auto const& s : d.sources) { ins += " " + escape_ninja_path(s); @@ -3047,17 +3083,17 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { } } append(std::format("build{} : stage_list{} | placements.list\n count = {}\n", - outs, ins, deployFiles.size())); + outs, ins, placedFiles.size())); if (placements) *placements = std::move(list); } else { - for (auto const& d : deployFiles) { + for (auto const& d : placedFiles) { std::string ins; for (auto const& s : d.sources) ins += " " + escape_ninja_path(s); append(std::format("build {} : stage_file{}\n", escape_ninja_path(d.dest), ins)); } } - if (!deployFiles.empty()) + if (!placedFiles.empty()) append("\n"); // ── Declared build-graph nodes (`mcpp:action=`) ───────────────────────── @@ -3267,7 +3303,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { defaults += " " + escape_ninja_path(alias); } } - for (auto const& d : deployFiles) { + for (auto const& d : placedFiles) { defaults += " " + escape_ninja_path(d.dest); } defaults += actionDefaults; diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 7231121c..9d74c1f6 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -107,6 +107,13 @@ struct LinkUnit { // every other unit. A dependency-owned `Binary` is always one of these; the // name is what `${mcpp.artifact:/}` is resolved against. std::string artifactOf; + // A workspace plan's link unit of a selected member (workspace design + // 2026-09-29 §15): the member's qualified package name, and the index of + // its link group in `BuildPlan::linkGroups`, which holds the link flags + // and the runtime files of the member's closure. Empty and -1 on every + // other unit. + std::string memberOf; + int linkGroup = -1; // Normally relative to plan.outputDir. A `role = "object"` action's outputs // land here ABSOLUTE, on purpose: ninja identifies a file by the string an // edge declares, and the action edge declares whatever prepare_actions @@ -324,6 +331,12 @@ struct BuildPlan { // library over a static package that something else reaches too. std::vector staticPlacementConflicts; std::string scheduleTag = "none"; + // What this graph was planned FOR, beyond what names its directory: the + // workspace members of the plan and the requested features, as one tag + // (`request_tag`). Neither moves the build directory (workspace design + // 2026-09-29 §3), so build.ninja records the tag and the fast paths + // compare it before replaying the file. + std::string requestTag; // Whether `--accel` / `--no-accel` selected this graph's device variant // over `[build] accel`. The variant is in the fingerprint, so the two // builds land in different directories -- and the fast path, which runs @@ -480,8 +493,55 @@ struct BuildPlan { // musl-gcc 15.1 modules failed to emit vector>'s // move-ctor instantiation across the module boundary (release link error). std::vector runtimeProviders; + + // A workspace plan's per-member link data (workspace design 2026-09-29 + // §15). A selected member's link units link the member's closure -- the + // member and what it reaches without an `artifacts` or member edge -- with + // that closure's own link flags, and the files its runtime needs are + // placed in its product directory. The fields repeat the plan's own, so a + // group is read by swapping it into a copy of the plan + // (`swap_link_group`). Empty outside a workspace plan, where the plan's + // own fields serve the root's units. + struct Placement { + std::filesystem::path source; // relative to outputDir + std::filesystem::path dest; // relative to outputDir + }; + struct LinkGroup { + std::string member; // qualified package name + std::filesystem::path productDir; // relative to outputDir + std::vector ldflags; // the plan's, then the closure's + std::vector runtimeLibraryDirs; + std::vector depRuntimeLibraryDirs; + std::vector runtimeRequirements; + std::vector runtimeArtifacts; + mcpp::manifest::LinkIntent linkIntent; + std::vector runtimeDeployFiles; + std::vector runtimeDlopenLibs; + std::vector runtimeCapabilities; + std::vector runtimeProviders; + // The graph-built shared libraries the member's units load, placed + // beside them (a hard link where the file system allows, §5.3). + std::vector placements; + }; + std::vector linkGroups; }; +// Exchange a link group's fields with the plan's own. Called twice, it +// restores both; between the calls, the plan describes the group's units to +// every reader of those fields (`compute_flags` in particular). +void swap_link_group(BuildPlan& plan, BuildPlan::LinkGroup& g) { + std::swap(plan.manifest.buildConfig.ldflags, g.ldflags); + std::swap(plan.runtimeLibraryDirs, g.runtimeLibraryDirs); + std::swap(plan.depRuntimeLibraryDirs, g.depRuntimeLibraryDirs); + std::swap(plan.runtimeRequirements, g.runtimeRequirements); + std::swap(plan.runtimeArtifacts, g.runtimeArtifacts); + std::swap(plan.linkIntent, g.linkIntent); + std::swap(plan.runtimeDeployFiles, g.runtimeDeployFiles); + std::swap(plan.runtimeDlopenLibs, g.runtimeDlopenLibs); + std::swap(plan.runtimeCapabilities, g.runtimeCapabilities); + std::swap(plan.runtimeProviders, g.runtimeProviders); +} + // Merge the generic facts exported by the already-selected xlings // RuntimeBinding. This is data ingestion only: xlings has already selected // providers and materialized artifacts before mcpp sees this snapshot. @@ -1433,147 +1493,156 @@ make_plan(const mcpp::manifest::Manifest& manifest, plan.stdBmiPath = stdBmiPath; plan.stdObjectPath = stdObjectPath; - auto runtimeContract = resolve_runtime_contract(packages); - plan.runtimeRequirements = std::move(runtimeContract.requirements); - plan.runtimeArtifacts = std::move(runtimeContract.artifacts); - plan.linkIntent = std::move(runtimeContract.linkIntent); - plan.runtimeProviders = std::move(runtimeContract.providers); - - for (auto const& dir : plan.linkIntent.runtimeSearchDirs) { - append_unique_path(plan.runtimeLibraryDirs, dir); - append_unique_path(plan.depRuntimeLibraryDirs, dir); - } - for (auto const& requirement : plan.runtimeRequirements) { - // Optional requirements remain first-class provenance in - // resolution.json, but they cannot become hard ABI/doctor inputs. - if (!requirement.required) continue; - if (requirement.kind == "soname") { - if (std::ranges::find(plan.runtimeDlopenLibs, requirement.value) - == plan.runtimeDlopenLibs.end()) - plan.runtimeDlopenLibs.push_back(requirement.value); - } else if (requirement.kind == "capability") { - if (std::ranges::find(plan.runtimeCapabilities, requirement.value) - == plan.runtimeCapabilities.end()) - plan.runtimeCapabilities.push_back(requirement.value); + // What the runtime needs beside and around the artifacts of a set of + // packages: the plan's, over every package, and a workspace member's, + // over the member's closure, placed in its product directory (workspace + // design 2026-09-29 §15). `out` is the plan or a link group; both carry + // the same fields. + auto derive_runtime = [&](const std::vector& pkgs, + const std::filesystem::path& binDir, auto& out) { + auto runtimeContract = resolve_runtime_contract(pkgs); + out.runtimeRequirements = std::move(runtimeContract.requirements); + out.runtimeArtifacts = std::move(runtimeContract.artifacts); + out.linkIntent = std::move(runtimeContract.linkIntent); + out.runtimeProviders = std::move(runtimeContract.providers); + + for (auto const& dir : out.linkIntent.runtimeSearchDirs) { + append_unique_path(out.runtimeLibraryDirs, dir); + append_unique_path(out.depRuntimeLibraryDirs, dir); + } + for (auto const& requirement : out.runtimeRequirements) { + // Optional requirements remain first-class provenance in + // resolution.json, but they cannot become hard ABI/doctor inputs. + if (!requirement.required) continue; + if (requirement.kind == "soname") { + if (std::ranges::find(out.runtimeDlopenLibs, requirement.value) + == out.runtimeDlopenLibs.end()) + out.runtimeDlopenLibs.push_back(requirement.value); + } else if (requirement.kind == "capability") { + if (std::ranges::find(out.runtimeCapabilities, requirement.value) + == out.runtimeCapabilities.end()) + out.runtimeCapabilities.push_back(requirement.value); + } } - } - // `toDir` is a `runtime.deploy` destination, relative to the executable's - // directory; empty and "." both mean that directory itself, which is where - // every `deploy_files` entry goes. The check keys on the full relative - // destination, so two files of one name in two directories do not - // collide. - // - // Two sources for one destination are no longer refused HERE (SPEC-007 - // R4.2, #723): at planning time a generated source may not exist yet, so - // its content cannot be compared. Both stay as inputs of the one - // `stage_file` edge this destination becomes (ninja_backend.cppm), and - // `mcpp stage` (mcpp.build.stage) is where the invariant — one - // destination, one content — is actually checked, once the sources exist. - // A source already listed for this destination (the ordinary case: the - // same file reached through two graph edges) is not duplicated. - // On a PE target a destination is compared without case - // (DeployFile::is_destination). - const bool peTarget = targetTriple.empty() ? bool(mcpp::platform::is_windows) - : targetTriple.is_pe(); - auto add_deploy = [&](const std::filesystem::path& source, - std::string_view toDir = {}) { - const auto normalized = source.lexically_normal(); - auto destDir = std::filesystem::path("bin"); - if (!toDir.empty() && toDir != ".") destDir /= std::filesystem::path(toDir); - const auto dest = destDir / source.filename(); - auto existing = std::ranges::find_if(plan.runtimeDeployFiles, - [&](auto const& value) { return value.is_destination(dest, peTarget); }); - if (existing != plan.runtimeDeployFiles.end()) { - if (std::ranges::find(existing->sources, normalized) - == existing->sources.end()) - existing->sources.push_back(normalized); - return; - } - plan.runtimeDeployFiles.push_back({{normalized}, dest}); - }; - // Structured deploy files are explicit and platform-neutral. Legacy - // library_dirs keeps its one-train DLL discovery behavior below. - for (auto const& source : plan.linkIntent.deployFiles) { - add_deploy(source); - } - for (auto const& entry : plan.linkIntent.deploy) { - add_deploy(entry.from, entry.to); - } - // A DLL found in a runtime search directory is a DERIVED candidate - // (SPEC-007 R4.3). The plan lists it and does not decide: which file sits - // beside the program -- a declaration, the toolset's C++ runtime, or this - // one -- is mcpp.build.runtime_placement's answer, read through - // `CompileFlags::runtimeDeploy`. A difference between a declared file and - // a search directory's copy is stated by the post-link placement edge, - // once, through the edge-advice channel (mcpp.build.advice). - // - // Sorted within each directory: directory order is not a stable input, and - // this list reaches build.ninja. - for (auto const& dir : plan.linkIntent.runtimeSearchDirs) { - std::error_code dirEc; - if (!std::filesystem::is_directory(dir, dirEc)) continue; - std::vector dlls; - for (auto const& entry : std::filesystem::directory_iterator(dir, dirEc)) { - if (!entry.is_regular_file()) continue; - auto ext = entry.path().extension().string(); - std::ranges::transform(ext, ext.begin(), - [](unsigned char c){ return std::tolower(c); }); - if (ext == ".dll") dlls.push_back(entry.path().lexically_normal()); - } - std::ranges::sort(dlls); - for (auto const& dll : dlls) - plan.runtimeDeployFiles.push_back( - {{dll}, std::filesystem::path("bin") / dll.filename(), - BuildPlan::DeployFile::Origin::Derived}); - } - // The same private runtime directories embedded as executable RUNPATH are - // also needed in the process environment for libraries reached only via - // dlopen(), because their own DT_NEEDED closure does not consult the main - // executable's RUNPATH. - for (auto const& dir : tc.linkRuntimeDirs) { - append_unique_path(plan.runtimeLibraryDirs, dir); - } - // The MSVC toolset's own redistributable CRT (#718), on the `mcpp run`/ - // `mcpp test` search path exactly as the staged copy is beside the - // artifact (`flags.cppm`'s toolchain-coupled staging) — the two are the - // same directory for both the cl.exe row (already inside - // `linkRuntimeDirs` above, so this is a harmless duplicate there) and the - // LLVM row (where `linkRuntimeDirs` holds LLVM's own directories instead - // and would otherwise never mention it). - if (!tc.msvcRedistDir.empty()) - append_unique_path(plan.runtimeLibraryDirs, tc.msvcRedistDir); - // The private glibc payload exists here for ONE reason: a dlopen()'d - // library, whose own DT_NEEDED closure never consults the main - // executable's RUNPATH, must still resolve the same libc the executable - // was linked against. So it is only published when this build actually has - // such a library. - // - // It is published through the ARTIFACT — the link model already emits - // `-Wl,-rpath,` alongside --dynamic-linker wherever a payload - // exists — and never through the environment. - // LD_LIBRARY_PATH is inherited by the whole process subtree, and a child - // that is a HOST binary (/bin/sh, reached via popen()) loads the HOST - // loader — PT_INTERP is baked in and no environment variable overrides it - // — while the variable hands it the payload libc.so.6. libc and ld.so are - // version-locked through GLIBC_PRIVATE, so the shell dies during - // relocation, before main: glibc 2.44's libc.so.6 needs - // `__pointer_chk_guard` from its own loader (mcpp#401), and older payloads - // segfault in the linker instead (mcpp#291). It does NOT reproduce when - // host and payload glibc happen to match, which is how it survived. - // - // process.cppm's strip_private_glibc already removes this entry from - // mcpp's OWN children. It cannot help one hop further out — what the - // target spawns is beyond mcpp's reach. Putting the directory in the - // artifact's RUNPATH instead is: DT_RUNPATH reaches the object that - // carries it and the dlopen() it performs, and nothing else. - if constexpr (mcpp::platform::publishes_via_environment( - mcpp::platform::kPrivateLibcSearchScope)) { - if (tc.payloadPaths && !plan.depRuntimeLibraryDirs.empty()) { - append_unique_path(plan.runtimeLibraryDirs, - tc.payloadPaths->glibcLib); + // `toDir` is a `runtime.deploy` destination, relative to the executable's + // directory; empty and "." both mean that directory itself, which is where + // every `deploy_files` entry goes. The check keys on the full relative + // destination, so two files of one name in two directories do not + // collide. + // + // Two sources for one destination are no longer refused HERE (SPEC-007 + // R4.2, #723): at planning time a generated source may not exist yet, so + // its content cannot be compared. Both stay as inputs of the one + // `stage_file` edge this destination becomes (ninja_backend.cppm), and + // `mcpp stage` (mcpp.build.stage) is where the invariant — one + // destination, one content — is actually checked, once the sources exist. + // A source already listed for this destination (the ordinary case: the + // same file reached through two graph edges) is not duplicated. + // On a PE target a destination is compared without case + // (DeployFile::is_destination). + const bool peTarget = targetTriple.empty() ? bool(mcpp::platform::is_windows) + : targetTriple.is_pe(); + auto add_deploy = [&](const std::filesystem::path& source, + std::string_view toDir = {}) { + const auto normalized = source.lexically_normal(); + auto destDir = binDir; + if (!toDir.empty() && toDir != ".") destDir /= std::filesystem::path(toDir); + const auto dest = destDir / source.filename(); + auto existing = std::ranges::find_if(out.runtimeDeployFiles, + [&](auto const& value) { return value.is_destination(dest, peTarget); }); + if (existing != out.runtimeDeployFiles.end()) { + if (std::ranges::find(existing->sources, normalized) + == existing->sources.end()) + existing->sources.push_back(normalized); + return; + } + out.runtimeDeployFiles.push_back({{normalized}, dest}); + }; + // Structured deploy files are explicit and platform-neutral. Legacy + // library_dirs keeps its one-train DLL discovery behavior below. + for (auto const& source : out.linkIntent.deployFiles) { + add_deploy(source); } - } + for (auto const& entry : out.linkIntent.deploy) { + add_deploy(entry.from, entry.to); + } + // A DLL found in a runtime search directory is a DERIVED candidate + // (SPEC-007 R4.3). The plan lists it and does not decide: which file sits + // beside the program -- a declaration, the toolset's C++ runtime, or this + // one -- is mcpp.build.runtime_placement's answer, read through + // `CompileFlags::runtimeDeploy`. A difference between a declared file and + // a search directory's copy is stated by the post-link placement edge, + // once, through the edge-advice channel (mcpp.build.advice). + // + // Sorted within each directory: directory order is not a stable input, and + // this list reaches build.ninja. + for (auto const& dir : out.linkIntent.runtimeSearchDirs) { + std::error_code dirEc; + if (!std::filesystem::is_directory(dir, dirEc)) continue; + std::vector dlls; + for (auto const& entry : std::filesystem::directory_iterator(dir, dirEc)) { + if (!entry.is_regular_file()) continue; + auto ext = entry.path().extension().string(); + std::ranges::transform(ext, ext.begin(), + [](unsigned char c){ return std::tolower(c); }); + if (ext == ".dll") dlls.push_back(entry.path().lexically_normal()); + } + std::ranges::sort(dlls); + for (auto const& dll : dlls) + out.runtimeDeployFiles.push_back( + {{dll}, binDir / dll.filename(), + BuildPlan::DeployFile::Origin::Derived}); + } + // The same private runtime directories embedded as executable RUNPATH are + // also needed in the process environment for libraries reached only via + // dlopen(), because their own DT_NEEDED closure does not consult the main + // executable's RUNPATH. + for (auto const& dir : tc.linkRuntimeDirs) { + append_unique_path(out.runtimeLibraryDirs, dir); + } + // The MSVC toolset's own redistributable CRT (#718), on the `mcpp run`/ + // `mcpp test` search path exactly as the staged copy is beside the + // artifact (`flags.cppm`'s toolchain-coupled staging) — the two are the + // same directory for both the cl.exe row (already inside + // `linkRuntimeDirs` above, so this is a harmless duplicate there) and the + // LLVM row (where `linkRuntimeDirs` holds LLVM's own directories instead + // and would otherwise never mention it). + if (!tc.msvcRedistDir.empty()) + append_unique_path(out.runtimeLibraryDirs, tc.msvcRedistDir); + // The private glibc payload exists here for ONE reason: a dlopen()'d + // library, whose own DT_NEEDED closure never consults the main + // executable's RUNPATH, must still resolve the same libc the executable + // was linked against. So it is only published when this build actually has + // such a library. + // + // It is published through the ARTIFACT — the link model already emits + // `-Wl,-rpath,` alongside --dynamic-linker wherever a payload + // exists — and never through the environment. + // LD_LIBRARY_PATH is inherited by the whole process subtree, and a child + // that is a HOST binary (/bin/sh, reached via popen()) loads the HOST + // loader — PT_INTERP is baked in and no environment variable overrides it + // — while the variable hands it the payload libc.so.6. libc and ld.so are + // version-locked through GLIBC_PRIVATE, so the shell dies during + // relocation, before main: glibc 2.44's libc.so.6 needs + // `__pointer_chk_guard` from its own loader (mcpp#401), and older payloads + // segfault in the linker instead (mcpp#291). It does NOT reproduce when + // host and payload glibc happen to match, which is how it survived. + // + // process.cppm's strip_private_glibc already removes this entry from + // mcpp's OWN children. It cannot help one hop further out — what the + // target spawns is beyond mcpp's reach. Putting the directory in the + // artifact's RUNPATH instead is: DT_RUNPATH reaches the object that + // carries it and the dlopen() it performs, and nothing else. + if constexpr (mcpp::platform::publishes_via_environment( + mcpp::platform::kPrivateLibcSearchScope)) { + if (tc.payloadPaths && !out.depRuntimeLibraryDirs.empty()) { + append_unique_path(out.runtimeLibraryDirs, + tc.payloadPaths->glibcLib); + } + } + }; + derive_runtime(packages, std::filesystem::path("bin"), plan); // 1a. Object addressing. // @@ -1644,7 +1713,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, if (ec || rel.empty()) continue; if (rel.generic_string().starts_with("..")) continue; auto len = packages[p].root.generic_string().size(); - if (!found || len > bestLen) { best = p; bestLen = len; found = true; } + // Equal roots are a workspace plan's virtual root and the rooted + // workspace's own package, which owns the sources (§15). + if (!found || len > bestLen || (len == bestLen && best == 0)) { + best = p; bestLen = len; found = true; + } } if (!found) return std::nullopt; return best; @@ -2707,6 +2780,228 @@ make_plan(const mcpp::manifest::Manifest& manifest, plan.linkUnits.push_back(std::move(lu)); } + // 6. The targets of the selected workspace members (workspace design + // 2026-09-29 §15). + // + // Each target of a selected member is a link unit of this plan, linked + // from the member's closure the way the member's own build linked it when + // it was a root: its objects, those of every package it reaches, and its + // entry. Its output is in the member's product directory, `bin//` + // (`bin/` for a rooted workspace's own package), with the files its + // runtime needs beside it: the deploy set of its closure and the + // graph-built shared libraries it loads. The link flags and the runtime + // files are the closure's, held in the member's link group, so a member + // links nothing that only another member needs. + std::set memberDevDepPackages; + for (std::size_t mi = 1; mi < packages.size(); ++mi) { + auto const& pkg = packages[mi]; + if (!pkg.memberProducts) continue; + const auto owner = qualified_package_name(pkg.manifest); + const auto productDir = pkg.memberProducts->empty() + ? std::filesystem::path("bin") + : std::filesystem::path("bin") / *pkg.memberProducts; + auto place = [&](const std::filesystem::path& o) { return productDir / o.filename(); }; + + // The closure: the member, what it reaches through its dependency + // edges (no `artifacts` edge), and under `mcpp test` its + // dev-dependencies, which are its own and are not walked further + // than their own dependencies. + std::set closureIdx{mi}; + { + std::vector work{mi}; + auto push = [&](std::size_t j) { + if (closureIdx.insert(j).second) work.push_back(j); + }; + for (auto const& [depName, spec] : pkg.manifest.devDependencies) + for (auto const& candidate : dependency_name_candidates(depName, spec)) + if (auto it = packageIndexByName.find(candidate); + it != packageIndexByName.end() && it->second != mi) { + memberDevDepPackages.insert( + qualified_package_name(packages[it->second].manifest)); + push(it->second); + break; + } + while (!work.empty()) { + const auto i = work.back(); work.pop_back(); + if (auto it = directPackageDeps.find(i); it != directPackageDeps.end()) + for (auto j : it->second) { + if (artifactEdges.contains({i, j})) continue; + push(j); + } + } + } + std::set closure; + for (auto i : closureIdx) closure.insert(qualified_package_name(packages[i].manifest)); + + // The member's link group: the closure's link flags after the + // plan's own, in the order a root build gives them (the member, then + // its dependencies in discovery order), and the closure's runtime. + BuildPlan::LinkGroup group; + group.member = owner; + group.productDir = productDir; + group.ldflags = manifest.buildConfig.ldflags; + for (auto i : closureIdx) + for (auto const& f : packages[i].linkUsage.ldflags) + group.ldflags.push_back(f); + { + std::vector closurePackages; + for (auto i : closureIdx) closurePackages.push_back(packages[i]); + derive_runtime(closurePackages, productDir, group); + } + const int groupIndex = static_cast(plan.linkGroups.size()); + + bool memberTestMode = std::ranges::any_of(pkg.manifest.targets, [](auto const& t) { + return t.kind == mcpp::manifest::Target::TestBinary; + }); + const auto memberExtTable = mcpp::extension_table_for( + pkg.manifest.buildConfig.moduleExtensions, + pkg.manifest.buildConfig.deviceExtensions); + std::vector units; + for (auto const& t : pkg.manifest.targets) { + if (memberTestMode && t.kind != mcpp::manifest::Target::TestBinary) continue; + // A member's shared libraries are linked once, with the graph's + // shared libraries (above), and placed in its product directory. + if (t.kind == mcpp::manifest::Target::SharedLibrary) { + const auto out = target_output(t, naming); + group.placements.push_back({out, place(out)}); + for (auto const& alias : runtime_aliases_for_target(t, naming)) + group.placements.push_back({alias, place(alias)}); + continue; + } + LinkUnit lu; + lu.targetName = t.name; + lu.memberOf = owner; + lu.linkGroup = groupIndex; + if (t.kind == mcpp::manifest::Target::Library) { + lu.kind = LinkUnit::StaticLibrary; + lu.output = place(target_output(t, naming)); + } else if (t.kind == mcpp::manifest::Target::Application + && mcpp::toolchain::triple::application_form(targetTriple) + == mcpp::toolchain::triple::ApplicationForm::SharedObject) { + lu.kind = LinkUnit::SharedLibrary; + lu.output = place(target_output(t, naming, /*asSharedObject=*/true)); + lu.importLibrary = import_library_for(t, naming); + if (!lu.importLibrary.empty()) lu.importLibrary = place(lu.importLibrary); + lu.soname = t.soname; + lu.exportPatterns = t.exportPatterns; + for (auto const& alias : runtime_aliases_for_target(t, naming)) + lu.runtimeAliases.push_back(place(alias)); + } else { + lu.kind = t.kind == mcpp::manifest::Target::TestBinary + ? LinkUnit::TestBinary : LinkUnit::Binary; + lu.output = t.kind == mcpp::manifest::Target::TestBinary + ? productDir / target_output(t, naming).lexically_relative("bin") + : place(target_output(t, naming)); + lu.windowsSubsystem = t.windowsSubsystem; + lu.windowsEntry = t.windowsEntry; + } + lu.loaderTagFlag = loader_tag_flag(lu.kind); + for (auto const& other : plan.linkUnits) + if (other.output == lu.output) + return std::unexpected(std::format( + "target '{}' of workspace member '{}' would be written to " + "'{}', which target '{}' of this build also produces", + t.name, owner, lu.output.generic_string(), other.targetName)); + + for (auto const& cu : plan.compileUnits) { + if (!closure.contains(cu.packageName)) continue; + if (sharedDepPackages.contains(cu.packageName)) continue; + if (mcpp::links_unconditionally(cu.kind)) lu.objects.push_back(cu.object); + } + if (!t.main.empty() && lu.kind != LinkUnit::StaticLibrary) { + const auto entry = pkg.root / t.main; + lu.entryMain = entry; + std::optional entryObject; + for (auto const& cu : plan.compileUnits) + if (cu.source == entry) { entryObject = cu.object; break; } + if (!entryObject) { + CompileUnit main_cu; + main_cu.source = entry; + main_cu.packageName = owner; + main_cu.kind = mcpp::classify(entry, memberExtTable); + if (pkg.usageResolved) { + main_cu.localIncludeDirs = pkg.privateBuild.includeDirs; + main_cu.localIncludeDirsAfter = pkg.privateBuild.includeDirsAfter; + main_cu.packageCflags = pkg.privateBuild.cflags; + main_cu.packageCxxflags = pkg.privateBuild.cxxflags; + } else { + main_cu.localIncludeDirs = local_include_dirs_for_manifest(pkg.root, pkg.manifest); + main_cu.localIncludeDirsAfter = + local_include_dirs_after_for_manifest(pkg.root, pkg.manifest); + main_cu.packageCflags = pkg.manifest.buildConfig.cflags; + main_cu.packageCxxflags = pkg.manifest.buildConfig.cxxflags; + } + mcpp::modgraph::normalize_include_flags(pkg.root, main_cu.packageCflags); + mcpp::modgraph::normalize_include_flags(pkg.root, main_cu.packageCxxflags); + apply_c_standard(main_cu); + const auto scanned = mcpp::modgraph::scan_entry_file(entry, owner, memberExtTable); + for (auto const& req : scanned.requires_) main_cu.imports.push_back(req.logicalName); + main_cu.declaration = scanned.provides + ? mcpp::modgraph::ModuleDeclaration::Unknown : scanned.declaration; + main_cu.object = object_for(entry, owner, + std::filesystem::relative(entry, pkg.root), mi).object; + plan.compileUnits.push_back(main_cu); + entryObject = main_cu.object; + } + lu.objects.push_back(*entryObject); + // Per-target entry-scoped flags (issue #131), on the unit that + // builds this target's entry, as for a root's target. + if (!t.defines.empty() || !t.cflags.empty() || !t.cxxflags.empty()) { + for (auto& cu : plan.compileUnits) { + if (cu.source != entry) continue; + for (auto const& d : t.defines) { + const auto element = mcpp::manifest::flag_element("-D" + d); + cu.packageCflags.push_back(element); + cu.packageCxxflags.push_back(element); + } + for (auto const& f : t.cflags) cu.packageCflags.push_back(f); + for (auto const& f : t.cxxflags) cu.packageCxxflags.push_back(f); + break; + } + } + } + const bool entryDefinesMain = lu.entryMain && source_defines_main(*lu.entryMain); + for (auto const& cu : plan.compileUnits) { + if (!closure.contains(cu.packageName)) continue; + if (sharedDepPackages.contains(cu.packageName)) continue; + if (!is_implementation_source(cu.kind)) continue; + if (lu.entryMain && cu.source == *lu.entryMain) continue; + if (entryFilesAcrossTargets.contains(cu.source)) continue; + // A dev-dependency's own main-providing object (gtest_main.o) + // only for a test with no main of its own, as for a root. + if (entryDefinesMain && memberDevDepPackages.contains(cu.packageName) + && source_defines_main(cu.source)) continue; + lu.objects.push_back(cu.object); + } + if (lu.kind != LinkUnit::StaticLibrary) { + const auto before = lu.implicitInputs.size(); + // The shared libraries the closure's packages link, and no + // other member's. + for (auto i : closureIdx) + if (!sharedDepPackages.contains(qualified_package_name(packages[i].manifest)) + && !placedInImage.contains(qualified_package_name(packages[i].manifest))) + append_direct_shared_deps(lu, i); + // The graph-built shared libraries this unit loads are placed + // beside it. + for (std::size_t k = before; k < lu.implicitInputs.size(); ++k) { + auto const& in = lu.implicitInputs[k]; + if (in.parent_path() == productDir) continue; + // What a PE consumer links is the import library; the + // loader reads the `.dll` beside it. + if (naming.sharedNeedsImportLib + && in.filename().string().ends_with(naming.staticLibExt)) + continue; + const bool seen = std::ranges::any_of(group.placements, + [&](auto const& pl) { return pl.source == in; }); + if (!seen) group.placements.push_back({in, place(in)}); + } + } + units.push_back(std::move(lu)); + } + plan.linkGroups.push_back(std::move(group)); + for (auto& u : units) plan.linkUnits.push_back(std::move(u)); + } + // The single derivation. Deliberately at the END of make_plan, after every // producer of a link unit has run: a dependency resolved to the shared // form arrives as an ordinary SharedLibrary unit, so this one predicate diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 6b6242b5..2b30c8aa 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -475,6 +475,24 @@ export struct BuildContext { // already existed inside prepare_build; the banner and mcpp.lock were simply // reading the input instead of the output. Both now read this. std::map resolvedVersions; + // The selected members of a workspace plan (workspace design 2026-09-29 + // §15), in selection order: each member's manifest (its hooks, its + // `[pack]`, its identity), its root, and the directory below `bin/` its + // products are in. Empty outside a workspace plan, whose subject is + // `manifest` itself. + struct WorkspaceMember { + std::string name; // qualified package name + std::string memberPath; // as `[workspace] members` writes it + std::filesystem::path root; + std::string productDir; // empty: `bin/` itself + mcpp::manifest::Manifest manifest; + }; + std::vector workspaceMembers; + // The command's selection and this plan's group, each the member paths + // joined by a unit separator; empty outside a workspace. A fast-path + // record is written, and matched, per selection and group (§15). + std::string workspaceRequest; + std::string workspaceGroup; }; // The ONE cache-mode resolver, for the same reason resolve_profile_name exists: @@ -488,6 +506,15 @@ export struct BuildContext { // Precedence: --cache > MCPP_BUILD_CACHE > [build] cache > global. An // unparseable value falls through to the next source rather than silently // meaning "global" — see prepare_build, which also reports it. +// A plan of one workspace member, read as that member (workspace design +// 2026-09-29 §15): the member's manifest and root become the context's, and +// its link group's runtime fields become the plan's, so a reader that asks +// "the package being built" -- `mcpp pack`, the runtime closure it stages -- +// is answered about the member rather than about the virtual root. Called +// after the plan's graph is written; the plan's own fields are exchanged, not +// lost. A context of any other shape is left as it is. +export void focus_on_member(BuildContext& ctx); + export CacheMode resolve_cache_mode(const mcpp::manifest::Manifest& m, std::string_view override_mode); @@ -611,6 +638,9 @@ export struct BuildOverrides { // configuration group (workspace design 2026-09-29 §15). Empty: the member // is selected from `package_filter` or the command's directory. std::vector workspace_members; + // Every member the COMMAND selected, across its configuration groups: the + // request a fast-path record names. Empty: `workspace_members`. + std::vector workspace_request; // --profile . Empty = fall through to `[build] default-profile`, then // to `profile_fallback` below, whose own default is "dev". The comment here // said "release" for as long as `mcpp build --help` did, and neither had diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 4e833561..2b3814c3 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -358,6 +358,14 @@ static std::string normalizeDepLdflag(const std::filesystem::path& depRoot, return flag; } +std::vector normalized_dependency_ldflags( + const std::filesystem::path& depRoot, const std::vector& ldflags) { + std::vector out; + for (auto const& word : mcpp::manifest::flag_words(ldflags)) + out.push_back(mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word))); + return out; +} + static std::vector propagateLinkFlags( PrepareState& state, const std::filesystem::path& depRoot, @@ -1550,7 +1558,12 @@ step4b_acquire_dependency_source(PrepareState& state, WorklistItemCtx& ctx) { "git clone of '{}' failed:\n{}", spec.git, r.output)); } } - if (item.consumerDepIndex == kMainConsumer) { + // A selected workspace member's own git dependencies are locked + // as a root's are (workspace design 2026-09-29 §15). + const bool consumerIsMember = item.consumerDepIndex != kMainConsumer + && item.consumerDepIndex + 1 < state.packages.size() + && state.packages[item.consumerDepIndex + 1].memberProducts.has_value(); + if (item.consumerDepIndex == kMainConsumer || consumerIsMember) { // Only root deps are locked: the writer below walks the root // manifest's [dependencies], so a transitive git branch dep // has no anchor and still resolves over the network. @@ -1736,7 +1749,15 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { // answered it; otherwise two indices containing the same short name // collapse in runtime provenance even though resolution distinguished // them correctly. - if (ctx.dep_manifest->package.namespace_.empty()) { + // + // A workspace member of a workspace plan is the project being + // developed, and keeps the identity its own manifest states, as when + // it was the root of its own build (workspace design 2026-09-29 §15): + // a member that declares no namespace is named by its bare name. + const bool workspaceMemberHere = state.workspacePlan() && sourceKind == "path" + && (ctx.dep_root.lexically_normal() == state.runtimeWorkspaceRoot.lexically_normal() + || !workspace_member_of(state, ctx.dep_root).empty()); + if (ctx.dep_manifest->package.namespace_.empty() && !workspaceMemberHere) { ctx.dep_manifest->package.namespace_ = key.ns.empty() ? std::string(mcpp::pm::kDefaultNamespace) : key.ns; } diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index a5857989..337704e6 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -71,6 +71,7 @@ select_workspace_members(PrepareState& state, const std::filesystem::path& wsRoo // `--features` names features of the selected members: each member takes // the tokens it declares, and a token no selected member declares is // refused. + state.requestedFeatures = state.overrides.features; std::vector tokens; for (auto const& t : mcpp::build::feature_request_tokens(state.overrides.features)) tokens.push_back(t); @@ -125,7 +126,7 @@ select_workspace_members(PrepareState& state, const std::filesystem::path& wsRoo std::string shown; for (auto const& [key, spec] : state.m->dependencies) - shown += (shown.empty() ? "" : ", ") + spec.shortName; + shown += (shown.empty() ? "" : ", ") + key; mcpp::ui::status("Workspace", selection.size() == 1 ? std::format("building member '{}'", shown) : std::format("building {} members: {}", selection.size(), shown)); @@ -576,6 +577,15 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta // Read where the project keeps it. A planning pass that writes // elsewhere (plan_only) still resolves against the project's lock. auto lockPath = (state.overrides.plan_only ? *state.root : state.workRoot) / "mcpp.lock"; + // A workspace keeps one lock at its root (§15). Until its first build + // has written it, a selected member's own lock from before is read in + // its place, so the upgrade does not move a git dependency. + if (state.workspacePlan() && !std::filesystem::exists(lockPath)) + for (auto const& mp : state.selectedMemberPaths) + if (std::filesystem::exists(state.runtimeWorkspaceRoot / mp / "mcpp.lock")) { + lockPath = state.runtimeWorkspaceRoot / mp / "mcpp.lock"; + break; + } if (std::filesystem::exists(lockPath)) { if (auto lock = mcpp::pm::load(lockPath); lock) { for (auto const& p : lock->packages) { diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 271da635..e0f808b7 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -162,8 +162,43 @@ static std::expected step13_source_packages(PrepareState& sta if (std::find(roots.begin(), roots.end(), normalized) == roots.end()) roots.push_back(std::move(normalized)); } + // A workspace plan's members are its projects: their trees are what + // the fast path sweeps, the workspace's own package included. + if (state.workspacePlan()) + for (std::size_t i = 1; i < state.packages.size(); ++i) { + if (!state.packages[i].memberProducts) continue; + auto normalized = state.packages[i].root.lexically_normal(); + if (std::find(roots.begin(), roots.end(), normalized) == roots.end()) + roots.push_back(normalized); + } ctx.depSourceRoots = std::move(roots); } + // The selected members, in selection order (§15). + if (state.workspacePlan()) { + for (auto const& mp : state.selectedMemberPaths) { + std::error_code ec; + auto dir = std::filesystem::weakly_canonical(state.runtimeWorkspaceRoot / mp, ec); + if (ec) dir = (state.runtimeWorkspaceRoot / mp).lexically_normal(); + for (std::size_t i = 1; i < state.packages.size(); ++i) { + auto const& pkg = state.packages[i]; + if (!pkg.memberProducts) continue; + std::error_code pec; + auto root = std::filesystem::weakly_canonical(pkg.root, pec); + if (pec) root = pkg.root.lexically_normal(); + if (root != dir) continue; + ctx.workspaceMembers.push_back({ + .name = pkg.manifest.package.namespace_.empty() + ? pkg.manifest.package.name + : pkg.manifest.package.namespace_ + "." + pkg.manifest.package.name, + .memberPath = mp, + .root = pkg.root, + .productDir = *pkg.memberProducts, + .manifest = pkg.manifest, + }); + break; + } + } + } return {}; } @@ -365,11 +400,34 @@ static std::expected step13_link_forms(PrepareState& state, B } static std::expected step13_make_plan(PrepareState& state, BuildContext& ctx) { + // A workspace plan links each member's closure with that closure's own + // flags (workspace design 2026-09-29 §15): each package's link flags, + // its build program's included, with search paths made absolute against + // the package, as the root's pooled list holds them outside a workspace. + if (state.workspacePlan()) + for (std::size_t i = 1; i < state.packages.size(); ++i) + state.packages[i].linkUsage.ldflags = normalized_dependency_ldflags( + state.packages[i].root, state.packages[i].manifest.buildConfig.ldflags); auto planResult = mcpp::build::make_plan(*state.m, *state.tc, state.fp, state.scan.graph, state.report.topoOrder, state.packages, *state.root, ctx.outputDir, state.stdBmiPath, state.stdObjectPath, state.storeRoots); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); + // The request the graph is planned for (§3, §15): the plan's members and + // the requested features, which name no directory. + if (state.workspacePlan()) { + auto join = [](const std::vector& v) { + std::string out; + for (auto const& x : v) { if (!out.empty()) out += '\x1e'; out += x; } + return out; + }; + ctx.workspaceGroup = join(state.selectedMemberPaths); + ctx.workspaceRequest = state.overrides.workspace_request.empty() + ? ctx.workspaceGroup : join(state.overrides.workspace_request); + } + ctx.plan.requestTag = mcpp::build::request_tag( + ctx.workspaceGroup, + state.workspacePlan() ? state.requestedFeatures : state.overrides.features); // SPEC-007 R4.3: a declared deploy outranks a search directory's file of // the same name, and a difference between the two is stated ONCE, by the // post-link placement edge, through the edge-advice channel @@ -2127,7 +2185,8 @@ std::expected phase13_finish(PrepareState& state) { ctx.runtimeSelection = state.runtimeSelection; ctx.runtimeBinding = state.runtimeBindingSnapshot; ctx.profile = state.effectiveProfile; - ctx.activeFeatureRequest = state.overrides.features; + ctx.activeFeatureRequest = state.workspacePlan() ? state.requestedFeatures + : state.overrides.features; ctx.compilerChoice = { std::string(tc_origin_name(state.tcOrigin)), state.graphCompilerRequiredBy, state.graphCompilerReplaced.empty() ? state.pinReplacedDefault @@ -2158,4 +2217,13 @@ std::expected phase13_finish(PrepareState& state) { return ctx; } +void focus_on_member(BuildContext& ctx) { + if (ctx.workspaceMembers.size() != 1) return; + auto& m = ctx.workspaceMembers.front(); + for (auto& g : ctx.plan.linkGroups) + if (g.member == m.name) { swap_link_group(ctx.plan, g); break; } + ctx.manifest = m.manifest; + ctx.projectRoot = m.root; +} + } // namespace mcpp::build diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 5bef5bfd..68078605 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -121,8 +121,19 @@ std::expected step13_lockfile(PrepareState& state, BuildConte } // Git deps: root-declared only, unchanged (see the note above). - for (auto const& [name, spec] : state.m->dependencies) { + // The root's git dependencies, and in a workspace plan each selected + // member's (§15). + std::vector> gitDeps; + for (auto const& [name, spec] : state.m->dependencies) gitDeps.emplace_back(name, spec); + if (state.workspacePlan()) + for (std::size_t i = 1; i < state.packages.size(); ++i) + if (state.packages[i].memberProducts) + for (auto const& [name, spec] : state.packages[i].manifest.dependencies) + gitDeps.emplace_back(name, spec); + std::set gitLocked; + for (auto const& [name, spec] : gitDeps) { if (!spec.isGit()) continue; + if (!gitLocked.insert(name).second) continue; mcpp::lockfile::LockedPackage lp; lp.name = name; lp.version = spec.gitRev; @@ -169,6 +180,35 @@ std::expected step13_lockfile(PrepareState& state, BuildConte lp.hash = mcpp::pm::index_package_digest(sourceIndex, lp.name, lp.version); lock.packages.push_back(std::move(lp)); } + // A workspace's lock is at its root and records every member's + // resolution (§15). A plan of some of its members updates the entries + // it resolved and keeps the others; a plan of all of them writes the + // whole record. + bool partialWorkspace = false; + if (state.workspacePlan() && state.wsManifest) { + std::set all(state.wsManifest->workspace.members.begin(), + state.wsManifest->workspace.members.end()); + if (!state.wsManifest->package.name.empty()) all.insert("."); + std::set planned(state.selectedMemberPaths.begin(), + state.selectedMemberPaths.end()); + for (auto const& mp : state.overrides.workspace_request) planned.insert(mp); + partialWorkspace = !std::ranges::includes(planned, all); + } + if (partialWorkspace) { + if (auto prior = mcpp::lockfile::load(state.workRoot / "mcpp.lock")) { + auto keyOf = [](const mcpp::lockfile::LockedPackage& p) { + return p.namespace_ + "\x1f" + p.name; + }; + std::set resolvedHere; + for (auto const& p : lock.packages) resolvedHere.insert(keyOf(p)); + for (auto const& p : prior->packages) + if (!resolvedHere.contains(keyOf(p))) lock.packages.push_back(p); + std::set indicesHere; + for (auto const& i : lock.indices) indicesHere.insert(i.name); + for (auto const& i : prior->indices) + if (!indicesHere.contains(i.name)) lock.indices.push_back(i); + } + } if (!lock.packages.empty() || !lock.indices.empty()) { auto lockPath = state.workRoot / "mcpp.lock"; // `--locked` ASSERTS THAT THIS RESOLUTION IS THE RECORDED ONE. @@ -206,8 +246,11 @@ std::expected step13_lockfile(PrepareState& state, BuildConte if (it == was.end()) drift.push_back(k + " " + v + " (not in the lock)"); else if (it->second != v) drift.push_back(k + " " + it->second + " -> " + v); } - for (auto const& [k, v] : was) - if (!now.contains(k)) drift.push_back(k + " " + v + " (no longer resolved)"); + // A plan of some of a workspace's members resolves some of + // the lock; what it did not resolve is not drift. + if (!partialWorkspace) + for (auto const& [k, v] : was) + if (!now.contains(k)) drift.push_back(k + " " + v + " (no longer resolved)"); if (!drift.empty()) { std::string msg = "--locked was given and this resolution " "differs from mcpp.lock:"; diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 0f444874..b6bdc7d0 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -304,6 +304,10 @@ struct PrepareState { // which a root receives; in a workspace plan every selected member does. std::vector profileCflags, profileCxxflags; bool workspacePlan() const { return !selectedMemberPaths.empty(); } + // `--features` as the command gave it. A workspace plan hands the tokens to + // its members and clears `overrides.features`; the request is still what + // the build was asked for, and what its fast-path record names. + std::string requestedFeatures; // A package of a workspace plan that is a member of the workspace, selected // or reached as another member's dependency (the workspace's own package // included): its build program runs where a root's does (§15). @@ -535,6 +539,10 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta // plan.cpp: the member path of a package root within the workspace this build // runs in, or empty (W3). std::string workspace_member_of(const PrepareState& state, const std::filesystem::path& root); +// graph.cpp: a dependency's link flags as its consumer's link reads them -- +// word by word, each search path made absolute against the package. +std::vector normalized_dependency_ldflags( + const std::filesystem::path& depRoot, const std::vector& ldflags); std::expected phase1_toolchain_spec_and_axes(PrepareState& state); std::expected phase2_define_toolchain_resolver(PrepareState& state); std::expected phase3_xlings_before_graph(PrepareState& state); diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index a13e9f69..8934e452 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -38,58 +38,156 @@ import mcpp.wire; namespace mcpp::cli { -// Decide whether a build/test invocation fans out over workspace members, and -// if so which. Fan out when `--workspace` is given, or at a *virtual* workspace +// Decide whether a build/test invocation acts on several workspace members, and +// if so which. It does when `--workspace` is given, or at a *virtual* workspace // root with no `-p` (the intuitive "act on the whole workspace"). Returns the -// member paths to iterate, or nullopt for the single-package / single-`-p` / -// rooted-bare path (handled by the existing per-package pipeline). +// member paths as `[workspace] members` writes them -- a rooted workspace's own +// package first, as "." (workspace design 2026-09-29 §7.1) -- or nullopt for +// the single-package / single-`-p` / rooted-bare path. Inside a member, the +// workspace is the one that lists it. std::optional> workspace_fanout_members(bool wantAll, const std::string& package_filter) { auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) return std::nullopt; auto m = mcpp::manifest::load(*root / "mcpp.toml"); + if (m && !m->workspace.present && wantAll) { + auto wsRoot = mcpp::project::find_workspace_root(*root); + if (wsRoot.empty()) return std::nullopt; + m = mcpp::manifest::load(wsRoot / "mcpp.toml"); + } if (!m || !m->workspace.present || m->workspace.members.empty()) return std::nullopt; bool virtualWs = m->package.name.empty(); - if (wantAll || (virtualWs && package_filter.empty())) - return m->workspace.members; - return std::nullopt; + if (!(wantAll || (virtualWs && package_filter.empty()))) return std::nullopt; + std::vector members; + if (!virtualWs) members.push_back("."); + members.insert(members.end(), m->workspace.members.begin(), m->workspace.members.end()); + return members; +} + +// The workspace a build command acts on and the members it selects (workspace +// design 2026-09-29 §7.1, §15): `--workspace`, and a virtual root without +// `-p`, select every member (a rooted workspace's own package first, as "."); +// `-p X` selects X; a command in a member's directory selects that member; a +// command at a rooted workspace's root selects the workspace's own package. +// nullopt outside a workspace. +struct WorkspaceSelection { + std::filesystem::path root; + std::vector members; +}; +std::expected, std::string> +workspace_selection(bool wantAll, const std::string& package_filter) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) return std::optional{}; + auto m = mcpp::manifest::load(*root / "mcpp.toml", {.insideWorkspace = true}); + if (!m) return std::optional{}; + WorkspaceSelection sel{*root, {}}; + std::string inside; + if (!m->workspace.present) { + auto wsRoot = mcpp::project::find_workspace_root(*root); + if (wsRoot.empty()) return std::optional{}; + sel.root = wsRoot; + m = mcpp::manifest::load(wsRoot / "mcpp.toml"); + if (!m || !m->workspace.present) return std::optional{}; + const auto rel = root->lexically_normal() + .lexically_relative(wsRoot.lexically_normal()); + for (auto const& mp : m->workspace.members) + if (std::filesystem::path(mp).lexically_normal() == rel) inside = mp; + if (inside.empty()) return std::optional{}; + } + const bool rooted = !m->package.name.empty(); + std::vector all; + if (rooted) all.push_back("."); + all.insert(all.end(), m->workspace.members.begin(), m->workspace.members.end()); + if (wantAll) { sel.members = std::move(all); return sel; } + if (!package_filter.empty()) { + auto dir = mcpp::project::resolve_member_dir(*m, sel.root, package_filter); + if (!dir) return std::unexpected(dir.error()); + auto rel = dir->empty() ? std::string(".") + : dir->lexically_normal().lexically_relative(sel.root.lexically_normal()).generic_string(); + if (rel.empty()) rel = "."; + for (auto const& mp : m->workspace.members) + if (std::filesystem::path(mp).lexically_normal() == std::filesystem::path(rel)) + rel = mp; + sel.members = {rel}; + return sel; + } + if (!inside.empty()) { sel.members = {inside}; return sel; } + if (!rooted) { sel.members = std::move(all); return sel; } + sel.members = {"."}; + return sel; +} + +// The workspace a fan-out acts on, and its members grouped by configuration +// (workspace design 2026-09-29 §15): members whose root-position values are +// equal are planned together, in one graph, in one build directory. +std::expected>, std::string> +workspace_groups(const std::filesystem::path& wsRoot, const std::vector& members) { + auto ws = mcpp::manifest::load(wsRoot / "mcpp.toml"); + if (!ws) return std::unexpected(ws.error().format()); + std::vector> groups; + std::map byKey; + for (auto const& mp : members) { + auto mm = mcpp::project::load_member_manifest(*ws, wsRoot, mp); + if (!mm) return std::unexpected(mm.error()); + const auto key = mcpp::project::root_position_key(*mm); + auto [it, fresh] = byKey.try_emplace(key, groups.size()); + if (fresh) groups.emplace_back(); + groups[it->second].push_back(mp); + } + return groups; } // run_build_plan, wrapped in the project's `[hooks]` lifecycle (#496). // -// The hooks come off the context's own manifest, so they are the ones belonging -// to the package being built — which in a workspace fan-out is the MEMBER, once -// per member. The lifecycle is deliberately paired: build_finished/build_failed -// are only ever reached after build_start has run, so a project that could not -// be prepared at all (bad manifest, unresolvable dependency, no toolchain) -// fires nothing — its hook programs may be exactly what preparation failed to -// install. +// The hooks are those of the package being built: the root's, or in a +// workspace plan those of each selected member, in selection order (workspace +// design 2026-09-29 §15), around the one build of the plan. The lifecycle is +// deliberately paired: build_finished/build_failed are only ever reached after +// build_start has run, so a project that could not be prepared at all (bad +// manifest, unresolvable dependency, no toolchain) fires nothing — its hook +// programs may be exactly what preparation failed to install. int run_build_with_hooks(mcpp::build::BuildContext& ctx, bool verbose, bool no_cache, std::string_view targetOverride) { - auto const& hooks = ctx.manifest.hooks; + struct Subject { + const mcpp::manifest::Hooks* hooks; + std::filesystem::path root; + }; + std::vector subjects; + if (ctx.workspaceMembers.empty()) + subjects.push_back({&ctx.manifest.hooks, ctx.projectRoot}); + else + for (auto const& m : ctx.workspaceMembers) subjects.push_back({&m.manifest.hooks, m.root}); // `during_build` opens first and closes last: its interval is the one that // spans everything below. Its output is discarded unless --verbose, which // is the only way it could interleave into a compiler diagnostic. - mcpp::hooks::Span span(hooks, ctx.projectRoot, /*inheritOutput=*/verbose); - if (!span.ok()) return 1; + std::vector> spans; + for (auto const& sub : subjects) { + spans.push_back(std::make_unique( + *sub.hooks, sub.root, /*inheritOutput=*/verbose)); + if (!spans.back()->ok()) return 1; + } - if (!mcpp::hooks::invoke(hooks, mcpp::hooks::Event::BuildStart, - ctx.projectRoot)) - return 1; + for (auto const& sub : subjects) + if (!mcpp::hooks::invoke(*sub.hooks, mcpp::hooks::Event::BuildStart, sub.root)) + return 1; int rc = mcpp::build::run_build_plan(ctx, verbose, no_cache, targetOverride); // Closed BEFORE the terminal hook. A "build finished" sound competing with // the background music it replaces is the ordering this line settles. - bool spanOk = span.finish(); + bool spanOk = true; + for (auto it = spans.rbegin(); it != spans.rend(); ++it) + spanOk = (*it)->finish() && spanOk; auto terminalEvent = rc == 0 ? mcpp::hooks::Event::BuildFinished : mcpp::hooks::Event::BuildFailed; // The build's own exit code outranks the hook's: `mcpp build` returning // "the notifier failed" for a compile error would answer a question nobody // asked. A hook failure only decides the exit code of a build that worked. - bool hookOk = mcpp::hooks::invoke(hooks, terminalEvent, ctx.projectRoot); + bool hookOk = true; + for (auto const& sub : subjects) + hookOk = mcpp::hooks::invoke(*sub.hooks, terminalEvent, sub.root) && hookOk; return rc != 0 ? rc : ((spanOk && hookOk) ? 0 : 1); } @@ -173,26 +271,80 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { return mcpp::build::run_configure_plan(*ctx, verbose); }; - // Workspace fan-out: build every member, one per the existing per-package - // pipeline (continue-on-failure; first non-zero exit wins). Checked before - // the fast path, which is single-package only. - if (auto members = workspace_fanout_members(parsed.is_flag_set("workspace"), - ov.package_filter)) { - int rc = 0; - for (auto& mp : *members) { - mcpp::build::BuildOverrides mo = ov; - mo.package_filter = mp; - if (configure_only) { - int r = configure_member(std::move(mo), mp); + // A workspace: the selected members, planned by configuration group + // (workspace design 2026-09-29 §15). Each group is one plan, one graph and + // one build directory, so a member that others use is compiled once. + // Groups are independent (a member reached from two groups is a node of + // each), and a failed group does not stop the others; the first non-zero + // exit wins. + auto selection = workspace_selection(parsed.is_flag_set("workspace"), ov.package_filter); + if (!selection) { std::println(stderr, "error: {}", selection.error()); return 2; } + if (*selection) { + auto const& members = (*selection)->members; + if (configure_only) { + int rc = 0; + for (auto const& mp : members) { + mcpp::build::BuildOverrides mo = ov; + mo.package_filter = mp; + int r = configure_member(std::move(mo), members.size() > 1 ? mp : ""); if (r != 0) rc = r; - continue; } + return rc; + } + auto groups = workspace_groups((*selection)->root, members); + if (!groups) { std::println(stderr, "error: {}", groups.error()); return 2; } + std::vector request; + for (auto const& g : *groups) + for (auto const& mp : g) request.push_back(mp); + const bool plain = !print_fp && ov.target_triple.empty() && !ov.force_static + && ov.profile.empty() && ov.features.empty() && !ov.strict + && ov.capabilities.empty() && ov.cache_mode.empty() && ov.accel.empty(); + if (plain) { + if (auto rc = mcpp::build::try_fast_workspace_build( + (*selection)->root, *groups, verbose, no_cache)) + return *rc; + } + // Every group is planned first, one after another: planning runs + // build programs and narrates what it resolves. The groups' builds + // are then independent (§6), and run at the same time, each with a + // static share of the machine's jobs, so the total stays within what + // one build would use. + int rc = 0; + std::vector contexts; + for (auto const& g : *groups) { + mcpp::build::BuildOverrides mo = ov; + mo.package_filter.clear(); + mo.project_root = (*selection)->root; + mo.workspace_members = g; + mo.workspace_request = request; auto ctx = mcpp::build::prepare_build(print_fp, /*includeDevDeps=*/false, /*extraTargets=*/{}, mo); - if (!ctx) { std::println(stderr, "error: {}: {}", mp, ctx.error()); rc = 2; continue; } - int r = run_build_with_hooks(*ctx, verbose, no_cache, mo.target_triple); - if (r != 0) rc = r; + if (!ctx) { std::println(stderr, "error: {}", ctx.error()); rc = 2; continue; } + contexts.push_back(std::move(*ctx)); + } + if (contexts.size() == 1) { + const int r = run_build_with_hooks(contexts.front(), verbose, no_cache, + ov.target_triple); + return r != 0 ? r : rc; + } + const std::size_t hw = std::max(1u, std::thread::hardware_concurrency()); + for (auto& c : contexts) { + const std::size_t want = c.plan.scheduleNinjaJobs > 0 + ? static_cast(c.plan.scheduleNinjaJobs) : hw + 2; + c.plan.scheduleNinjaJobs = static_cast( + std::max(1, want / contexts.size())); + } + std::vector results(contexts.size(), 0); + { + std::vector builds; + for (std::size_t i = 0; i < contexts.size(); ++i) + builds.emplace_back([&, i] { + results[i] = run_build_with_hooks(contexts[i], verbose, no_cache, + ov.target_triple); + }); } + for (int r : results) + if (r != 0 && rc == 0) rc = r; return rc; } diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 5dd669df..fe0c5b89 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -153,12 +153,16 @@ export int build_and_pack_library(const std::string& targetName, if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } // ── the target, and what its kind means here ────────────────── + // In a workspace plan the package is its one selected member + // (workspace design 2026-09-29 §15). + const auto& subject = ctx->workspaceMembers.size() == 1 + ? ctx->workspaceMembers.front().manifest : ctx->manifest; const mcpp::manifest::Target* target = nullptr; - for (auto const& t : ctx->manifest.targets) + for (auto const& t : subject.targets) if (t.name == targetName) { target = &t; break; } if (!target) { std::string names; - for (auto const& t : ctx->manifest.targets) { + for (auto const& t : subject.targets) { if (!names.empty()) names += ", "; names += t.name; } @@ -195,6 +199,10 @@ export int build_and_pack_library(const std::string& targetName, mcpp::ui::error(br.error().message); return 1; } + // Everything below reads the package being packed. + const auto kept = *target; + mcpp::build::focus_on_member(*ctx); + target = &kept; std::filesystem::path artifact; std::filesystem::path importLib; for (auto const& lu : ctx->plan.linkUnits) { diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 9fd97899..10d129e1 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -275,8 +275,11 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // Manifest may override mode only when neither --mode nor an // equivalent flag (--target *-musl → static) was given. - if (!modeFromUser && !ctx->manifest.packConfig.defaultMode.empty()) { - if (auto m = mcpp::pack::parse_mode(ctx->manifest.packConfig.defaultMode)) + // A workspace plan's subject is its one selected member (§15). + const auto& subjectManifest = ctx->workspaceMembers.size() == 1 + ? ctx->workspaceMembers.front().manifest : ctx->manifest; + if (!modeFromUser && !subjectManifest.packConfig.defaultMode.empty()) { + if (auto m = mcpp::pack::parse_mode(subjectManifest.packConfig.defaultMode)) opts.mode = *m; } @@ -370,6 +373,9 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, mcpp::ui::error(br.error().message); return PackOutcome{1}; } + // Everything below reads the package being packed: in a workspace plan, + // its selected member (workspace design 2026-09-29 §15). + mcpp::build::focus_on_member(*ctx); // ─── Pick the main binary target ───────────────────────────────── // diff --git a/src/project.cppm b/src/project.cppm index 122729b6..75394c8e 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -559,6 +559,15 @@ resolve_member_dir(const mcpp::manifest::Manifest& rootManifest, const std::filesystem::path& rootDir, std::string_view package_filter) { if (!rootManifest.workspace.present) return std::filesystem::path{}; + // "." and the rooted workspace's own package name select that package + // (workspace design 2026-09-29 §7.1): it is a member of every command + // that acts on the whole workspace, and `-p` names it like any other. + if (!rootManifest.package.name.empty() + && (package_filter == "." || package_filter == rootManifest.package.name + || (!rootManifest.package.namespace_.empty() + && package_filter == rootManifest.package.namespace_ + "." + + rootManifest.package.name))) + return rootDir; if (package_filter.empty()) { if (rootManifest.package.name.empty()) { return std::unexpected(std::string( @@ -768,7 +777,6 @@ virtual_workspace_root(const mcpp::manifest::Manifest& workspace, v.package.virtualRoot = true; v.package.standard = first.package.standard; v.package.standardDeclared = first.package.standardDeclared; - v.package.mcppFloor = workspace.workspace.inherited.mcppFloor; v.language.standard = first.language.standard; v.cppStandard = first.cppStandard; v.toolchain = first.toolchain; diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index 3b84f14b..d6fd45dd 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -112,7 +112,7 @@ namespace pinned { // no output (mcpp#693), and under an MCPP_HOME outside it the xlings mcpp // vendors could not initialise its sandbox. It now declares the UTF-8 code // page, as mcpp.exe does. - inline constexpr std::string_view kXlingsVersion = "2026.9.28.2"; + inline constexpr std::string_view kXlingsVersion = "2026.9.29.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } diff --git a/tests/e2e/120_ws_root_indices.sh b/tests/e2e/120_ws_root_indices.sh index a5a2466b..8e727478 100755 --- a/tests/e2e/120_ws_root_indices.sh +++ b/tests/e2e/120_ws_root_indices.sh @@ -80,9 +80,11 @@ x = { path = "local-index" } ylib = { path = "ylib" } EOF +# A workspace's project-local payloads and its lock live at its root: the +# workspace is the unit of build (workspace design 2026-09-29 §15). mkdir -p member-a/src \ - member-a/.mcpp/.xlings/data/xpkgs/x.widget2/1.0.0/src -cat > member-a/.mcpp/.xlings/data/xpkgs/x.widget2/1.0.0/src/widget2.cppm <<'EOF' + .mcpp/.xlings/data/xpkgs/x.widget2/1.0.0/src +cat > .mcpp/.xlings/data/xpkgs/x.widget2/1.0.0/src/widget2.cppm <<'EOF' export module widget2; export int widget2_value() { @@ -129,8 +131,8 @@ EOF exit 1 } -grep -q '\[package\."x.widget2"\]' member-a/mcpp.lock || { - cat member-a/mcpp.lock 2>/dev/null || true +grep -q '\[package\."x.widget2"\]' mcpp.lock || { + cat mcpp.lock 2>/dev/null || true echo "FAIL: expected x.widget2 lock entry" exit 1 } diff --git a/tests/e2e/211_configure_only_cdb.sh b/tests/e2e/211_configure_only_cdb.sh index a7afb0e2..5f5be208 100755 --- a/tests/e2e/211_configure_only_cdb.sh +++ b/tests/e2e/211_configure_only_cdb.sh @@ -158,8 +158,10 @@ test_bin="$bindir/smoke$exe_suffix" exit 1 } -# A virtual workspace is configured member-by-member, with each member's CDB -# scoped to its own package. `-p` must select the same scope as normal build. +# A workspace has one compile database per configuration, at its root +# (workspace design 2026-09-29 §7.1): configuring every member fills it with +# every member's units, and `-p` configures that member's closure into the +# same database. mkdir -p "$TMP/ws/a/src" "$TMP/ws/a/tests" "$TMP/ws/b/src" "$TMP/ws/b/tests" cat > "$TMP/ws/mcpp.toml" <<'EOF' [workspace] @@ -180,21 +182,24 @@ cd "$TMP/ws" cat configure-workspace.log exit 1 } +cdb=compile_commands.json +[[ -s "$cdb" ]] || { echo "missing $cdb"; exit 1; } for member in a b; do - cdb="$member/compile_commands.json" - [[ -s "$cdb" ]] || { echo "missing $cdb"; exit 1; } - grep -q "src[\\/][\\/]*main.cpp" "$cdb" || { cat "$cdb"; exit 1; } - grep -q "tests[\\/][\\/]*main.cpp" "$cdb" || { cat "$cdb"; exit 1; } + [[ ! -e "$member/compile_commands.json" ]] || { + echo "$member/compile_commands.json written beside the workspace's database"; exit 1; } + grep -q "$member[\\/][\\/]*src[\\/][\\/]*main.cpp" "$cdb" || { cat "$cdb"; exit 1; } + grep -q "$member[\\/][\\/]*tests[\\/][\\/]*main.cpp" "$cdb" || { cat "$cdb"; exit 1; } done -rm -f a/compile_commands.json +# From a clean workspace, `-p a` configures a's closure and nothing of b. +rm -rf target compile_commands.json "$MCPP" build --configure-only -p a > configure-member.log 2>&1 || { cat configure-member.log exit 1 } -grep -q 'src[\/][\/]*main.cpp' a/compile_commands.json || { cat a/compile_commands.json; exit 1; } -if grep -q 'b[\/][\/]*src[\/][\/]*main.cpp' a/compile_commands.json; then - echo "-p a leaked member b into the CDB" - cat a/compile_commands.json +grep -q 'a[\/][\/]*src[\/][\/]*main.cpp' compile_commands.json || { cat compile_commands.json; exit 1; } +if grep -q 'b[\/][\/]*src[\/][\/]*main.cpp' compile_commands.json; then + echo "-p a configured member b" + cat compile_commands.json exit 1 fi diff --git a/tests/e2e/218_module_extensions_graph_shape.sh b/tests/e2e/218_module_extensions_graph_shape.sh index c95cb7c5..62580707 100755 --- a/tests/e2e/218_module_extensions_graph_shape.sh +++ b/tests/e2e/218_module_extensions_graph_shape.sh @@ -23,17 +23,14 @@ # to a full prepare for the wrong reason — the assertion below would hold # while proving nothing. # -# * Part 2 — the key must reach the fingerprint. +# * Part 2 — the key must reach the graph. # # `module_extensions` decides which units emit a BMI and which objects link -# unconditionally, i.e. it is a build VARIANT. mcpp.toml's mtime alone only -# protects the fast path inside one output dir; it does not stop a BMI -# cache entry built under one classification from being served under -# another. Changing the key must land in a different `target///`. -# -# (Contrast `[build] build_program_timeout`, which is deliberately NOT -# fingerprinted — it changes no edge, and folding it in would make raising a -# timeout rebuild the whole project.) +# unconditionally. It is an attribute of the package that declares it, not +# of the configuration (workspace design 2026-09-29 §3): the build +# directory stays, and the graph written into it must classify the files +# the key now names. A `.ccm` file the key starts naming becomes a module +# interface with its own BMI edge, in the same directory. set -e TMP=$(mktemp -d) @@ -90,7 +87,8 @@ out="$("$MCPP" run 2>&1)" [[ "$out" == *"42"* ]] || { echo "FAIL: expected 42 (41+1), got: $out"; exit 1; } echo " ok: the new dependency edge is real (41+1 = 42)" -# ── 2. Changing module_extensions must change the fingerprint ────────────── +# ── 2. Changing module_extensions must change the graph ──────────────────── +printf 'export module gshape.extra;\nexport auto extra() -> int { return 2; }\n' > src/extra.ccm cat > mcpp.toml <<'EOF' [package] name = "gshape" @@ -103,12 +101,15 @@ EOF "$MCPP" build > b3.log 2>&1 || { cat b3.log; echo "FAIL: build after key change"; exit 1; } FP_AFTER="$(fingerprint)" -[[ "$FP_BEFORE" != "$FP_AFTER" ]] || { - echo "FAIL: module_extensions changed but the output dir did not" - echo " before=$FP_BEFORE after=$FP_AFTER" - echo " (the key is missing from the canonical compile-flags string)" +[[ "$FP_BEFORE" == "$FP_AFTER" ]] || { + echo "FAIL: module_extensions moved the build directory ($FP_BEFORE -> $FP_AFTER)" + echo " (a package attribute entered the configuration's name)" + exit 1; } +ninja_file="$(find target -path "*/$FP_AFTER/build.ninja" | head -1)" +grep -q 'gshape\.extra' "$ninja_file" || { + echo "FAIL: the .ccm named by module_extensions is not a module interface of the graph" exit 1; } -echo " ok: the key is fingerprinted ($FP_BEFORE -> $FP_AFTER)" +echo " ok: the key reaches the graph in the same directory ($FP_AFTER)" # ── 3. A dead entry is reported, not silently ignored ────────────────────── # diff --git a/tests/e2e/321_workspace_inheritance.sh b/tests/e2e/321_workspace_inheritance.sh index e7d988f6..583f42fd 100755 --- a/tests/e2e/321_workspace_inheritance.sh +++ b/tests/e2e/321_workspace_inheritance.sh @@ -56,13 +56,24 @@ for m in silent pinned adds; do printf '#include \n#if !defined(FROM_WORKSPACE) || !defined(SHARED_HEADER_FOUND)\n#error "workspace [build] did not reach the member"\n#endif\nint main(){return 0;}\n' > "$m/src/main.cpp" done +# The workspace keeps one compile database per configuration at its root +# (workspace design 2026-09-29 §7.1); a member's command is its entry there. +entry_of() { # $1 = member dir: the member's main.cpp entry, one word per line + python3 -c ' +import json, sys +for e in json.load(open("compile_commands.json")): + if e["file"].replace(chr(92), "/").endswith(sys.argv[1] + "/src/main.cpp"): + print("\n".join(e.get("arguments") or e["command"].split())) + break +' "$1" +} flags_of() { # $1 = member dir - grep -oE '\-std=c\+\+[0-9a-z]+|\-DFROM_[A-Z]+=1' "$1/compile_commands.json" \ - | sort -u | tr '\n' ' ' + grep -oE '\-std=c\+\+[0-9a-z]+|\-DFROM_[A-Z]+=1' "$1.entry" | sort -u | tr '\n' ' ' } for m in silent pinned adds; do "$MCPP" build -p "$m" > "build_$m.log" 2>&1 || { cat "build_$m.log"; exit 1; } + entry_of "$m" > "$m.entry" done # ── silent: inherits both ─────────────────────────────────────────────────── @@ -77,7 +88,7 @@ case "$got" in *) echo "FAIL: member 'silent' did not inherit [workspace.package] standard" echo " got: $got"; exit 1 ;; esac -grep -q '"version": *"0.4.2"' silent/target/*/*/resolution.json 2>/dev/null \ +grep -q '"version": *"0.4.2"' target/*/*/resolution.json 2>/dev/null \ || grep -q "0.4.2" build_silent.log \ || { echo "FAIL: member 'silent' did not inherit [workspace.package] version" cat build_silent.log; exit 1; } @@ -112,8 +123,8 @@ done # own flag has to come after the workspace's. The CDB is pretty-printed with one # argument per line, so the question is which line number comes first — matching # both on a single line would silently never fire. -ws_at=$(grep -n 'FROM_WORKSPACE=1' adds/compile_commands.json | head -1 | cut -d: -f1) -mem_at=$(grep -n 'FROM_MEMBER=1' adds/compile_commands.json | head -1 | cut -d: -f1) +ws_at=$(grep -n 'FROM_WORKSPACE=1' adds.entry | head -1 | cut -d: -f1) +mem_at=$(grep -n 'FROM_MEMBER=1' adds.entry | head -1 | cut -d: -f1) [ -n "$ws_at" ] && [ -n "$mem_at" ] || { echo "FAIL: could not locate both flags in the compile command"; exit 1; } [ "$ws_at" -lt "$mem_at" ] || { @@ -123,6 +134,7 @@ mem_at=$(grep -n 'FROM_MEMBER=1' adds/compile_commands.json | head -1 | cut -d: # ── inside: the second inheritance site ───────────────────────────────────── ( cd silent && "$MCPP" build > ../build_inside.log 2>&1 ) \ || { cat build_inside.log; exit 1; } +entry_of silent > silent.entry got=$(flags_of silent) case "$got" in *"-DFROM_WORKSPACE=1"*"-std=c++26"* | *"-std=c++26"*"-DFROM_WORKSPACE=1"*) ;; diff --git a/tests/e2e/35_workspace.sh b/tests/e2e/35_workspace.sh index 8bac59e3..6645d465 100755 --- a/tests/e2e/35_workspace.sh +++ b/tests/e2e/35_workspace.sh @@ -94,15 +94,18 @@ echo "=== Building from workspace root ===" echo "workspace build: ok" # ── Verify the binary runs correctly ──────────────────── -# target/ is created in the member dir (apps/hello/target/), not workspace root. +# One build directory per configuration, at the workspace root; each +# member's products in its own product directory, bin// +# (workspace design 2026-09-29 §5). # On Windows (MINGW/MSYS) the binary has a .exe suffix OS="$(uname -s)" if [[ "$OS" == MINGW* || "$OS" == MSYS* || "$OS" == CYGWIN* ]]; then - BIN=$(find apps/hello/target -type f -name hello.exe | head -1) + BIN=$(find target -type f -path '*/bin/hello/hello.exe' | head -1) else - BIN=$(find apps/hello/target -type f -name hello | head -1) + BIN=$(find target -type f -path '*/bin/hello/hello' | head -1) fi -test -n "$BIN" || { echo "FAIL: hello binary not found"; exit 1; } +test -n "$BIN" || { echo "FAIL: hello binary not found in target///bin/hello/"; exit 1; } +test ! -e apps/hello/target || { echo "FAIL: a member directory received a build directory"; exit 1; } OUT=$("$BIN" 2>&1) echo "output: $OUT" test "$OUT" = "Hello, World!" || { echo "FAIL: unexpected output '$OUT'"; exit 1; } diff --git a/tests/e2e/770_workspace_member_as_dependency.sh b/tests/e2e/770_workspace_member_as_dependency.sh index 7f1f040b..2eccd7bd 100644 --- a/tests/e2e/770_workspace_member_as_dependency.sh +++ b/tests/e2e/770_workspace_member_as_dependency.sh @@ -128,12 +128,16 @@ EOF printf 'int lib_v();\nint main() { return lib_v() == 7 ? 0 : 1; }\n' > app/src/main.cpp # Each position is asserted right after its own build, so the root position's -# counts are read even when the sibling position does not build. +# counts are read even when the sibling position does not build. The database +# is removed before each build, so it holds that build's plan alone. for position in lib app; do + rm -f compile_commands.json target/*/*/compile_commands.json "$MCPP" build -p "$position" > "build-$position.log" 2>&1 \ || fail "A/B: -p $position" "build-$position.log" - cdb="$position/compile_commands.json" - [ -f "$cdb" ] || fail "A: no $cdb" + # One database per configuration, at the workspace root (workspace + # design 2026-09-29 §7.1); each position's plan writes it anew. + cdb="compile_commands.json" + [ -f "$cdb" ] || fail "A: no $cdb (-p $position)" for word in -DWS_DEF=1 -DWS_CXXFLAG=1 -DMEMBER_CXXFLAG=1 -DMEMBER_DEF=1 -DLEVEL=2; do n=$(count "$cdb" lib/lib.cpp "$word") [ "$n" = 1 ] || fail "A: $word occurs $n times in lib.cpp (-p $position)" diff --git a/tests/e2e/806_dash_p_resolves_the_package_first.sh b/tests/e2e/806_dash_p_resolves_the_package_first.sh index 9266f0f1..5bb07d11 100755 --- a/tests/e2e/806_dash_p_resolves_the_package_first.sh +++ b/tests/e2e/806_dash_p_resolves_the_package_first.sh @@ -45,12 +45,20 @@ cat > mcpp.toml <<'EOF' members = ["modules/base", "ns1/common", "ns2/common"] EOF +# Which member a `-p` selected is read from the workspace's compile database, +# which a build from a clean workspace fills with that member's closure alone +# (workspace design 2026-09-29 §7.1). +fresh() { rm -rf target compile_commands.json; } +built() { # built : the member's source is in the database + [ -f compile_commands.json ] && grep -q "$1/x.cpp" compile_commands.json +} + # ── package name, directory basename, and full path all select the member ── for filter in ws-base base modules/base; do - rm -rf modules/base/compile_commands.json + fresh "$MCPP" build -p "$filter" > "sel-$(basename "$filter").log" 2>&1 \ || fail "-p $filter did not build" "sel-$(basename "$filter").log" - [ -f modules/base/compile_commands.json ] \ + built modules/base \ || fail "-p $filter did not select modules/base" grep -qi 'warning' "sel-$(basename "$filter").log" \ && fail "-p $filter warned when no other member could conflict" \ @@ -66,10 +74,12 @@ grep -q 'ns1.ws-common' ambiguous.log \ grep -q 'ns2.ws-common' ambiguous.log \ || fail "the ambiguity refusal must name 'ns2.ws-common'" ambiguous.log +fresh "$MCPP" build -p ns1.ws-common > qns1.log 2>&1 || fail "-p ns1.ws-common did not build" qns1.log -[ -f ns1/common/compile_commands.json ] || fail "-p ns1.ws-common did not select ns1/common" +built ns1/common || fail "-p ns1.ws-common did not select ns1/common" +fresh "$MCPP" build -p ns2.ws-common > qns2.log 2>&1 || fail "-p ns2.ws-common did not build" qns2.log -[ -f ns2/common/compile_commands.json ] || fail "-p ns2.ws-common did not select ns2/common" +built ns2/common || fail "-p ns2.ws-common did not select ns2/common" echo "ok: the bare name is refused; each qualified name selects its own member" # ── a second member's directory basename collides with the first's package ─ @@ -78,13 +88,13 @@ cat > mcpp.toml <<'EOF' [workspace] members = ["modules/base", "ns1/common", "ns2/common", "dup/ws-base"] EOF -rm -rf modules/base/compile_commands.json +fresh "$MCPP" build -p ws-base > name-vs-dir.log 2>&1 \ || fail "-p ws-base did not build once a same-named directory existed" name-vs-dir.log -[ -f modules/base/compile_commands.json ] \ +built modules/base \ || fail "-p ws-base must still select the package 'ws-base' (modules/base)" -[ -f dup/ws-base/compile_commands.json ] \ +built dup/ws-base \ && fail "-p ws-base must not have built dup/ws-base" grep -qi 'warning' name-vs-dir.log \ || fail "-p ws-base must warn once a directory shares its spelling" name-vs-dir.log diff --git a/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh b/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh new file mode 100755 index 00000000..74d68b48 --- /dev/null +++ b/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh @@ -0,0 +1,150 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 833_a_workspace_is_one_graph_per_configuration.sh -- workspace design +# 2026-09-29 (§4, §5, §7, §11). +# +# A workspace is planned as one graph per configuration, with a virtual root +# that depends on the selected members. The criteria of §11: +# +# G1 a member two others use is compiled once in a workspace build; +# G2 `-p X` after `--workspace` compiles nothing: one build directory; +# G3 an edit to one member's flags recompiles that member only; +# G4 a member of another configuration (another C++ standard) builds, in +# its own build directory; +# G5 two members with a program of the same name both build, each in its +# product directory `bin//`; +# G6 two members with one package name in two namespaces get qualified +# product directories; +# G7 no member directory receives a build directory, and `clean --stale` +# removes the ones members held before, keeping `target/.build-mcpp/`. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" +EXE="" +case "$(uname -s)" in MINGW*|MSYS*|CYGWIN*) EXE=".exe" ;; esac + +cat > mcpp.toml <<'EOF' +[workspace] +members = ["core", "cli", "gui", "ns1/tool", "ns2/tool", "modern"] +EOF + +mkdir -p core/src cli/src gui/src ns1/tool/src ns2/tool/src modern/src +cat > core/mcpp.toml <<'EOF' +[package] +name = "core" +version = "0.1.0" + +[targets.core] +kind = "lib" +EOF +printf 'export module shared_core;\nexport int core_v() { return 40; }\n' > core/src/core.cppm + +for m in cli gui; do + cat > $m/mcpp.toml <\nimport shared_core;\nint main() { std::printf("%%d\\n", core_v() + 1); return 0; }\n' > cli/src/main.cpp +printf '#include \nimport shared_core;\nint main() { std::printf("%%d\\n", core_v() + 2); return 0; }\n' > gui/src/main.cpp + +for ns in ns1 ns2; do + cat > $ns/tool/mcpp.toml < $ns/tool/src/main.cpp +done + +cat > modern/mcpp.toml <<'EOF' +[package] +name = "modern" +version = "0.1.0" +standard = "c++26" + +[targets.modern] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int main() { return 0; }\n' > modern/src/main.cpp + +"$MCPP" build --workspace > ws.log 2>&1 || fail "the workspace build failed" ws.log + +# G4: two configurations, two build directories. +dirs=$(find target -mindepth 2 -maxdepth 2 -type d -path 'target/*/*' ! -name '.*' | sort) +[ "$(echo "$dirs" | wc -l | tr -d ' ')" = 2 ] || fail "G4: expected two build directories, got: $dirs" ws.log +main_dir="" +for d in $dirs; do + [ -f "$d/bin/cli/app$EXE" ] && main_dir="$d" +done +[ -n "$main_dir" ] || fail "G5: bin/cli/app not found in any build directory" ws.log +[ -f "$main_dir/bin/gui/app$EXE" ] || fail "G5: bin/gui/app not beside bin/cli/app" ws.log +other=$(echo "$dirs" | grep -v "^$main_dir$") +[ -f "$other/bin/modern/modern$EXE" ] || fail "G4: the c++26 member is not in its own build directory" ws.log + +# G5: each program is its member's. +[ "$("$main_dir/bin/cli/app$EXE")" = 41 ] || fail "G5: bin/cli/app is not cli's program" +[ "$("$main_dir/bin/gui/app$EXE")" = 42 ] || fail "G5: bin/gui/app is not gui's program" + +# G6: qualified product directories for one name in two namespaces. +[ -f "$main_dir/bin/ns1.tool/tool$EXE" ] || fail "G6: bin/ns1.tool/tool missing" ws.log +[ -f "$main_dir/bin/ns2.tool/tool$EXE" ] || fail "G6: bin/ns2.tool/tool missing" ws.log + +# G1: the core module is compiled once, in the one graph. +log="$main_dir/.ninja_log" +n=$(grep -c 'core\.m\.o' "$log" || true) +[ "$n" = 1 ] || fail "G1: core's object was built $n times" "$log" + +# G2: `-p cli` after `--workspace` compiles nothing. +before=$(wc -l < "$log") +"$MCPP" build -p cli > p.log 2>&1 || fail "G2: -p cli failed" p.log +after=$(wc -l < "$log") +[ "$before" = "$after" ] || fail "G2: -p cli recompiled after --workspace" "$log" + +# G3: an edit to gui's flags recompiles gui only. +sleep 1.1 +cat >> gui/mcpp.toml <<'EOF' + +[build] +cxxflags = ["-DGUI_EXTRA=1"] +EOF +before=$(wc -l < "$log") +"$MCPP" build --workspace > g3.log 2>&1 || fail "G3: the build after the edit failed" g3.log +tail -n +$((before + 1)) "$log" | cut -f4 > g3.edges +grep -q 'gui' g3.edges || fail "G3: gui was not rebuilt after its flags changed" g3.edges +if grep -Eq 'core|/cli/|_cli|ns1|ns2' g3.edges; then + fail "G3: an edit to gui's flags rebuilt another member" g3.edges +fi +[ -d "$main_dir" ] || fail "G3: the flag edit moved the build directory" + +# G7: no member directory holds a build directory; the old ones are stale. +for m in core cli gui ns1/tool ns2/tool modern; do + [ ! -d "$m/target/$(basename "$(dirname "$main_dir")")" ] \ + || fail "G7: $m received a build directory" +done +mkdir -p cli/target/x86_64-linux-gnu/0123456789abcdef cli/target/.build-mcpp +echo keep > cli/target/.build-mcpp/keep +"$MCPP" clean --stale > clean.log 2>&1 || fail "G7: clean --stale failed" clean.log +[ ! -d cli/target/x86_64-linux-gnu/0123456789abcdef ] \ + || fail "G7: clean --stale kept a member's old build directory" clean.log +[ -f cli/target/.build-mcpp/keep ] || fail "G7: clean --stale removed a member's build-program outputs" clean.log + +echo "PASS: 833_a_workspace_is_one_graph_per_configuration" diff --git a/tests/e2e/834_a_workspace_plans_once_and_replays_in_one_check.sh b/tests/e2e/834_a_workspace_plans_once_and_replays_in_one_check.sh new file mode 100755 index 00000000..34368783 --- /dev/null +++ b/tests/e2e/834_a_workspace_plans_once_and_replays_in_one_check.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash +# requires: unix-shell python3 +# 834_a_workspace_plans_once_and_replays_in_one_check.sh -- workspace design +# 2026-09-29 §14. +# +# 2026.9.28.3 planned a shared member as the root of a nested build, and the +# nested build met the same condition again: a chain of n members cost 2^n +# plans. Before it, every member was planned in full on every command. A +# workspace is now one plan per configuration and one fast-path record per +# selection and configuration. +# +# P1 `--workspace` over a chain of five libraries and a program resolves +# the toolchain once; +# P2 `-p app`, at the end of the chain, resolves it once; +# P3 with nothing changed, `--workspace`, `-p app` and a build inside the +# member are each replayed without a plan (no toolchain resolution) in +# well under a second. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" +PY=python3 + +cat > mcpp.toml <<'EOF' +[workspace] +members = ["lib1", "lib2", "lib3", "lib4", "lib5", "app"] +EOF +prev="" +for i in 1 2 3 4 5; do + mkdir -p lib$i/src + { + printf '[package]\nname = "lib%s"\nversion = "0.1.0"\n\n[targets.lib%s]\nkind = "lib"\n' "$i" "$i" + [ -n "$prev" ] && printf '\n[dependencies]\n%s = { path = "../%s" }\n' "$prev" "$prev" + } > lib$i/mcpp.toml + if [ -n "$prev" ]; then + printf 'export module lib%s;\nimport %s;\nexport int v%s() { return v%s() + 1; }\n' \ + "$i" "$prev" "$i" "${prev#lib}" > lib$i/src/lib$i.cppm + else + printf 'export module lib1;\nexport int v1() { return 1; }\n' > lib1/src/lib1.cppm + fi + prev=lib$i +done +mkdir -p app/src +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[dependencies]\nlib5 = { path = "../lib5" }\n' > app/mcpp.toml +printf 'import lib5;\nint main() { return v5() == 5 ? 0 : 1; }\n' > app/src/main.cpp + +resolutions() { grep -c "Resolving toolchain" "$1" || true; } + +# P1 +"$MCPP" build --workspace > p1.log 2>&1 || fail "P1: the workspace build failed" p1.log +[ "$(resolutions p1.log)" = 1 ] || fail "P1: --workspace resolved the toolchain $(resolutions p1.log) times" p1.log + +# P2 +rm -rf target +"$MCPP" build -p app > p2.log 2>&1 || fail "P2: -p app failed" p2.log +[ "$(resolutions p2.log)" = 1 ] || fail "P2: -p app resolved the toolchain $(resolutions p2.log) times" p2.log + +# P3: each command once to record it, then once more with nothing changed. +timed() { # timed : runs mcpp, prints the wall time in ms + local log="$1" dir="$2"; shift 2 + "$PY" - "$MCPP" "$dir" "$log" "$@" <<'EOF' +import subprocess, sys, time +mcpp, cwd, log, args = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4:] +t = time.monotonic() +with open(log, "w") as out: + rc = subprocess.call([mcpp] + args, cwd=cwd, stdout=out, stderr=subprocess.STDOUT) +print(int((time.monotonic() - t) * 1000) if rc == 0 else -1) +EOF +} +for sel in ws p in; do + case $sel in + ws) dir=.; args="build --workspace" ;; + p) dir=.; args="build -p app" ;; + in) dir=app; args="build" ;; + esac + "$MCPP" $args > /dev/null 2>&1 || true + (cd "$dir" && "$MCPP" $args > "$TMP/p3-$sel-first.log" 2>&1) || fail "P3: $args failed" "p3-$sel-first.log" + ms=$(timed "$TMP/p3-$sel.log" "$dir" $args) + [ "$ms" -ge 0 ] || fail "P3: the replay of '$args' failed" "p3-$sel.log" + [ "$(resolutions "p3-$sel.log")" = 0 ] || fail "P3: '$args' with nothing changed was planned" "p3-$sel.log" + [ "$ms" -lt 1000 ] || fail "P3: '$args' with nothing changed took ${ms} ms" "p3-$sel.log" +done + +echo "PASS: 834_a_workspace_plans_once_and_replays_in_one_check" diff --git a/tests/e2e/835_a_shared_library_is_placed_by_a_link.sh b/tests/e2e/835_a_shared_library_is_placed_by_a_link.sh new file mode 100755 index 00000000..5d2a5f80 --- /dev/null +++ b/tests/e2e/835_a_shared_library_is_placed_by_a_link.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +# requires: elf +# 835_a_shared_library_is_placed_by_a_link.sh -- workspace design 2026-09-29 +# §5.1, §5.3. +# +# A member's product directory is its program's runtime closure: the +# graph-built shared libraries the program loads are placed beside it. Two +# product directories that place one library share its bytes by a hard link +# where the file system allows one. +# +# L1 each program runs from its product directory; +# L2 the library in the two product directories is one file (one inode), +# or, where links are not possible, two copies with equal content. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" + +cat > mcpp.toml <<'EOF' +[workspace] +members = ["shlib", "p1", "p2"] +EOF +mkdir -p shlib/src p1/src p2/src +cat > shlib/mcpp.toml <<'EOF' +[package] +name = "shlib" +version = "0.1.0" + +[targets.shlib] +kind = "shared" +EOF +printf 'export module shlib;\nexport int shared_v() { return 7; }\n' > shlib/src/shlib.cppm +for p in p1 p2; do + cat > $p/mcpp.toml < $p/src/main.cpp +done + +"$MCPP" build --workspace > b.log 2>&1 || fail "the workspace build failed" b.log +dir=$(dirname "$(dirname "$(find target -path '*/bin/p1/p1' -type f | head -1)")") +[ -n "$dir" ] && [ -d "$dir" ] || fail "no product directory for p1" b.log + +# L1 +for p in p1 p2; do + lib=$(find "$dir/$p" -maxdepth 1 -name 'libshlib.so*' | head -1) + [ -n "$lib" ] || fail "L1: libshlib is not placed in bin/$p/" b.log + "$dir/$p/$p" || fail "L1: bin/$p/$p did not run from its product directory" +done + +# L2 +a=$(find "$dir/p1" -maxdepth 1 -name 'libshlib.so' | head -1) +b=$(find "$dir/p2" -maxdepth 1 -name 'libshlib.so' | head -1) +[ -n "$a" ] && [ -n "$b" ] || fail "L2: libshlib.so missing from a product directory" +ia=$(stat -c %i "$a" 2>/dev/null || stat -f %i "$a") +ib=$(stat -c %i "$b" 2>/dev/null || stat -f %i "$b") +if [ "$ia" != "$ib" ]; then + cmp -s "$a" "$b" || fail "L2: the two placements differ" + echo "note: the placements are copies on this file system" +fi + +echo "PASS: 835_a_shared_library_is_placed_by_a_link" diff --git a/tests/unit/test_graph_shape.cpp b/tests/unit/test_graph_shape.cpp index 88c63f55..b5ea4da7 100644 --- a/tests/unit/test_graph_shape.cpp +++ b/tests/unit/test_graph_shape.cpp @@ -39,14 +39,29 @@ std::filesystem::path write_graph(const std::string& first) { TEST(GraphShape, TheHeaderNamesShapeScheduleSelectionAndFormat) { EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false), - "# mcpp:graph=normal;schedule=none;accel=default;dist=none"); + "# mcpp:graph=normal;schedule=none;accel=default;dist=none;request="); EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::WithTests, "two-phase", true), - "# mcpp:graph=test;schedule=two-phase;accel=override;dist=none"); + "# mcpp:graph=test;schedule=two-phase;accel=override;dist=none;request="); // An empty format reads as "none" rather than as an empty field: the value // has to be a word, because `read_pack_format` returning "" already means // "this file predates the field", and the two must not collide. EXPECT_EQ(mcpp::build::header_line(mcpp::build::GraphShape::Normal, "none", false, "appimage"), - "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage"); + "# mcpp:graph=normal;schedule=none;accel=default;dist=appimage;request="); +} + +// Workspace design 2026-09-29 §3: the request a graph was planned for rides +// the same line, because the features and the workspace members no longer +// name the directory. One tag per request, whatever the spelling. +TEST(GraphShape, TheHeaderNamesTheRequest) { + using namespace mcpp::build; + const auto tag = request_tag("cli\x1egui", "b, a"); + EXPECT_EQ(tag, request_tag("cli\x1egui", "a,b")); + EXPECT_NE(tag, request_tag("cli", "a,b")); + EXPECT_NE(tag, request_tag("cli\x1egui", "a")); + EXPECT_EQ(read_request(write_graph(header_line(GraphShape::Normal, "none", false, {}, tag))), + tag); + EXPECT_EQ(read_request(write_graph("# mcpp:graph=normal;schedule=none;accel=default;dist=none")), + ""); } TEST(GraphShape, OnlyAPlainGraphWithTheManifestsVariantIsReplayed) { diff --git a/tests/unit/test_workspace_plan.cpp b/tests/unit/test_workspace_plan.cpp new file mode 100644 index 00000000..d6e78f7c --- /dev/null +++ b/tests/unit/test_workspace_plan.cpp @@ -0,0 +1,103 @@ +#include + +import std; +import mcpp.manifest; +import mcpp.project; + +// The unit-level statements of a workspace plan (workspace design 2026-09-29 +// §15): which member values separate two plans, what the virtual root holds, +// and where a member's products are placed. The end-to-end halves are +// tests/e2e/834_a_workspace_is_one_graph_per_configuration.sh and its +// neighbours. + +namespace { + +mcpp::manifest::Manifest member(std::string name) { + mcpp::manifest::Manifest m; + m.package.name = std::move(name); + m.package.version = "0.1.0"; + return m; +} + +mcpp::project::WorkspaceMember listed(std::string path, std::string ns, std::string name) { + return {.memberPath = path, .dir = path, .namespace_ = std::move(ns), + .name = std::move(name)}; +} + +} // namespace + +// A member's own flags, sources and dependencies are attributes of its node: +// they never separate it from another member. +TEST(WorkspacePlan, PackageAttributesDoNotSeparateMembers) { + auto a = member("a"); + auto b = member("b"); + b.buildConfig.cxxflags = {"-DONLY_B=1"}; + b.buildConfig.ldflags = {"-lm"}; + b.buildConfig.sources = {"src/**/*.cpp"}; + b.buildConfig.cStandard = "c17"; + b.dependencies["fmt"] = mcpp::manifest::DependencySpec{.version = "11.0.0"}; + EXPECT_EQ(mcpp::project::root_position_key(a), mcpp::project::root_position_key(b)); +} + +// What every node of one graph shares does separate them: the toolchain, the +// standard, the dialect flags, the target and the profile. +TEST(WorkspacePlan, RootPositionValuesSeparateMembers) { + const auto ref = mcpp::project::root_position_key(member("a")); + { auto m = member("a"); m.package.standard = "c++26"; + EXPECT_NE(mcpp::project::root_position_key(m), ref); } + { auto m = member("a"); m.toolchain.byPlatform["default"] = "gcc@16.1.0"; + EXPECT_NE(mcpp::project::root_position_key(m), ref); } + { auto m = member("a"); m.buildConfig.dialectCxxflags = {"-freflection"}; + EXPECT_NE(mcpp::project::root_position_key(m), ref); } + { auto m = member("a"); m.buildConfig.target = "x86_64-linux-musl"; + EXPECT_NE(mcpp::project::root_position_key(m), ref); } + { auto m = member("a"); m.buildConfig.defaultProfile = "release"; + EXPECT_NE(mcpp::project::root_position_key(m), ref); } + { auto m = member("a"); m.buildConfig.cxxRuntime = "static"; + EXPECT_NE(mcpp::project::root_position_key(m), ref); } +} + +// The virtual root carries the plan's values and nothing a package owns. +TEST(WorkspacePlan, TheVirtualRootHoldsNoPackageContent) { + mcpp::manifest::Manifest ws; + ws.workspace.present = true; + ws.workspace.members = {"a"}; + auto first = member("a"); + first.package.standard = "c++26"; + first.toolchain.byPlatform["default"] = "gcc@16.1.0"; + first.buildConfig.dialectCxxflags = {"-freflection"}; + first.buildConfig.cxxflags = {"-DMEMBER=1"}; + first.buildConfig.ldflags = {"-lmember"}; + first.buildConfig.sources = {"src/**/*.cpp"}; + first.targets.push_back(mcpp::manifest::Target{.name = "a"}); + first.dependencies["fmt"] = mcpp::manifest::DependencySpec{.version = "11.0.0"}; + + const auto v = mcpp::project::virtual_workspace_root(ws, first, "/ws"); + EXPECT_TRUE(v.package.virtualRoot); + EXPECT_EQ(v.package.standard, "c++26"); + EXPECT_EQ(v.toolchain.byPlatform.at("default"), "gcc@16.1.0"); + EXPECT_EQ(v.buildConfig.dialectCxxflags, first.buildConfig.dialectCxxflags); + EXPECT_TRUE(v.buildConfig.cxxflags.empty()); + EXPECT_TRUE(v.buildConfig.ldflags.empty()); + EXPECT_TRUE(v.buildConfig.sources.empty()); + EXPECT_TRUE(v.buildConfig.sourcesDeclared); + EXPECT_TRUE(v.targets.empty()); + EXPECT_TRUE(v.dependencies.empty()); + // The root's values and the group's are one statement. + EXPECT_EQ(mcpp::project::root_position_key(v), mcpp::project::root_position_key(first)); +} + +// A member's products are in `bin//`, qualified when another +// member of the workspace has the same name; the workspace's own package +// keeps `bin/`. +TEST(WorkspacePlan, ProductDirectoriesAreNamedByPackage) { + const std::vector all = { + listed("apps/cli", "", "cli"), + listed("ns1/common", "ns1", "common"), + listed("ns2/common", "ns2", "common"), + }; + EXPECT_EQ(mcpp::project::product_directory_name(all, "apps/cli"), "cli"); + EXPECT_EQ(mcpp::project::product_directory_name(all, "ns1/common"), "ns1.common"); + EXPECT_EQ(mcpp::project::product_directory_name(all, "ns2/common"), "ns2.common"); + EXPECT_EQ(mcpp::project::product_directory_name(all, "."), ""); +} From 687e154a89c040233c6b5076e7f0e70d38c3b7fa Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 05:20:15 +0800 Subject: [PATCH 04/11] Workspace build graph: member targets gated by features, the workspace package reads its own workspace dependencies, pooled flags kept for units no member owns, the group link line checked for hermeticity, runtime files placed in the product directory --- ...2026-09-29-workspace-build-graph-design.md | 32 +++++++++++++++++++ cdb.json | 0 src/build/flags.cppm | 2 +- src/build/ninja_backend.cppm | 16 ++++++++++ src/build/plan.cppm | 13 ++++++-- src/build/prepare/features.cpp | 12 +++---- src/build/prepare/graph.cpp | 18 ++++++++--- 7 files changed, 78 insertions(+), 15 deletions(-) create mode 100644 cdb.json diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md index d3047da1..3531dc7d 100644 --- a/.agents/docs/2026-09-29-workspace-build-graph-design.md +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -497,3 +497,35 @@ version. The other repositories change only where the release requires it. | T13 | the release canaries (xlings, mcppls), both rooted workspaces, and the plugins' CI | xlings, mcppls, mcpp-plugins | T11 | | T14 | sandbox verification with the CN mirror; issues commented and closed | all | T12, T13 | | T15 | acceptance on the validation project's pull request: its Windows CI against 1828 s, the core library compiled once, the Release layout | the validation project | T14 | + +## 17. Implementation record (2026-09-29) + +Implemented in one change on `feat/workspace-build-graph`, released as +2026.9.29.1. Where the implementation settled a question the sections above +left open, the answer is recorded here. + +| Question | Answer | Where | +|---|---|---| +| The root of a rooted workspace | Every workspace plan has a virtual root; the workspace's own package is the member `"."`, whose products keep `bin/`. A command at the root selects it alone. The package is then the same node in every selection. | `select_workspace_members` (prepare/manifest.cpp) | +| Which values separate plans | Every root-position value (`root_position_key`): the toolchain and target rows, the standard, the graph-wide `[build]` keys, the profiles, the indices, the capability and tool pins. The virtual root copies them from the group's first member, so a group and its root cannot disagree. | `mcpp.project` | +| A member's build program | Runs after every dependency's program, where a root's runs, with its own artifacts directory and a graph document of its closure in which it is `root`. A workspace member reached as another member's dependency runs there too, so its program sees one environment in every selection. | `step9_member_build_programs` (prepare/target_side.cpp) | +| A member's identity | A path member that declares no namespace keeps its bare name (a root's identity) instead of the default namespace a path dependency receives. | prepare/graph.cpp | +| Per-member link data | Each package's normalised link flags are read when the plan is made; a member's link group holds the plan's flags followed by its closure's, and its closure's runtime contract, and `compute_flags` renders it from a copy of the plan with the group swapped in. The link edge carries the group's `ldflags` and `c_ldflags`. | `make_plan` step 6, `swap_link_group`, ninja_backend.cppm | +| Graph-built shared libraries | Linked once at `bin/`, placed in each product directory whose units load them. | `LinkGroup::placements` | +| Linked placement | Shared libraries only; other deployed files are copied, since a program may write a file beside itself and a link would carry the write to its source. A linked destination is never written in place. | mcpp.build.stage | +| `-p X` and `--workspace` in one directory | `build.ninja` records a request tag (the plan's members and the requested features), and every fast path compares it; one `.build_cache` entry per selection and configuration group. Alternating selections re-plan (one plan of the selection) and recompile nothing. | graph_shape.cppm, execute.cppm | +| The lock | `/mcpp.lock`. A plan of all members writes the whole record; a plan of some keeps the other entries, and `--locked` then reports no entry of another member as drift. A selected member's git dependencies are locked as a root's. | prepare/records.cpp | +| Tests | `mcpp test` keeps one plan per member (`-p X` each), in the shared directory; the members' dev-dependencies are their own. | cmd_build.cppm | +| Concurrency | Groups are planned in turn and built on threads with a static share of the jobs; the `.build_cache` write is one locked step. | cmd_build.cppm | + +Readings with the implementation (Linux, llvm 22.1.8): + +| Reading | 2026.9.28.3 | 2026.9.29.1 | +|---|---|---| +| chain of five libraries and a program, `--workspace`, nothing built | 36.8 s | 0.51 s | +| the same, nothing changed | not replayed | 3 ms | +| `-p app`, nothing changed | not replayed | 3 ms | + +mcpp's own repository, a rooted workspace, builds itself with the new engine +(full build, gcc 16.1.0, 115 s), and its module tests run through member plans +(`mcpp test -p graph`, `-p versioning`). diff --git a/cdb.json b/cdb.json new file mode 100644 index 00000000..e69de29b diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 6f07d44d..43d50c55 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -1668,7 +1668,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { // same build re-runs edges for no reason. std::ranges::sort(sources); for (auto const& src : sources) - in.candidates.push_back({{src}, std::filesystem::path("bin") / src.filename(), + in.candidates.push_back({{src}, plan.productDir / src.filename(), rp::Kind::Toolchain}); } auto decision = rp::resolve(in); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index b7fb1aae..6db0456a 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -3923,6 +3923,22 @@ std::expected NinjaBackend::build(const BuildPlan& plan flags.graphLinkIsolated, plan.packageRoots); !h) { return std::unexpected(BuildError{h.error(), {}}); } + // A workspace member's units link with their group's line (§15), which + // is checked the same way. + if (!plan.linkGroups.empty()) { + BuildPlan view = plan; + std::set checked{flags.ld}; + for (auto& g : view.linkGroups) { + swap_link_group(view, g); + const auto gf = compute_flags(view); + swap_link_group(view, g); + if (!checked.insert(gf.ld).second) continue; + if (auto h = verify_hermetic_link(plan.toolchain, gf.ld, plan.outputDir, + plan.manifest.buildConfig.allowHostLibs, + gf.graphLinkIsolated, plan.packageRoots); !h) + return std::unexpected(BuildError{h.error(), {}}); + } + } stage("hermetic-check"); // When the toolchain comes from mcpp's private sandbox, use the diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 9d74c1f6..9322729e 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -493,6 +493,10 @@ struct BuildPlan { // musl-gcc 15.1 modules failed to emit vector>'s // move-ctor instantiation across the module boundary (release link error). std::vector runtimeProviders; + // Where the files the runtime needs are placed beside the programs, + // relative to outputDir: `bin`, or a workspace member's product directory + // while its link group is swapped in (§15). + std::filesystem::path productDir = "bin"; // A workspace plan's per-member link data (workspace design 2026-09-29 // §15). A selected member's link units link the member's closure -- the @@ -540,6 +544,7 @@ void swap_link_group(BuildPlan& plan, BuildPlan::LinkGroup& g) { std::swap(plan.runtimeDlopenLibs, g.runtimeDlopenLibs); std::swap(plan.runtimeCapabilities, g.runtimeCapabilities); std::swap(plan.runtimeProviders, g.runtimeProviders); + std::swap(plan.productDir, g.productDir); } // Merge the generic facts exported by the already-selected xlings @@ -2834,12 +2839,14 @@ make_plan(const mcpp::manifest::Manifest& manifest, for (auto i : closureIdx) closure.insert(qualified_package_name(packages[i].manifest)); // The member's link group: the closure's link flags after the - // plan's own, in the order a root build gives them (the member, then - // its dependencies in discovery order), and the closure's runtime. + // root's own (`packages[0]` is snapshotted before any dependency is + // loaded, so it holds the profile's flags and nothing pooled), in the + // order a root build gives them (the member, then its dependencies in + // discovery order), and the closure's runtime. BuildPlan::LinkGroup group; group.member = owner; group.productDir = productDir; - group.ldflags = manifest.buildConfig.ldflags; + group.ldflags = packages[0].linkUsage.ldflags; for (auto i : closureIdx) for (auto const& f : packages[i].linkUsage.ldflags) group.ldflags.push_back(f); diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 1599d467..fc70f7a2 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -323,9 +323,12 @@ static std::expected step6_activate_features(PrepareState& st // place costs nothing. What the gate exists for is the shape where // a target's mere presence changes how the package is linked into // every consumer — and that is exactly a `shared` or `lib` target. + // A selected workspace member builds every target, so every one + // of its targets is gated, as a root's are (§15). std::erase_if(pkg.manifest.targets, [&](const mcpp::manifest::Target& t) { - if (t.kind != mcpp::manifest::Target::Library + if (!pkg.memberProducts + && t.kind != mcpp::manifest::Target::Library && t.kind != mcpp::manifest::Target::SharedLibrary) return false; for (auto const& rf : t.requiredFeatures) @@ -2040,11 +2043,8 @@ static std::expected step6_dependency_build_programs(PrepareS // A CLAIM THAT ONLY EVER TIGHTENS. if (bcDep.runExclusive && !exclusiveBefore) state.m->buildConfig.runExclusive = true; - // A workspace plan reads each package's own flags when it links - // a member's closure; nothing is pooled in its root (§15). - if (!state.workspacePlan()) - state.m->buildConfig.ldflags.insert(state.m->buildConfig.ldflags.end(), - bcDep.ldflags.begin() + ldN, bcDep.ldflags.end()); + state.m->buildConfig.ldflags.insert(state.m->buildConfig.ldflags.end(), + bcDep.ldflags.begin() + ldN, bcDep.ldflags.end()); } // apply() may have added interface defines to packages' publicUsage diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 2b3814c3..d189d295 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -377,13 +377,13 @@ static std::vector propagateLinkFlags( // flags with the same reading its own flags receive, and an element // that packs several tokens is several words on both sides. // - // A workspace plan links each member's closure with that closure's own - // flags (workspace design 2026-09-29 §15), read from each package when - // the plan is made, so nothing is pooled in the virtual root. + // In a workspace plan the pooled list serves the units no member owns + // (a dependency's shared library); a member's units link with its own + // closure's flags, which the plan reads from each package (§15). std::vector added; for (auto const& word : mcpp::manifest::flag_words(depManifest.buildConfig.ldflags)) { auto normalized = mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word)); - if (!state.workspacePlan()) state.m->buildConfig.ldflags.push_back(normalized); + state.m->buildConfig.ldflags.push_back(normalized); added.push_back(std::move(normalized)); } return added; @@ -1637,7 +1637,15 @@ step4b_acquire_dependency_source(PrepareState& state, WorklistItemCtx& ctx) { const std::filesystem::path& wsRoot) { return inherit_as_workspace_member(*dep_manifest, ws, wsRoot, dep_root); }; - if (depIsMember) { + // A rooted workspace's own package, the member "." of a workspace + // plan (§15), is the workspace's manifest: it reads its own + // `[workspace.dependencies]`, as it did as the root. + const bool depIsWorkspacePackage = state.workspacePlan() && state.wsManifest + && dep_root.lexically_normal() == state.runtimeWorkspaceRoot.lexically_normal(); + if (depIsWorkspacePackage) { + mcpp::project::merge_workspace_deps(*dep_manifest, *state.wsManifest, + state.runtimeWorkspaceRoot); + } else if (depIsMember) { if (auto bad = inheritAsMember(*state.wsManifest, state.runtimeWorkspaceRoot)) return std::unexpected(*bad); } else if (!gitMember.empty()) { From 08a0d45bb1241de84a60ad8bc1fcaee4a55aa387 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 05:26:05 +0800 Subject: [PATCH 05/11] Workspace build graph: members' dependencies announced, the module-name limit documented --- .../2026-09-29-workspace-build-graph-design.md | 1 + docs/07-workspace.md | 3 +++ docs/zh/07-workspace.md | 2 ++ src/build/execute.cppm | 17 ++++++++++++++++- 4 files changed, 22 insertions(+), 1 deletion(-) diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md index 3531dc7d..04328a88 100644 --- a/.agents/docs/2026-09-29-workspace-build-graph-design.md +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -517,6 +517,7 @@ left open, the answer is recorded here. | The lock | `/mcpp.lock`. A plan of all members writes the whole record; a plan of some keeps the other entries, and `--locked` then reports no entry of another member as drift. A selected member's git dependencies are locked as a root's. | prepare/records.cpp | | Tests | `mcpp test` keeps one plan per member (`-p X` each), in the shared directory; the members' dev-dependencies are their own. | cmd_build.cppm | | Concurrency | Groups are planned in turn and built on threads with a static share of the jobs; the `.build_cache` write is one locked step. | cmd_build.cppm | +| Module names across members | A graph has one module namespace (BMIs are found by name), so two members that each provide a module of one name are refused in one `--workspace` plan, as two `artifacts` programs are (#732, which tracks per-provider BMI names). Measured over the package index's 172 members: no module name is provided twice. | scanner | Readings with the implementation (Linux, llvm 22.1.8): diff --git a/docs/07-workspace.md b/docs/07-workspace.md index 83cbf601..8c638370 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -438,6 +438,9 @@ member that several members use is compiled once. member order. - **No-op builds.** A command repeated with nothing changed is answered by one check per configuration, without planning. +- **Module names.** Members built in one graph share one module namespace: + two members that each provide a module of the same name cannot be built in + one `--workspace` command; build each with `-p`. ## 6. Directory Layout diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index 933b7021..7c0a54cc 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -405,6 +405,8 @@ mcpp test --workspace --workspace-timeout 1800 # whole fan-out (default 0 = no 命令被拒绝。 - **Hooks。** 每个被选成员的 `[hooks]` 按成员顺序在构建前后运行。 - **无事可做的构建。** 在没有任何改动时重复执行的命令，每个配置只做一次检查，不重新规划。 +- **模块名。** 在同一张图中构建的成员共享一个模块命名空间：两个成员各自提供同名模块时，不能在 + 同一条 `--workspace` 命令中构建；分别用 `-p` 构建。 ## 6. 目录布局 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 7fc35ea7..6923c2c2 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -941,7 +941,22 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, mcpp::ui::status("Compiling", std::format("{} v{} (.)", ctx.manifest.package.name, ctx.manifest.package.version)); - for (auto& [name, spec] : ctx.manifest.dependencies) { + // The packages announced: the root's dependencies, and in a workspace + // plan each member followed by the member's own dependencies (§15), as + // the member's own build announced them. + std::vector> announcedDeps; + for (auto const& [name, spec] : ctx.manifest.dependencies) { + announcedDeps.emplace_back(name, spec); + if (!spec.workspaceMember) continue; + for (auto const& m : ctx.workspaceMembers) { + std::error_code e1, e2; + if (std::filesystem::weakly_canonical(spec.path, e1) + != std::filesystem::weakly_canonical(m.root, e2)) + continue; + for (auto const& [dn, ds] : m.manifest.dependencies) announcedDeps.emplace_back(dn, ds); + } + } + for (auto& [name, spec] : announcedDeps) { if (announced.contains(name)) continue; announced.insert(name); // Two keys that resolved to one identity (#634, A2) are one package From ba4b0c6b406979bf48c5077673c55e83dd3b592f Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 05:32:42 +0800 Subject: [PATCH 06/11] Workspace build graph: the dependency flag normalisation moves to the shared helpers (file length), the design record carries its front matter, and PackageRoot holds no optional string (clang with the MSVC STL cannot copy one) --- ...2026-09-29-workspace-build-graph-design.md | 5 ++ .agents/docs/README.md | 4 +- src/build/plan.cppm | 6 +- src/build/prepare/config.cpp | 37 ++++++++++++ src/build/prepare/features.cpp | 2 +- src/build/prepare/graph.cpp | 57 ++++--------------- src/build/prepare/graph_load.cpp | 2 +- src/build/prepare/plan.cpp | 8 +-- src/build/prepare/records.cpp | 2 +- src/modgraph/scanner.cppm | 12 ++-- 10 files changed, 73 insertions(+), 62 deletions(-) diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md index 04328a88..1adb8d27 100644 --- a/.agents/docs/2026-09-29-workspace-build-graph-design.md +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -1,3 +1,8 @@ +--- +subject: design +status: active +--- + # The workspace as the unit of build: one graph per configuration, one scheduler, product directories, and a reusable graph module - Status: design, agreed; implementation in progress (revision 4) diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 6771e4e2..023f8cd4 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -316 records. +317 records. ## By subject @@ -30,6 +30,7 @@ Records that declare one. Everything else is listed by date below. ### design +- [The workspace as the unit of build: one graph per configuration, one scheduler, product directories, and a reusable graph module](2026-09-29-workspace-build-graph-design.md) — active - [An ecosystem design for mcpp and xlings: one authority per fact, and the work that follows from it](2026-09-28-ecosystem-design-and-optimisation-plan.md) — landed - [Build cost, foreign toolsets, the build-plugin architecture and the library surface: engine, plugin and index design (#734)](2026-09-28-build-cost-foreign-toolsets-and-library-surface-design.md) — active - [The compile database, `emit build-database`, and #701/#702: triage against the specifications, and one design](2026-09-26-compile-database-and-issue-699-design.md) — landed @@ -107,6 +108,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [The workspace as the unit of build: one graph per configuration, one scheduler, product directories, and a reusable graph module](2026-09-29-workspace-build-graph-design.md) — active - [Two days of mcpp and xlings: a review of what merged, what is known, and what is open](2026-09-28-ecosystem-review-of-two-days-of-mcpp-and-xlings.md) — active - [An ecosystem design for mcpp and xlings: one authority per fact, and the work that follows from it](2026-09-28-ecosystem-design-and-optimisation-plan.md) — landed - [Build cost, foreign toolsets, the build-plugin architecture and the library surface: engine, plugin and index design (#734)](2026-09-28-build-cost-foreign-toolsets-and-library-surface-design.md) — active diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 9322729e..39130c1f 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -2800,11 +2800,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, std::set memberDevDepPackages; for (std::size_t mi = 1; mi < packages.size(); ++mi) { auto const& pkg = packages[mi]; - if (!pkg.memberProducts) continue; + if (!pkg.selectedMember) continue; const auto owner = qualified_package_name(pkg.manifest); - const auto productDir = pkg.memberProducts->empty() + const auto productDir = pkg.memberProducts.empty() ? std::filesystem::path("bin") - : std::filesystem::path("bin") / *pkg.memberProducts; + : std::filesystem::path("bin") / pkg.memberProducts; auto place = [&](const std::filesystem::path& o) { return productDir / o.filename(); }; // The closure: the member, what it reaches through its dependency diff --git a/src/build/prepare/config.cpp b/src/build/prepare/config.cpp index 4fe182e2..7c406dd5 100644 --- a/src/build/prepare/config.cpp +++ b/src/build/prepare/config.cpp @@ -817,4 +817,41 @@ bool graph_or_targets_import_std(const mcpp::modgraph::Graph& graph, return false; } +static std::string normalizeDepLdflag(const std::filesystem::path& depRoot, + const std::string& flag) { + auto absolute_path = [&](std::string_view raw) { + std::filesystem::path p{std::string(raw)}; + // A loader token stays as written; see the predicate. + if (p.is_absolute() || mcpp::build::is_loader_relative_search_path(raw)) + return p; + return depRoot / p; + }; + + if (flag.starts_with("-L") && flag.size() > 2) { + return "-L" + absolute_path(std::string_view(flag).substr(2)).string(); + } + + constexpr std::string_view rpathPrefix = "-Wl,-rpath,"; + if (flag.starts_with(rpathPrefix) && flag.size() > rpathPrefix.size()) { + return std::string(rpathPrefix) + + absolute_path(std::string_view(flag).substr(rpathPrefix.size())).string(); + } + + return flag; +} + +// A dependency's link flags as its consumer's link reads them. Word by word +// (SPEC-004 §8, #703): a search path is made absolute per word, and each word +// is written back as an element that reads as exactly that word, so the +// consumer's renderer reads the dependency's flags with the same reading its +// own flags receive, and an element that packs several tokens is several +// words on both sides. +std::vector normalized_dependency_ldflags( + const std::filesystem::path& depRoot, const std::vector& ldflags) { + std::vector out; + for (auto const& word : mcpp::manifest::flag_words(ldflags)) + out.push_back(mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word))); + return out; +} + } // namespace mcpp::build diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index fc70f7a2..0a0dec29 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -327,7 +327,7 @@ static std::expected step6_activate_features(PrepareState& st // of its targets is gated, as a root's are (§15). std::erase_if(pkg.manifest.targets, [&](const mcpp::manifest::Target& t) { - if (!pkg.memberProducts + if (!pkg.selectedMember && t.kind != mcpp::manifest::Target::Library && t.kind != mcpp::manifest::Target::SharedLibrary) return false; diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index d189d295..9364e07c 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -335,57 +335,17 @@ static void recordDependencyEdge( }); } -static std::string normalizeDepLdflag(const std::filesystem::path& depRoot, - const std::string& flag) { - auto absolute_path = [&](std::string_view raw) { - std::filesystem::path p{std::string(raw)}; - // A loader token stays as written; see the predicate. - if (p.is_absolute() || mcpp::build::is_loader_relative_search_path(raw)) - return p; - return depRoot / p; - }; - - if (flag.starts_with("-L") && flag.size() > 2) { - return "-L" + absolute_path(std::string_view(flag).substr(2)).string(); - } - - constexpr std::string_view rpathPrefix = "-Wl,-rpath,"; - if (flag.starts_with(rpathPrefix) && flag.size() > rpathPrefix.size()) { - return std::string(rpathPrefix) - + absolute_path(std::string_view(flag).substr(rpathPrefix.size())).string(); - } - - return flag; -} - -std::vector normalized_dependency_ldflags( - const std::filesystem::path& depRoot, const std::vector& ldflags) { - std::vector out; - for (auto const& word : mcpp::manifest::flag_words(ldflags)) - out.push_back(mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word))); - return out; -} - static std::vector propagateLinkFlags( PrepareState& state, const std::filesystem::path& depRoot, const mcpp::manifest::Manifest& depManifest) { - // Word by word (SPEC-004 §8, #703): a search path is made absolute - // per word, and each word is written back as an element that reads as - // exactly that word, so the consumer's renderer reads the dependency's - // flags with the same reading its own flags receive, and an element - // that packs several tokens is several words on both sides. - // + // Word by word (SPEC-004 §8, #703): see normalized_dependency_ldflags. // In a workspace plan the pooled list serves the units no member owns // (a dependency's shared library); a member's units link with its own // closure's flags, which the plan reads from each package (§15). - std::vector added; - for (auto const& word : mcpp::manifest::flag_words(depManifest.buildConfig.ldflags)) { - auto normalized = mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word)); - state.m->buildConfig.ldflags.push_back(normalized); - added.push_back(std::move(normalized)); - } + auto added = normalized_dependency_ldflags(depRoot, depManifest.buildConfig.ldflags); + for (auto const& f : added) state.m->buildConfig.ldflags.push_back(f); return added; } @@ -1562,7 +1522,7 @@ step4b_acquire_dependency_source(PrepareState& state, WorklistItemCtx& ctx) { // as a root's are (workspace design 2026-09-29 §15). const bool consumerIsMember = item.consumerDepIndex != kMainConsumer && item.consumerDepIndex + 1 < state.packages.size() - && state.packages[item.consumerDepIndex + 1].memberProducts.has_value(); + && state.packages[item.consumerDepIndex + 1].selectedMember; if (item.consumerDepIndex == kMainConsumer || consumerIsMember) { // Only root deps are locked: the writer below walks the root // manifest's [dependencies], so a transitive git branch dep @@ -1835,7 +1795,10 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { const auto depPackageIndex = state.packages.size(); auto depPackage = makePackageRoot(state, ctx.dep_root, *state.dep_manifests.back()); if (!depPackage) return std::unexpected(depPackage.error()); - if (selectedMember) depPackage->memberProducts = state.selectedMembers.at(*selectedMember); + if (selectedMember) { + depPackage->selectedMember = true; + depPackage->memberProducts = state.selectedMembers.at(*selectedMember); + } state.packages.push_back(std::move(*depPackage)); recordDependencyEdge(state, item.consumerDepIndex, depPackageIndex, spec, item.buildOnly, name); @@ -2033,7 +1996,7 @@ static void step4b_define_lookup_closures(PrepareState& state) { // this plan ships. state.isWorkspaceMemberPackage = [&](std::size_t i) { if (i == 0 || i >= state.packages.size() || !state.workspacePlan()) return false; - if (state.packages[i].memberProducts) return true; + if (state.packages[i].selectedMember) return true; const auto& r = state.packages[i].root; if (r.lexically_normal() == state.runtimeWorkspaceRoot.lexically_normal()) return true; return !workspace_member_of(state, r).empty(); @@ -2042,7 +2005,7 @@ static void step4b_define_lookup_closures(PrepareState& state) { // (workspace design 2026-09-29 §15). state.compilesHere = [&](std::size_t i) { return i == 0 || !state.isProgramOnlyPackage(state.packages[i].manifest) - || state.isArtifactPackage(i) || state.packages[i].memberProducts.has_value(); + || state.isArtifactPackage(i) || state.packages[i].selectedMember; }; diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index d8a1d569..4e5104be 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -110,7 +110,7 @@ static void step4a_define_split_and_identity_closures(PrepareState& state) { ? state.activeFeaturesByPackage[i] : std::vector{}; for (auto const& spec : applicable_xlings_addresses( man, feats, state.toolPurpose, /*isRoot=*/i == 0 - || state.packages[i].memberProducts.has_value())) + || state.packages[i].selectedMember)) claims.push_back({spec, describe(i), i == 0 ? 0 : 1}); } auto unified = addrset::unify(claims); diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index e0f808b7..4c671d91 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -166,7 +166,7 @@ static std::expected step13_source_packages(PrepareState& sta // the fast path sweeps, the workspace's own package included. if (state.workspacePlan()) for (std::size_t i = 1; i < state.packages.size(); ++i) { - if (!state.packages[i].memberProducts) continue; + if (!state.packages[i].selectedMember) continue; auto normalized = state.packages[i].root.lexically_normal(); if (std::find(roots.begin(), roots.end(), normalized) == roots.end()) roots.push_back(normalized); @@ -181,7 +181,7 @@ static std::expected step13_source_packages(PrepareState& sta if (ec) dir = (state.runtimeWorkspaceRoot / mp).lexically_normal(); for (std::size_t i = 1; i < state.packages.size(); ++i) { auto const& pkg = state.packages[i]; - if (!pkg.memberProducts) continue; + if (!pkg.selectedMember) continue; std::error_code pec; auto root = std::filesystem::weakly_canonical(pkg.root, pec); if (pec) root = pkg.root.lexically_normal(); @@ -192,7 +192,7 @@ static std::expected step13_source_packages(PrepareState& sta : pkg.manifest.package.namespace_ + "." + pkg.manifest.package.name, .memberPath = mp, .root = pkg.root, - .productDir = *pkg.memberProducts, + .productDir = pkg.memberProducts, .manifest = pkg.manifest, }); break; @@ -272,7 +272,7 @@ static std::expected step13_runner_and_xlings(PrepareState& s ? state.activeFeaturesByPackage[i] : std::vector{}; for (auto const& spec : applicable_xlings_addresses( man, feats, ToolPurpose::Run, /*isRoot=*/i == 0 - || state.packages[i].memberProducts.has_value())) + || state.packages[i].selectedMember)) if (std::ranges::find(xlingsSpecs, spec) == xlingsSpecs.end()) { ctx.runTierPending = true; break; } } diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 68078605..4484d923 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -127,7 +127,7 @@ std::expected step13_lockfile(PrepareState& state, BuildConte for (auto const& [name, spec] : state.m->dependencies) gitDeps.emplace_back(name, spec); if (state.workspacePlan()) for (std::size_t i = 1; i < state.packages.size(); ++i) - if (state.packages[i].memberProducts) + if (state.packages[i].selectedMember) for (auto const& [name, spec] : state.packages[i].manifest.dependencies) gitDeps.emplace_back(name, spec); std::set gitLocked; diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 34c1633b..1bd173d5 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -148,10 +148,14 @@ struct PackageRoot { UsageRequirements publicUsage; UsageRequirements linkUsage; bool usageResolved = false; - // Set on a selected workspace member that the plan builds (workspace - // design 2026-09-29 §15): the directory below `bin/` its products are - // placed in. Absent on the plan's root and on every dependency. - std::optional memberProducts; + // A selected workspace member that the plan builds (workspace design + // 2026-09-29 §15), and the directory below `bin/` its products are placed + // in (empty: `bin/` itself). False on the plan's root and on every + // dependency. Two fields rather than an optional string: a + // `std::optional` member makes this type uncopyable under + // clang with the MSVC STL. + bool selectedMember = false; + std::string memberProducts; }; ScanResult scan_packages(const std::vector& packages); From de5328ddfdf9cfbfc02ccaaf7c162bdec3071a6e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 05:53:36 +0800 Subject: [PATCH 07/11] mcpp.graph keeps each site's order: the module graph's stack order (link order, and so Mach-O initializer order) and the host-module depth-first orders; groups sharing one build directory build one after the other; status lines written whole --- ...2026-09-29-workspace-build-graph-design.md | 1 + modules/graph/src/graph.cppm | 106 ++++++++++++++++++ modules/graph/tests/test_graph.cpp | 87 ++++++++++++++ src/build/prepare/features.cpp | 66 ++++------- src/build/stage.cppm | 4 +- src/cli/cmd_build.cppm | 19 +++- src/modgraph/graph.cppm | 11 +- src/project.cppm | 8 +- src/ui.cppm | 14 +++ 9 files changed, 261 insertions(+), 55 deletions(-) diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md index 1adb8d27..25408890 100644 --- a/.agents/docs/2026-09-29-workspace-build-graph-design.md +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -522,6 +522,7 @@ left open, the answer is recorded here. | The lock | `/mcpp.lock`. A plan of all members writes the whole record; a plan of some keeps the other entries, and `--locked` then reports no entry of another member as drift. A selected member's git dependencies are locked as a root's. | prepare/records.cpp | | Tests | `mcpp test` keeps one plan per member (`-p X` each), in the shared directory; the members' dev-dependencies are their own. | cmd_build.cppm | | Concurrency | Groups are planned in turn and built on threads with a static share of the jobs; the `.build_cache` write is one locked step. | cmd_build.cppm | +| The orders of `mcpp.graph` | Three: the stable order, the module graph's own (Kahn with the ready units on a stack, over the edges in their recorded order) and a depth-first post-order from roots in the given order. Each migrated site keeps the order it had: the module graph's unit order is the order of the objects on a link line, which Mach-O uses as initializer order, and the first migration to the stable order made openkal's `same-source` example crash at start on aarch64-macos from all three build hosts (CI, run 36486818199). With the orders restored, the link line of that example is byte-identical to 2026.9.28.3's. | modules/graph | | Module names across members | A graph has one module namespace (BMIs are found by name), so two members that each provide a module of one name are refused in one `--workspace` plan, as two `artifacts` programs are (#732, which tracks per-provider BMI names). Measured over the package index's 172 members: no module name is provided twice. | scanner | Readings with the implementation (Linux, llvm 22.1.8): diff --git a/modules/graph/src/graph.cppm b/modules/graph/src/graph.cppm index 0300b23e..82194f77 100644 --- a/modules/graph/src/graph.cppm +++ b/modules/graph/src/graph.cppm @@ -22,6 +22,23 @@ // a package's `[dependencies]`), so no call site has to invert its data to // call in. // +// THREE ORDERS, AND WHY A CALLER KEEPS THE ONE IT HAD +// +// An order among nodes that do not constrain each other is still observable +// when the order becomes a link order: Mach-O runs initializers in link order, +// and the module graph's unit order is the order of the objects on the link +// line. Changing it changed a program's initialization (measured 2026-09-29: +// openkal's `same-source` example, linked for aarch64-macos, crashed at start +// when its units moved to the stable order below). A caller whose order +// reaches an artifact therefore keeps its order, and this module offers each +// order the engine uses: +// +// topological_order Kahn, lowest ready index first (stable) +// stack_topological_order Kahn, most recently readied first, over edges in +// the order given (the module graph's unit order) +// depth_first_order dependencies first, depth first, from roots in +// the order given (the host-module orders) +// // STABILITY // // `topological_order` and `levels` are Kahn's algorithm with the ready set @@ -89,6 +106,27 @@ std::vector closure(const AdjacencyList& deps, std::span roots, bool include_roots = false); +// Kahn's algorithm with the ready nodes on a stack: the node readied most +// recently is emitted next. `edges` holds (dependent, dependency) pairs over +// nodes 0..n-1, in the order that decides the ties. This is the order of the +// module graph's units, and so of the objects on a link line (see the header). +std::expected, CycleError> +stack_topological_order(std::size_t n, + std::span> edges); + +// What a dependency on a node still on the depth-first path means. +enum class Cycles { + Error, // a cycle: the order is refused with the ring + Skip, // the edge is ignored; the node is emitted where it stands +}; + +// Dependencies first, depth first: from each root in the order given, a node +// is emitted after every node it depends on, its dependencies visited in the +// order of `deps[u]`. Only nodes reachable from `roots` are emitted. +std::expected, CycleError> +depth_first_order(const AdjacencyList& deps, std::span roots, + Cycles cycles = Cycles::Error); + } // namespace mcpp::graph namespace mcpp::graph { @@ -185,6 +223,74 @@ topological_order(const AdjacencyList& deps) { return kahn_order(deps); } +std::expected, CycleError> +stack_topological_order(std::size_t n, + std::span> edges) { + std::vector remaining(n, 0); + std::vector> dependents(n); + for (auto [dependent, dependency] : edges) { + remaining[dependent]++; + dependents[dependency].push_back(dependent); + } + std::vector order; + order.reserve(n); + std::vector ready; + for (std::size_t u = 0; u < n; ++u) + if (remaining[u] == 0) ready.push_back(u); + while (!ready.empty()) { + const std::size_t u = ready.back(); + ready.pop_back(); + order.push_back(u); + for (auto w : dependents[u]) + if (--remaining[w] == 0) ready.push_back(w); + } + if (order.size() == n) return order; + // The ring, found the way `topological_order` finds one. + AdjacencyList deps(n); + for (auto [dependent, dependency] : edges) deps[dependent].push_back(dependency); + auto ring = kahn_order(deps); + if (!ring) return std::unexpected(ring.error()); + return std::unexpected(CycleError{}); +} + +std::expected, CycleError> +depth_first_order(const AdjacencyList& deps, std::span roots, + Cycles cycles) { + const std::size_t n = deps.size(); + std::vector state(n, 0); // 0 unseen, 1 on the current path, 2 done + std::vector order; + std::vector path; + for (auto root : roots) { + if (root >= n || state[root] != 0) continue; + std::vector> frame{{root, 0}}; + state[root] = 1; + path.push_back(root); + while (!frame.empty()) { + auto& [u, k] = frame.back(); + if (k < deps[u].size()) { + const auto v = deps[u][k++]; + if (v >= n || state[v] == 2) continue; + if (state[v] == 1) { + if (cycles == Cycles::Skip) continue; + CycleError err; + err.cycle.assign(std::ranges::find(path, v), path.end()); + err.cycle.push_back(v); + return std::unexpected(std::move(err)); + } + state[v] = 1; + path.push_back(v); + frame.push_back({v, 0}); + continue; + } + state[u] = 2; + order.push_back(u); + path.pop_back(); + frame.pop_back(); + } + } + return order; +} + std::expected, CycleError> levels(const AdjacencyList& deps) { auto order = kahn_order(deps); diff --git a/modules/graph/tests/test_graph.cpp b/modules/graph/tests/test_graph.cpp index c7eb00b3..fabae372 100644 --- a/modules/graph/tests/test_graph.cpp +++ b/modules/graph/tests/test_graph.cpp @@ -243,3 +243,90 @@ TEST(Closure, RootWithNoEdgesHasAnEmptyClosure) { auto c = g::closure(deps, std::array{std::size_t{0}}); EXPECT_TRUE(c.empty()); } + +// ── stack_topological_order: the module graph's unit order ────────────────── +// +// The order the module graph has always had, which is the order of the +// objects on a link line. The reference below is the hand-written loop it +// replaced, run on the same edges. +namespace { +std::vector reference_stack_order( + std::size_t n, const std::vector>& edges) { + std::vector indeg(n, 0); + std::vector> adj(n); + for (auto [c, p] : edges) { indeg[c]++; adj[p].push_back(c); } + std::vector order, queue; + for (std::size_t i = 0; i < n; ++i) if (indeg[i] == 0) queue.push_back(i); + while (!queue.empty()) { + auto u = queue.back(); queue.pop_back(); + order.push_back(u); + for (auto v : adj[u]) if (--indeg[v] == 0) queue.push_back(v); + } + return order; +} +} // namespace + +TEST(StackTopologicalOrder, IsTheModuleGraphsHistoricalOrder) { + const std::vector> edges = { + {3, 0}, {4, 0}, {4, 1}, {5, 3}, {5, 4}, {6, 2}, {7, 6}, {7, 5}, {1, 2}, + }; + auto order = g::stack_topological_order(8, edges); + ASSERT_TRUE(order.has_value()); + EXPECT_EQ(*order, reference_stack_order(8, edges)); + // It is not the stable order: ties go to the most recently readied node. + g::AdjacencyList deps(8); + for (auto [c, p] : edges) deps[c].push_back(p); + EXPECT_NE(*order, *g::topological_order(deps)); +} + +TEST(StackTopologicalOrder, ACycleIsReportedAsItsPath) { + const std::vector> edges = {{0, 1}, {1, 2}, {2, 0}}; + auto order = g::stack_topological_order(3, edges); + ASSERT_FALSE(order.has_value()); + auto const& ring = order.error().cycle; + ASSERT_GE(ring.size(), 2u); + EXPECT_EQ(ring.front(), ring.back()); +} + +// ── depth_first_order: the host-module orders ─────────────────────────────── + +TEST(DepthFirstOrder, DependenciesFirstFromEachRootInOrder) { + // 0 -> {2, 1}, 1 -> {3}, 2 -> {3}: from root 0, dependency 2 is visited + // before 1 because that is the order `deps[0]` lists them in. + g::AdjacencyList deps(4); + deps[0] = {2, 1}; + deps[1] = {3}; + deps[2] = {3}; + auto order = g::depth_first_order(deps, std::array{std::size_t{0}}); + ASSERT_TRUE(order.has_value()); + EXPECT_EQ(*order, (std::vector{3, 2, 1, 0})); +} + +TEST(DepthFirstOrder, OnlyWhatTheRootsReachIsEmitted) { + g::AdjacencyList deps(4); + deps[1] = {0}; + deps[3] = {2}; + auto order = g::depth_first_order(deps, std::array{std::size_t{1}}); + ASSERT_TRUE(order.has_value()); + EXPECT_EQ(*order, (std::vector{0, 1})); +} + +TEST(DepthFirstOrder, ACycleIsAnErrorNamingTheRing) { + g::AdjacencyList deps(3); + deps[0] = {1}; + deps[1] = {2}; + deps[2] = {1}; + auto order = g::depth_first_order(deps, std::array{std::size_t{0}}); + ASSERT_FALSE(order.has_value()); + EXPECT_EQ(order.error().cycle, (std::vector{1, 2, 1})); +} + +TEST(DepthFirstOrder, ACycleCanBeSkipped) { + g::AdjacencyList deps(2); + deps[0] = {1}; + deps[1] = {0}; + auto order = g::depth_first_order(deps, std::array{std::size_t{0}, std::size_t{1}}, + g::Cycles::Skip); + ASSERT_TRUE(order.has_value()); + EXPECT_EQ(*order, (std::vector{1, 0})); +} diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 0a0dec29..d2b0f68b 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -1019,28 +1019,25 @@ static std::vector host_module_units(const PrepareState& st, s byName.emplace(pending[i].name, i); // deps[u] = the pending-list indices `u` imports BY NAME within this - // same package. mcpp.graph's stable tie-break (lowest index first among - // unconstrained nodes) is what preserves path order in the - // unconstrained case; the old hand-written DFS achieved the same thing - // by always trying the path-sorted list from the front. + // same package, in import order. A depth-first order from every unit in + // path order emits the first unit that can be emitted, which is what + // preserves path order in the unconstrained case, and is the order the + // build program's objects are linked in. mcpp::graph::AdjacencyList deps(pending.size()); for (std::size_t u = 0; u < pending.size(); ++u) for (auto const& want : pending[u].imports) if (auto it = byName.find(want); it != byName.end()) deps[u].push_back(it->second); + std::vector roots(pending.size()); + std::iota(roots.begin(), roots.end(), std::size_t{0}); // A CYCLE IS LEFT TO THE COMPILER, ON PURPOSE. It is ill-formed C++ and // the compiler says so with the two units named; refusing here would // report the same fact in a worse place, and getting the ordering wrong - // is no longer possible either way -- so on a cycle this falls back to - // declaration order instead of surfacing mcpp.graph's CycleError. + // is no longer possible either way -- so the edge that closes a cycle is + // skipped. std::vector order; - if (auto ordered = mcpp::graph::topological_order(deps)) { - order = std::move(*ordered); - } else { - order.resize(pending.size()); - std::iota(order.begin(), order.end(), std::size_t{0}); - } + order = *mcpp::graph::depth_first_order(deps, roots, mcpp::graph::Cycles::Skip); for (auto i : order) push(pending[i].path, std::move(pending[i].name)); return out; @@ -1163,39 +1160,24 @@ step6_host_module_registration(PrepareState& state) { } } - // Dependency-first order, so a rule's own host modules are - // compiled BEFORE it. That ordering is the entire mechanism: - // the compile loop in build_program.cppm accumulates the - // module flags as it goes, so each entry sees the BMIs of - // everything ahead of it, and "a rule may import another - // rule" needs no second machinery — only this sort. - // - // Restricted to `c`'s own reachable closure (not the whole - // project) and reindexed to a dense local graph before - // calling mcpp.graph: a cycle elsewhere among packages `c` - // never reaches must not fail `c`'s build. - auto reach = mcpp::graph::closure(pkgHostDeps, direct, - /*include_roots=*/true); - std::map local; - for (std::size_t i = 0; i < reach.size(); ++i) - local.emplace(reach[i], i); - mcpp::graph::AdjacencyList sub(reach.size()); - for (std::size_t i = 0; i < reach.size(); ++i) - for (auto q : pkgHostDeps[reach[i]]) - if (auto it = local.find(q); it != local.end()) - sub[i].push_back(it->second); - - auto topo = mcpp::graph::topological_order(sub); + // Post-order DFS, so a rule's own host modules are compiled + // BEFORE it. That ordering is the entire mechanism: the + // compile loop in build_program.cppm accumulates the module + // flags as it goes, so each entry sees the BMIs of everything + // ahead of it, and "a rule may import another rule" needs no + // second machinery — only this sort. It walks only what `c` + // reaches, so a cycle elsewhere never fails `c`'s build. + auto topo = mcpp::graph::depth_first_order(pkgHostDeps, direct); if (!topo) { // A cycle, reported AS a cycle and naming the packages // on it. A depth limit would answer a different // question and would answer it later. std::string ring; bool first = true; - for (auto li : topo.error().cycle) { + for (auto q : topo.error().cycle) { if (!first) ring += " -> "; first = false; - ring += identity(reach[li]); + ring += identity(q); } return std::unexpected(std::format( "build rules form an import cycle: {}\n" @@ -1205,8 +1187,7 @@ step6_host_module_registration(PrepareState& state) { } std::vector ordered; - for (auto li : *topo) { - auto p = reach[li]; + for (auto p : *topo) { for (auto& hm : units(p)) { hm.importable = isDirect.contains(p); ordered.push_back(std::move(hm)); @@ -1214,9 +1195,10 @@ step6_host_module_registration(PrepareState& state) { } // Every provider on this consumer's rule closure, transitive - // ones included -- `reach` is exactly that set, and a rule - // imported by another rule declares payloads just as directly. - state.hostModuleProvidersByConsumer[c].assign(reach.begin(), reach.end()); + // ones included, and a rule imported by another rule declares + // payloads just as directly. + std::set reached(topo->begin(), topo->end()); + state.hostModuleProvidersByConsumer[c].assign(reached.begin(), reached.end()); if (auto clash = prov::host_module_collision(ordered)) return std::unexpected(*clash); diff --git a/src/build/stage.cppm b/src/build/stage.cppm index fec199bc..aa3fd5e0 100644 --- a/src/build/stage.cppm +++ b/src/build/stage.cppm @@ -173,7 +173,9 @@ bool links_on_placement(const std::filesystem::path& p) { // it and the link can be made, and a copy otherwise (another volume, FAT or // exFAT, a permission), so the fallback is today's behaviour. // Returns an empty error_code on success; otherwise the most informative error -// (the in-place one — that's where 1224 / 32 shows up). +// (the in-place one — that's where 1224 / 32 shows up), or the rename's when +// the destination shares its bytes with another name and is not written in +// place. std::error_code write_once(const std::filesystem::path& src, const std::filesystem::path& dst) { auto tmp = temp_sibling(dst); diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 8934e452..76e794b5 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -328,19 +328,28 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { return r != 0 ? r : rc; } const std::size_t hw = std::max(1u, std::thread::hardware_concurrency()); + std::set directories; + for (auto const& c : contexts) directories.insert(c.outputDir.lexically_normal()); for (auto& c : contexts) { const std::size_t want = c.plan.scheduleNinjaJobs > 0 ? static_cast(c.plan.scheduleNinjaJobs) : hw + 2; c.plan.scheduleNinjaJobs = static_cast( - std::max(1, want / contexts.size())); + std::max(1, want / directories.size())); } + // Concurrency is across build directories. Two groups whose values + // resolve to one directory (one toolchain spelled two ways) are built + // one after the other in it, since one ninja owns a directory. + std::map> byDirectory; + for (std::size_t i = 0; i < contexts.size(); ++i) + byDirectory[contexts[i].outputDir.lexically_normal()].push_back(i); std::vector results(contexts.size(), 0); { std::vector builds; - for (std::size_t i = 0; i < contexts.size(); ++i) - builds.emplace_back([&, i] { - results[i] = run_build_with_hooks(contexts[i], verbose, no_cache, - ov.target_triple); + for (auto const& [dir, indices] : byDirectory) + builds.emplace_back([&, indices] { + for (auto i : indices) + results[i] = run_build_with_hooks(contexts[i], verbose, no_cache, + ov.target_triple); }); } for (int r : results) diff --git a/src/modgraph/graph.cppm b/src/modgraph/graph.cppm index 4944c863..ea3dab88 100644 --- a/src/modgraph/graph.cppm +++ b/src/modgraph/graph.cppm @@ -135,12 +135,11 @@ namespace mcpp::modgraph { std::expected, CycleError> topo_sort(const Graph& g) { // g.edges: (consumer, producer) pairs, "consumer depends on producer". - // mcpp.graph's adjacency direction is the same: deps[u] lists what u - // depends on, so an edge translates straight across with no inversion. - mcpp::graph::AdjacencyList deps(g.units.size()); - for (auto [c, p] : g.edges) deps[c].push_back(p); - - auto order = mcpp::graph::topological_order(deps); + // The unit order is the order of the objects on a link line, which + // Mach-O uses as the initializer order, so this is the order the module + // graph has always had: Kahn's algorithm with the ready units on a stack, + // over the edges in the order the graph records them (see mcpp.graph). + auto order = mcpp::graph::stack_topological_order(g.units.size(), g.edges); if (!order) return std::unexpected(CycleError{std::move(order.error().cycle)}); return std::move(*order); } diff --git a/src/project.cppm b/src/project.cppm index 75394c8e..374bab98 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -834,8 +834,14 @@ export std::string product_directory_name(const std::vector& al for (auto& c : s) if (c == '/' || c == '\\') c = '.'; return s; } + // Compared without case: a Windows or macOS file system gives two names + // that differ only in case one directory. + auto folded = [](std::string v) { + for (auto& c : v) c = static_cast(std::tolower(static_cast(c))); + return v; + }; const auto shared = std::ranges::count_if(all, [&](const WorkspaceMember& m) { - return m.name == it->name; + return folded(m.name) == folded(it->name); }); return shared > 1 ? qualified_member_name(*it) : it->name; } diff --git a/src/ui.cppm b/src/ui.cppm index c249f6a8..345aa67d 100644 --- a/src/ui.cppm +++ b/src/ui.cppm @@ -315,7 +315,16 @@ void set_line_buffered() { #endif } +// The configuration groups of one workspace command build on threads +// (workspace design 2026-09-29 §6), and each narrates its build: one line is +// written as a whole. +std::mutex& line_mutex() { + static std::mutex m; + return m; +} + void status(std::string_view verb, std::string_view message) { + std::lock_guard line(line_mutex()); if (g_quiet) return; init(); auto v = verb_padded(verb); @@ -329,6 +338,7 @@ void status(std::string_view verb, std::string_view message) { } void info(std::string_view verb, std::string_view message) { + std::lock_guard line(line_mutex()); if (g_quiet) return; init(); auto v = verb_padded(verb); @@ -343,6 +353,7 @@ void info(std::string_view verb, std::string_view message) { void finished(std::string_view profile, std::chrono::milliseconds elapsed, std::string_view descriptor) { + std::lock_guard line(line_mutex()); if (g_quiet) return; init(); auto v = verb_padded("Finished"); @@ -365,6 +376,7 @@ void finished(std::string_view profile, std::chrono::milliseconds elapsed, } void warning(std::string_view message) { + std::lock_guard line(line_mutex()); init(); if (g_color) { std::println(stderr, "{}{}warning:{} {}", kBold, kYellow, kReset, message); @@ -374,6 +386,7 @@ void warning(std::string_view message) { } void error(std::string_view message) { + std::lock_guard line(line_mutex()); init(); if (g_color) { std::println(stderr, "{}{}error:{} {}", kBold, kBrightRed, kReset, message); @@ -383,6 +396,7 @@ void error(std::string_view message) { } void note(std::string_view message) { + std::lock_guard line(line_mutex()); init(); if (g_color) { std::println(stderr, "{}{}note:{} {}", kBold, kCyan, kReset, message); From 80f5959069b1da1adb2ec7a50e728ddf61dd56f7 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 05:53:50 +0800 Subject: [PATCH 08/11] CHANGELOG: mcpp.graph changes no order --- CHANGELOG.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c842f9a1..359cb386 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -80,11 +80,12 @@ to 2026.9.29.1. - **The fast path records the request.** `build.ninja` states the workspace members and the features it was planned for, and a fast path replays it only for the same request; one record is kept per selection and configuration. -- **`mcpp.graph`.** One module holds the engine's graph algorithms: a stable - topological order with the cycle reported as its path, dependency levels and - the transitive closure. Module order, host-module order, unit order by - imports, the dependency closure, the package-cycle check and the build-key - fold use it; a cycle of modules is now reported as `a -> b -> a`. +- **`mcpp.graph`.** One module holds the engine's graph algorithms: the + topological orders the engine uses, dependency levels and the transitive + closure. Module order, host-module order, unit order by imports, the + dependency closure, the package-cycle check and the build-key fold use it, + each with the order it had, so no compile or link order changes; a cycle of + modules is now reported as `a -> b -> a`. ## [2026.9.28.3] - 2026-09-28 From dab99e272c16fe9eddaa044233af88c510798bac Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 06:13:52 +0800 Subject: [PATCH 09/11] A selected member's redundant CRT word is stated as a root's is --- src/build/prepare/scan.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index db397010..df30b308 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -291,7 +291,9 @@ step11_msvc_crt_word_check(PrepareState& state) { state.m->buildConfig.linkage, state.m->buildConfig.cxxRuntime); for (std::size_t i = 0; i < state.packages.size(); ++i) { auto const& pkg = state.packages[i]; - const bool isRoot = i == 0; + // A selected workspace member is the project being developed and + // is spoken to as a root (§15). + const bool isRoot = i == 0 || pkg.selectedMember; // A word `[workspace.build]` contributed is named at the table // that states it (WS3): the member's `[build]` does not contain // it, and a reader sent there finds nothing to remove. From d44d61c188b47620c341e549e9e97d837f93dc5c Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 06:41:16 +0800 Subject: [PATCH 10/11] The staged cache is placed by its own ninja pass (ninja 1.12.1 use-after-free, ninja-build/ninja#2662); pack reads runtime files against the product directory; a program shipped through artifacts is linked with its package's intermediates and placed in each member's product directory; a file never takes a product directory's path --- src/build/ninja_backend.cppm | 27 +++++++++++- src/build/plan.cppm | 42 +++++++++++++++++++ src/pack/pipeline.cppm | 12 ++++-- ...orkspace_is_one_graph_per_configuration.sh | 30 +++++++++++-- 4 files changed, 103 insertions(+), 8 deletions(-) diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 6db0456a..f682183a 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1141,6 +1141,11 @@ std::string filter_ninja_output(std::string_view output, return filtered; } +// The aggregate target for everything staged out of the global cache. Named +// with a leading underscore so it cannot collide with a module or target name +// (both of which are identifiers or paths). +constexpr std::string_view kStagedCacheGoal = "_mcpp_staged_cache"; + std::string emit_ninja_string(const BuildPlan& plan) { return emit_ninja_string(plan, nullptr); } @@ -2205,7 +2210,7 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements) { // Aggregate target for everything staged out of the global cache. Named // with a leading underscore so it cannot collide with a module or target // name (both of which are identifiers or paths). - constexpr std::string_view kStagedCachePhony = "_mcpp_staged_cache"; + constexpr std::string_view kStagedCachePhony = kStagedCacheGoal; auto bmi_path = [&traits](std::string_view name) { std::string s(traits.bmiDir); @@ -4051,6 +4056,26 @@ std::expected NinjaBackend::build(const BuildPlan& plan for (auto& ev : plan.toolchain.envOverrides) nenv.emplace_back(ev.key, ev.value); + // THE STAGED CACHE IS PLACED BY A PASS OF ITS OWN, before the graph that + // reads it. ninja 1.12.1 loads a dyndep file whose producer is up to date + // as soon as that producer's order-only inputs are ready, from inside the + // completion of the node that readied them; when the dyndep file names that + // same node as an input, loading it grows the list the completion is + // iterating, a use-after-free (ninja-build/ninja#2662, unfixed upstream). + // The scans wait on `_mcpp_staged_cache` and the importers' dyndep files + // name the staged BMIs, so re-staging a dependency in a directory whose + // scans are current -- a dependency upgraded with its build served from the + // cache -- crashed ninja (e2e 196). After this pass the staged nodes are + // current, and the second pass loads those dyndep files when it starts. + if (manifest.find("\nbuild " + std::string(kStagedCacheGoal) + " : phony") != std::string::npos) { + std::vector pre{ninjaProgram, "--quiet", "-C", plan.outputDir.string(), + std::string(kStagedCacheGoal)}; + (void)mcpp::platform::process::capture_exec_deadline(pre, nenv, + std::chrono::milliseconds(static_cast(opts.buildTimeoutSecs) * 1000), + nullptr); + stage("ninja-staged-cache"); + } + bool buildTimedOut = false; auto cap = mcpp::platform::process::capture_exec_deadline( nargv, nenv, diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 39130c1f..bf48b222 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -2701,6 +2701,12 @@ make_plan(const mcpp::manifest::Manifest& manifest, lu.dependencyOwned = true; lu.artifactOf = owner; lu.output = target_output(r.target, naming); + // In a workspace plan `bin/` holds products only: the workspace's own + // package's, and each member's directory. The program is linked with + // its package's intermediate files and placed beside each member + // program that ships it (step 6, §15 of the 2026-09-29 design). + if (manifest.package.virtualRoot) + lu.output = std::filesystem::path("obj") / sanitize(owner) / lu.output.filename(); lu.windowsSubsystem = r.target.windowsSubsystem; lu.windowsEntry = r.target.windowsEntry; lu.loaderTagFlag = loader_tag_flag(lu.kind); @@ -3005,10 +3011,46 @@ make_plan(const mcpp::manifest::Manifest& manifest, } units.push_back(std::move(lu)); } + // A program the closure ships through `artifacts` (mcpp#711) is linked + // once, at `bin/`, and placed beside the member's programs, where a + // program that launches it looks for it. + for (auto const& r : artifactRequests) { + const bool requested = std::ranges::any_of(artifactEdges, [&](auto const& e) { + return closureIdx.contains(e.first) && e.second == r.packageIndex; + }); + if (!requested) continue; + std::filesystem::path out; + for (auto const& u : plan.linkUnits) + if (u.artifactOf == qualified_package_name(packages[r.packageIndex].manifest) + && u.targetName == r.target.name) { out = u.output; break; } + if (out.empty()) continue; + const bool seen = std::ranges::any_of(group.placements, + [&](auto const& pl) { return pl.source == out; }); + if (!seen && out.parent_path() != productDir) + group.placements.push_back({out, place(out)}); + } plan.linkGroups.push_back(std::move(group)); for (auto& u : units) plan.linkUnits.push_back(std::move(u)); } + // A product directory is a directory: nothing this plan writes may take + // its path as a file (a program of the workspace's own package named as a + // member, say). + for (auto const& g : plan.linkGroups) { + if (g.productDir == std::filesystem::path("bin")) continue; + for (auto const& u : plan.linkUnits) { + bool clash = u.output == g.productDir; + for (auto const& a : u.runtimeAliases) clash = clash || a == g.productDir; + if (clash) + return std::unexpected(std::format( + "target '{}' would be written to '{}', which is the product " + "directory of workspace member '{}'.\n" + " Rename the target, or give the member a namespace so " + "that its directory is qualified.", + u.targetName, u.output.generic_string(), g.member)); + } + } + // The single derivation. Deliberately at the END of make_plan, after every // producer of a link unit has run: a dependency resolved to the shared // form arrives as an ordinary SharedLibrary unit, so this one predicate diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 10d129e1..6753d327 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -500,20 +500,26 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // package that the contract says the host serves. namespace rp = mcpp::build::runtime_placement; const bool msvcAbi = mcpp::toolchain::is_msvc_target(ctx->plan.toolchain); + // Relative to the program's directory: `bin`, or a workspace member's + // product directory (§15 of the 2026-09-29 workspace design). + const auto& productDir = ctx->plan.productDir; for (auto const& d : flags.runtimeDeploy) { - if (msvcAbi && d.dest.parent_path() == "bin" + if (msvcAbi && d.dest.parent_path() == productDir && rp::is_msvc_crt_name(d.dest.filename().string())) continue; - opts.runtimeFiles.push_back(d.dest.lexically_relative("bin")); + opts.runtimeFiles.push_back(d.dest.lexically_relative(productDir)); } if (msvcAbi && !opts.carryToolchainRuntime && flags.runtimeCrtPolicy != "static") for (auto n : rp::kMsvcCrtNames) opts.hostProvidedLibs.emplace_back(n); // A dependency's program the manifest ships with this one (mcpp#711, // `artifacts = [...]`) is linked into `bin/` beside the executable, so // it is staged the way a deployed file is. + // In a workspace member's product directory it is the placed copy. for (auto const& u : ctx->plan.linkUnits) if (!u.artifactOf.empty()) - opts.runtimeFiles.push_back(u.output.lexically_relative("bin")); + opts.runtimeFiles.push_back(productDir == "bin" + ? u.output.lexically_relative("bin") + : std::filesystem::path(u.output.filename())); // #634 A3: the Android row reads its closure against the directories // its link declared -- a prebuilt library named through `[runtime] // link_library_dirs` is a file the link used and the device does not diff --git a/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh b/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh index 74d68b48..caf33e10 100755 --- a/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh +++ b/tests/e2e/833_a_workspace_is_one_graph_per_configuration.sh @@ -16,7 +16,9 @@ # G6 two members with one package name in two namespaces get qualified # product directories; # G7 no member directory receives a build directory, and `clean --stale` -# removes the ones members held before, keeping `target/.build-mcpp/`. +# removes the ones members held before, keeping `target/.build-mcpp/`; +# G8 a program a member ships through `artifacts` is beside that member's +# program, in its product directory. set -e TMP=$(mktemp -d) @@ -29,10 +31,10 @@ case "$(uname -s)" in MINGW*|MSYS*|CYGWIN*) EXE=".exe" ;; esac cat > mcpp.toml <<'EOF' [workspace] -members = ["core", "cli", "gui", "ns1/tool", "ns2/tool", "modern"] +members = ["core", "cli", "gui", "ns1/tool", "ns2/tool", "modern", "helper"] EOF -mkdir -p core/src cli/src gui/src ns1/tool/src ns2/tool/src modern/src +mkdir -p core/src cli/src gui/src ns1/tool/src ns2/tool/src modern/src helper/src cat > core/mcpp.toml <<'EOF' [package] name = "core" @@ -59,6 +61,22 @@ EOF done printf '#include \nimport shared_core;\nint main() { std::printf("%%d\\n", core_v() + 1); return 0; }\n' > cli/src/main.cpp printf '#include \nimport shared_core;\nint main() { std::printf("%%d\\n", core_v() + 2); return 0; }\n' > gui/src/main.cpp +cat >> gui/mcpp.toml <<'EOF' + +[dependencies.helper] +path = "../helper" +artifacts = ["helper"] +EOF +cat > helper/mcpp.toml <<'EOF' +[package] +name = "helper" +version = "0.1.0" + +[targets.helper] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int main() { return 0; }\n' > helper/src/main.cpp for ns in ns1 ns2; do cat > $ns/tool/mcpp.toml < Date: Tue, 29 Sep 2026 06:41:35 +0800 Subject: [PATCH 11/11] Record the ninja 1.12.1 finding and the product-directory rule --- .agents/docs/2026-09-29-workspace-build-graph-design.md | 2 ++ CHANGELOG.md | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md index 25408890..f90e7e36 100644 --- a/.agents/docs/2026-09-29-workspace-build-graph-design.md +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -523,6 +523,8 @@ left open, the answer is recorded here. | Tests | `mcpp test` keeps one plan per member (`-p X` each), in the shared directory; the members' dev-dependencies are their own. | cmd_build.cppm | | Concurrency | Groups are planned in turn and built on threads with a static share of the jobs; the `.build_cache` write is one locked step. | cmd_build.cppm | | The orders of `mcpp.graph` | Three: the stable order, the module graph's own (Kahn with the ready units on a stack, over the edges in their recorded order) and a depth-first post-order from roots in the given order. Each migrated site keeps the order it had: the module graph's unit order is the order of the objects on a link line, which Mach-O uses as initializer order, and the first migration to the stable order made openkal's `same-source` example crash at start on aarch64-macos from all three build hosts (CI, run 36486818199). With the orders restored, the link line of that example is byte-identical to 2026.9.28.3's. | modules/graph | +| ninja 1.12.1 and a shared directory | A directory is now reused by graphs of other dependency sets, and re-staging a cached dependency while the scans are current crashed ninja 1.12.1: it loads the importer's dyndep file inside the completion of the staged BMI, and the load grows the list that completion iterates (ninja-build/ninja#2662, a use-after-free; the fix, ninja-build/ninja#2840, is unmerged). Measured on e2e 196 under gcc; ninja 1.13.2 happened not to crash on the same state. The staged cache is placed by a ninja pass of its own, so the graph that reads it starts with it current and loads those dyndep files when it starts. | ninja_backend.cppm | +| `bin/` in a workspace plan | Products only: the workspace's own package's and each member's directory. A program shipped through `artifacts` is linked among its package's intermediate files and placed in each member's directory whose closure asks for it (a member named like the program otherwise took the program's path as its directory, e2e 833 G8). A file that would take a product directory's path is refused. | plan.cppm | | Module names across members | A graph has one module namespace (BMIs are found by name), so two members that each provide a module of one name are refused in one `--workspace` plan, as two `artifacts` programs are (#732, which tracks per-provider BMI names). Measured over the package index's 172 members: no module name is provided twice. | scanner | Readings with the implementation (Linux, llvm 22.1.8): diff --git a/CHANGELOG.md b/CHANGELOG.md index 359cb386..28be9563 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,11 @@ to 2026.9.29.1. beside several programs is one file with several names where the file system supports links, and a copy elsewhere. Other deployed files are copied, since a program may write a file beside itself (e2e 835). +- **Files served from the global cache are staged by a ninja pass of their + own**, before the build that reads them. ninja 1.12.1 crashed when a cached + dependency was staged again in a directory whose scans were current + (ninja-build/ninja#2662); a directory named by its configuration makes that + the ordinary case of a dependency upgrade. - **The fast path records the request.** `build.ninja` states the workspace members and the features it was planned for, and a fast path replays it only for the same request; one record is kept per selection and configuration.