From 3d4c4321310496b9c91809f5b29a471a0dd77ca8 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Fri, 25 Sep 2026 05:20:16 +0800 Subject: [PATCH 01/15] docs: #690 design record and implementation plan --- ...026-09-25-issue-690-implementation-plan.md | 82 ++++++ ...workspace-build-inheritance-consistency.md | 276 ++++++++++++++++++ .agents/docs/README.md | 6 +- 3 files changed, 363 insertions(+), 1 deletion(-) create mode 100644 .agents/docs/2026-09-25-issue-690-implementation-plan.md create mode 100644 .agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md diff --git a/.agents/docs/2026-09-25-issue-690-implementation-plan.md b/.agents/docs/2026-09-25-issue-690-implementation-plan.md new file mode 100644 index 00000000..dcc446a2 --- /dev/null +++ b/.agents/docs/2026-09-25-issue-690-implementation-plan.md @@ -0,0 +1,82 @@ +--- +subject: plan +status: active +--- + +# #690: implementation plan + +- Design: [2026-09-25-issue-690-workspace-build-inheritance-consistency.md](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) +- Base: `origin/main` b4824697 (mcpp 2026.9.24.1). Release target: **mcpp 2026.9.25.1**. +- Delivery: one pull request in `mcpp-community/mcpp` that carries every workstream, followed by the release chain in the repositories listed in section 5. The design record's per-workstream pull requests (its section 7) are merged into one, as requested in review. Each workstream keeps its own criterion inside that pull request. + +--- + +## 1. Tasks + +| Id | Workstream | Content | Files (owner) | +|---|---|---|---| +| T1 | W1 | Build inheritance moves to the dependency load site. `makePackageRoot` stops inheriting. Snapshot post-condition. | `src/build/prepare.cppm` (lead) | +| T2 | W2 | One `[workspace.build]` key table. `ios_deployment_target` accepted. | `modules/manifest/src/toml.cppm` (lead) | +| T3 | W3 | A git-hosted member inherits its repository's `[workspace.build]`. | `src/build/prepare.cppm` (lead) | +| T4 | W4 | `load_effective_manifest`. `publish`, `pack`, `emit xpkg` and `toolchain list` read it. | `src/project.cppm`, `src/publish/pipeline.cppm`, `src/pack/route.cppm`, `src/toolchain/lifecycle.cppm` (agent B) | +| T5 | W5 | Normalised published manifest, sibling-edge rewrite, reproducible archive from git objects, descriptor `deps` from the normalised manifest. | `src/pm/publisher.cppm`, `modules/libs/src/toml.cppm` (serialiser), new `src/publish/normalize.cppm` (agent B) | +| T6 | W6 | Root include broadcast removed (C, C++, NASM). `kCacheEpoch` 3 to 4. Consumer-include advice on a dependency's missing header. Command-identity unit test. | `src/build/flags.cppm`, `src/build/cache_key.cppm`, `src/build/ninja_backend.cppm`, `src/build/execute.cppm` (agent A) | +| T7 | W7 | `[build] defines` is a keyed set. `!NAME` removes an inherited entry. | `src/build/prepare.cppm` (`fold_build_defines_into_flags`, lead) | +| T8 | harness | `_inherit_toolchain.sh` links payloads per version, so a version installed by a test lands in the test's home and not in the developer's registry (#293 recurrence, measured 2026-09-25). | `tests/e2e/_inherit_toolchain.sh` (lead) | +| T9 | docs | `docs/07`, `docs/04`, `docs/11` (English and 简体中文), SPEC-004 §8 amendment and §9, CHANGELOG, version 2026.9.25.1. | lead, after T1 to T8 | + +## 2. Dependencies + +``` +T2 ----------------------------+ +T1 --> T3 --> T7 --------------+--> T9 --> PR --> CI --> review --> merge --> release chain +T4 --> T5 ---------------------+ +T6 ----------------------------+ +T8 ----------------------------+ +``` + +- T3 edits the same load site as T1 and follows it. +- T7 edits the fold that T1's post-condition names, and follows T1. +- T5 needs T4's loader. +- T6 and T8 are independent of every other task. +- Agents A (T6) and B (T4, T5) work in their own worktrees from the plan commit. The lead integrates by cherry-pick. File ownership in section 1 is exclusive. A task that needs a line in another owner's file reports it instead of editing it. + +## 3. Review angles + +Each angle names the property it requires and the evidence that shows it. + +| Angle | Requirement | Evidence | +|---|---|---| +| Architecture | One pipeline per member. One effective-manifest loader. Private requirements stay inside their package. | Removal of the `makePackageRoot` inheritance block and of the `flags.cppm` broadcast. Every raw `manifest::load` of a project manifest is reviewed and recorded. | +| Stability | No silent state. Internal invariants fail loudly. | Snapshot post-condition. Cache epoch increment. Harness fix T8. | +| Simplicity | No new manifest keys. One new value form (`!NAME`). | Diff of the parser's key table. | +| User experience | Every refusal names the file, the key and the fix. The dependency-header advice names the consumer directory. | e2e output assertions. | +| Compatibility | Older engines read every published normalised manifest. No previously valid manifest is refused by the build. | e2e with the published 2026.9.24.1 binary as a consumer of a normalised archive. | +| Cross-platform | Windows, macOS and Linux CI. Path relativisation and archive creation use git and `std::filesystem`, not host tools with platform-specific flags. | CI matrix. | +| Consistency | `-p lib` and `-p app` compile `lib` identically. A git consumer compiles the member as its repository does. | e2e 321 counts. Git-member e2e. | +| Seamless upgrade | No user action is required. The cost is one cold dependency-cache rebuild (epoch 4) and one rebuild of fingerprints whose duplicated flags disappear. | CHANGELOG. | +| Test coverage | Every workstream has a criterion that fails with its fix removed. | Section 4. | + +## 4. Criteria per task + +| Task | Criterion | Kind | +|---|---|---| +| T1 | e2e 321: each workspace word exactly once, in both positions, in `.cpp` and `.c` entries, workspace before member. | e2e | +| T1 | Snapshot with unfolded `defines` yields the internal error. | unit | +| T2 | A workspace declaring every table row parses, and every value is inherited. | unit | +| T3 | `file://` repository member: the consumer build passes an `#error` guard, and the flag occurs once. | e2e | +| T4 | In a member that omits `version`, `emit xpkg` and `publish --dry-run` succeed. In a member without `[toolchain]`, `toolchain list` marks the workspace's toolchain. | e2e | +| T5 | The archive's `mcpp.toml` carries the inherited values and a version edge. `mcpp.toml.orig` is present. The descriptor lists the sibling. A consumer of the unpacked archive builds, under the new engine and under 2026.9.24.1. An edge without `version` is refused with the sibling's version in the message. Two runs produce byte-identical archives. The serialiser round-trips. | e2e, unit | +| T6 | A root private `limits.h` containing `#error` does not reach a path dependency or an index dependency. The dependency's compile command is identical under two roots that differ in include settings. The root's units carry each root directory once. The advice names the root directory. | e2e, unit | +| T7 | `defines = ["X=1"]` inherited and `defines = ["X=2"]` in the member emit only `-DX=2`. `!X` emits no `-DX`. | e2e, unit | +| T8 | After a test installs a version that the developer registry lacks, the developer registry still lacks it. | e2e harness check | + +## 5. Release chain + +1. PR in `mcpp-community/mcpp` with the version 2026.9.25.1 in `mcpp.toml` and `modules/versioning/src/version.cppm`. All required checks green. Squash merge. +2. `origin/main` HEAD run green. +3. `release.yml` dispatched. Each archive is uploaded to GitCode with the local tools the moment it appears (`tools/mirror_res.sh` from xlings, GitCode leg only). GET 200 and byte comparison on both mirrors. +4. `openxlings/xim-pkgindex` bump pull request merged. The published index artifact is read to confirm `latest`. +5. Bootstrap pin pull request in `mcpp-community/mcpp` (`.xlings.json`). The open #689 is resolved first. +6. `mcpp-community/mcpp-index`: CI run against the release. No package change is expected (the design's search found no package that relies on the root include broadcast). If a package fails, it is fixed in one pull request. +7. Sandbox verification: `xlings subos use --sandbox --cmd ...` with the CN mirror set for both xlings and mcpp inside the sandbox. It covers the #690 fixture, the git-member fixture, the F7 shadow fixture, a publish dry run of a workspace member, and a build of two representative index packages. diff --git a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md new file mode 100644 index 00000000..8f65ee57 --- /dev/null +++ b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md @@ -0,0 +1,276 @@ +--- +subject: design +status: active +--- + +# Workspace inheritance, flag scoping and the published form: a unified repair plan (#690) + +- Issue: mcpp-community/mcpp#690 (2026-09-24), "`[workspace.build] defines` are lost for sibling path dependencies" +- Basis: `origin/main` b4824697 (mcpp 2026.9.24.1). Every measurement was taken on Linux x86_64 with gcc 16.1.0, using a binary built from that commit. The F7 A/B runs used the same commit with one environment-gated change, described in section 3.7. +- Status: plan awaiting review; nothing is implemented. Decision D1 (section 4) was accepted in review on 2026-09-25. + +--- + +## 0. Summary + +1. **#690 is a defect in mcpp; the reported usage is correct.** `docs/07-workspace.md` §4.1, the design record of #527 (D12), and the CHANGELOG entry of #538/#539 all promise that `[workspace.build]`, including `defines`, reaches a member that is compiled as a sibling's `path` dependency. The defect does not depend on the platform. + +2. **The audit found nine findings in three families.** None of them is an isolated slip: + - *Inheritance placement* (F1, F2, F8). #539 applied `[workspace.build]` while `makePackageRoot` captures the manifest into the build graph, instead of in the normalisation pipeline that runs before that capture. A dependency member therefore inherits after its `defines` have been folded (F1), and the root inherits twice (F2). + - *The effective manifest exists only inside `prepare_build`* (F3, F4, F5, F6). Other readers see the raw file or a partial merge: the parser's key table, git-hosted members, `publish`/`pack`/`emit xpkg`, and `toolchain list`. The published form of a member is not self-contained. It loses `[workspace.build]`, keeps sibling `path` edges that consumers cannot resolve, and omits those edges from the descriptor. + - *Flag scoping* (F7, F9). The root's include directories, including `private_include_dirs`, are broadcast to every dependency's translation units. The dependency cache key does not contain them, so a root's private header can change a dependency object that is then served to unrelated projects. Measured: project B received a cJSON object compiled against project A's private `float.h`. Separately, the documented override rule for inherited vectors ("the member's later flag wins") holds only for flags the compiler resolves last-wins. + +3. **The plan has seven workstreams and three structural rules.** Every member passes through one pipeline. Every reader of a member manifest reads one effective manifest. Private build requirements never cross a package boundary. The workstreams are ordered so that each lands with its own criterion (section 7). + +--- + +## 1. Method + +- Each finding is either *measured* (a fixture and a command whose output is quoted) or *reasoned* (a code path is cited). The distinction is stated for every finding. +- Criteria assert on `compile_commands.json`, link lines, program output and file contents, never on build success alone (design record of #527, section 5.6). +- F7 was measured as an A/B with one binary: `src/build/flags.cppm` gated the root include broadcast behind `MCPP_F7_NO_BROADCAST`, so both modes ran the same build of the same commit. +- A pre-existing corruption in this machine's `~/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44.3` affects `tests/e2e/31_transitive_deps.sh` in both modes. Seven files in that directory, including the `libc.so` and `libm.so` linker scripts, reference a deleted temporary registry (`/tmp/tmp.Iq9cIYV03q/h2/...`) and were rewritten on 2026-09-24 04:28, before this work began. This is the #293 write-through shape. Test 31 therefore carries no signal about F7. + +--- + +## 2. Principles + +The decisions below are derived from these rules. Each rule names its source in mcpp and its counterpart in established build systems. + +| | Rule | In mcpp | Elsewhere | +|---|---|---|---| +| **P1** | **Position independence.** A package's compile inputs are a function of the package and of graph-wide settings. They do not depend on which consumer reached it, on whether it is the root, or on how it was fetched (path, git, index). | `docs/04`: "mcpp deliberately does not compile a shared source two different ways within one build"; #359: a package's own statements "mean the same thing whether the package is the root or someone's dependency". | Cargo resolves `field.workspace = true` at manifest load for every member, including a git dependency whose repository is a workspace. | +| **P2** | **Merge, then normalise, then snapshot.** Derived fields are computed once, after every merge. Consumers read only the normalised snapshot, and the snapshot checks the pipeline's post-condition. | #229 comment on `merge_conditional_config`: merges run "always immediately BEFORE" the capture; the comment on `fold_build_defines_into_flags` states the same order. | General practice for configuration pipelines. A merge after normalisation creates a second, partially normalised state. | +| **P3** | **One source of truth per rule.** One merge function, one key table, one effective-manifest loader. | D12: "a fifth key added to one of them is a defect that compiles". | | +| **P4** | **Scope.** A package's private build requirements reach only its own units. Usage requirements flow from a dependency towards its consumers, never from a consumer into a dependency. | #101 (usage-requirements architecture): the root's `include_dirs` is its private build requirement. `docs/04`: `private_include_dirs` "stop at this package's own boundary". `docs/30`: `include_dir` "colours only the declaring package's own translation units". | CMake `PRIVATE`/`PUBLIC`/`INTERFACE`. Bazel `copts` are not inherited by dependencies. | +| **P5** | **Cache soundness.** Every input that reaches a cached command is part of its key. | Memory of #344 and the dependency BMI poisoning: "the generated artefact is the criterion; the source is the hypothesis". | ccache and Bazel key on the full command and the content of its inputs. | +| **P6** | **The published form is self-contained and equals the development form.** | `docs/11`; memory "published form differs from development form". | `cargo package` writes a normalised `Cargo.toml` (inherited fields inlined, `path` removed from dependencies, original kept as `Cargo.toml.orig`). pnpm rewrites `workspace:` ranges at publish. | +| **P7** | **Internal invariants fail loudly in release builds.** | `plan.cppm:1797` `"internal error: ... (please report)"` returned through `std::expected`. There is no `assert` in `src/`. | | +| **P8** | **Behaviour changes carry a measured blast radius.** A change that can turn green builds red is measured against the ecosystem first, and is degraded first when the condition is not a proven failure. | D13 in the #527 design record; the ecosystem CI with `MCPP_SOURCE_REF`. | | + +--- + +## 3. Findings + +### 3.1 Every inheritable key in every position + +`[workspace.build]` accepts fifteen keys (`modules/manifest/src/toml.cppm:4002-4050`). Counts are occurrences in the member's own translation units. The fixture declares every vector key in the workspace, and a member-own `defines`, `cxxflags` and `cflags` as the control. + +| Key | Read from | Root (`-p lib`) | Sibling (`-p app`) | Finding | +|---|---|---|---|---| +| `defines` | `privateBuild` after the fold | 1 | **0** | **F1** | +| `cxxflags` | `privateBuild.cxxflags` | **2** | 1 | **F2** | +| `cflags` (`.c` unit) | `privateBuild.cflags` | **2** | 1 | **F2** | +| member's own `defines`/`cxxflags`/`cflags` (control) | same | 1 | 1 | correct | +| `ldflags` | root link line | **2** | n/a | **F2** | +| `include_dirs`, `include_dirs_after` | `privateBuild` plus the root broadcast | 2 | 2 | see F7 | +| `dialect_cxxflags`, `c_standard`, `linkage`, `target`, `cxx_runtime`, `dependency_linkage`, `macos_deployment_target` | root manifest only | graph-wide | n/a | consistent | +| `ios_deployment_target` | parsed and inherited | **refused** | n/a | **F3** | + +### 3.2 Inheritance placement: F1, F2, F8 + +**F1 (measured): `defines` are lost for a member reached as a `path` dependency.** On the dependency branch, `fold_build_defines_into_flags(dep_manifest->buildConfig)` (`src/build/prepare.cppm:8235`) folds and clears `defines`. `makePackageRoot` (`prepare.cppm:6709`) then calls `inherit_workspace_build`, which prepends the workspace `defines` to the folded manifest. It copies `cflags`/`cxxflags` into `privateBuild` without folding again. The failure is silent unless the source guards the macro. Workspace-wide defines are typically layout- or ABI-affecting (`_ITERATOR_DEBUG_LEVEL`, `_WIN32_WINNT`, `UNICODE`, `FMT_HEADER_ONLY`, `SPDLOG_ACTIVE_LEVEL`), so a member compiled without them is an ODR violation against members compiled with them. + +**F2 (measured): the selected root member inherits twice.** The root inherits at load (`prepare.cppm:2365`/`2384`). `packages[0] = makePackageRoot(*root, *m)` (`prepare.cppm:6774`) finds it is a member and inherits again. The measured compile line is `-DFLAGLEVEL=1 -fno-exceptions -DFLAGLEVEL=1 -fno-exceptions -DFLAGLEVEL=2 -fexceptions`, which is the workspace entries twice followed by the member's once. The second copy is prepended as well, so the outcome of any override is unchanged. The cost falls on flags that are not idempotent, such as `-include x.h` without a guard, or options that accumulate. + +**F8 (reasoned): the F1 dependency's cache key records a define its compile does not carry.** `cache_key.cppm:534` reads `pkg.manifest.buildConfig.defines`, which retains the unfolded entries. + +Root cause: the build half of inheritance runs inside the snapshot, after the fold. + +``` +root: load -> inherit(package+build) -> cfg merge -> report -> fold -> snapshot +path member: load -> inherit(package) -> cfg merge -> report -> fold -> snapshot+inherit(build) (F1) +root again: snapshot+inherit(build) (F2) +``` + +### 3.3 The key table: F3 + +**F3 (measured): `[workspace.build] ios_deployment_target` is refused although it is parsed and inherited.** #612 (c58d61e6) added the assignment (`toml.cppm:4025`) and the inheritance (`src/project.cppm:284`), but not the `kKnown` entry (`toml.cppm:4027`) or the error text. The key set is written four times: assignments, `kKnown`, the error message, and `docs/07`. + +### 3.4 Git-hosted members: F4 + +**F4 (measured): a member of a git-hosted workspace inherits `[workspace.package]` but not `[workspace.build]`.** The repository builds its member with `mcpp build -p lib`. A consumer of the same commit through `{ git = "file://...", branch = "main" }` fails with `#error WS_FLAG missing`. The package half was added in #650 (`prepare.cppm:8210`). The build half is excluded by a comment in `makePackageRoot` that considers only the consumer's workspace. + +### 3.5 Readers outside `prepare_build`: F5 (load half), F6 + +**F5a (measured): `mcpp publish`, `mcpp pack` and `mcpp emit xpkg` reject a member that omits `version`.** `docs/07` §4.1 permits the omission. `publish/pipeline.cppm:29,86` and `pack/route.cppm:64` call `manifest::load` without `insideWorkspace` and without inheritance. + +**F6 (measured): `mcpp toolchain list` inside a member ignores the workspace's `[toolchain]`.** It marks `gcc 16.1.0 (default)` while `mcpp build` in the same directory resolves the workspace's `llvm@22.1.8`. `effective_default_toolchain` (`toolchain/lifecycle.cppm:412`) reads `./mcpp.toml` raw. + +### 3.6 The published form: F5 (publish half) + +Measured on a git repository whose workspace declares `version`, `license` and `cxxflags = ["-DWS_FLAG=1"]`, with a member `lib` that depends on a sibling `util` through `path = "../util"`: + +| Observation | Evidence | +|---|---| +| **F5b.** The source archive contains only the member directory. | `git -C archive HEAD` (`src/pm/publisher.cppm:374`). The listing is `lib-0.3.0/lib.cpp` and `lib-0.3.0/mcpp.toml`. | +| **F5c.** The archived `mcpp.toml` is the raw member file. | It still reads `"probe.util" = { path = "../util" }`. It carries neither `[workspace.package]` nor `[workspace.build]`. | +| **F5d.** The descriptor points consumers at that raw file. | `manifest = "mcpp.toml"` (`publisher.cppm:329`). `loadVersionDep` reads it from the installed archive. | +| **F5e.** The descriptor drops sibling edges silently, even when they carry a version. | `deps = {}`. `publisher.cppm:297` skips every `isPath()` edge. An edge written `{ path = "../lib", version = "0.1.0" }` is accepted by the manifest and builds, but it is still omitted. | +| **F5f.** A consumer of the archive cannot build it. | Depending on the unpacked archive (standing in for an index consumer, which reads the same file) fails with `path dependency 'probe.util' ... has no mcpp.toml`. With the sibling edge removed by hand, it fails with `#error WS_FLAG missing: the published form lost [workspace.build]`. | + +`mcpp pack` builds through `prepare_build`, so its binary artefacts carry the effective configuration once F5a is fixed (reasoned from `pack/library_pipeline.cppm:303`, which reads the prepared context's manifest). + +### 3.7 Flag scoping: F7 + +**F7a (measured): the root's `include_dirs`, including `private_include_dirs`, reach every dependency's units.** `flags.cppm:612-624` places `plan.manifest.buildConfig.includeDirs` and `includeDirsAfter` in the file-level `$cxxflags`/`$cflags` of `build.ninja`. Every rule reads those variables (`cxx_object`, `c_object`, `cxx_module`, `cxx_scan`), in addition to each unit's `$local_includes`. This broadcast predates the usage-requirements model (v0.0.1). #101 introduced `privateBuild` for the root but did not remove it. The NASM channel (`flags.cppm:1128`) has the same shape. + +- Path dependency: a root with `include_dirs = ["appinc", "appprivinc"]` and `private_include_dirs = ["appprivinc"]` puts both directories on the dependency's `lib.cpp` and `c.c`. The root's own units carry each directory twice. +- Index dependency (`compat.cjson` 1.7.19): a root `appprivinc/limits.h` containing `#error` stops `cJSON.c` from compiling (`In file included from .../cJSON.c:44`), with `--cache off`. The comment in `mcpp-index/pkgs/c/compat.godot-cpp.lua:163`, "a consumer-side header shadow never reaches it", does not hold for an uncached compile. + +**F7b (measured): the dependency cache key omits the broadcast, so the shadow crosses projects.** Two projects that differ in root `include_dirs` resolve the same key (`build-cache/v1/pkg/compat/compat.cjson@1.7.19/7bc4625c67943f6a`). The procedure was as follows. The cache entry was moved aside. Project A, whose root has a private `float.h` that redefines `DBL_EPSILON` to `0.5`, was built. Then project B, which has no include directories, was built. B reported `Cached compat.cjson` and printed `cJSON_Compare(1.0, 1.2) = 1`. With the original entry restored, B prints `0`. The object a project receives therefore depends on which project populated the cache first. This is the most severe finding in this record, because it is silent, it crosses project boundaries, and it survives until the cache entry is evicted. + +**Blast radius of removing the broadcast (measured).** With `MCPP_F7_NO_BROADCAST=1`: +- The root's own units receive each root directory exactly once. Dependency units receive none. A dependency's public directories still reach its consumer (`lib/inc` on `main.cpp`). +- A root `build.mcpp` that calls `mcpp::include_dir("gen")` still reaches the root's units. +- The twenty e2e scripts that exercise `include_dirs` give identical results in both modes: every one passes, except `31_transitive_deps`, which fails in both modes for the reason given in section 1. No script relies on the broadcast. +- No descriptor in `mcpp-index` states that a package expects a header from its consumer's include path (searched for the usual phrasings). + +### 3.8 Override semantics: F9 + +**F9 (measured): the documented override rule for inherited vectors holds only for last-wins flags.** `docs/07` §4.1 says a member's flag "comes later on the command line, where it wins". The measured program output is `LEVEL=2 FLAGLEVEL=2 header=workspace exceptions=1`: +- `-f`/`-fno-`, `-W`/`-Wno-`, `-O` and a redefined `-D` are overridden as documented. +- **Include directories are first-wins.** The workspace's `wsinc/pick.h` shadows the member's `lib/inc/pick.h`. +- **A redefined macro is a diagnostic.** `-DLEVEL=1 -DLEVEL=2` produces `warning: 'LEVEL' redefined`. mcpp does not show it on a successful build, and it is an error under `-Werror`. +- **A workspace define cannot be removed.** The fold appends every `defines` entry after all `cxxflags`, so a member's `-ULEVEL` precedes the workspace's `-DLEVEL=1`. Measured: `LEVEL` stays defined. + +### 3.9 Checked and consistent + +- The fast path with an inherited `target` produces the correct `Target` for each change in the workspace and in the member (measured). +- Graph-wide keys are read from the root manifest only (`flags.cppm:1038`, `prepare_inputs.cppm:568`, `prepare.cppm:11911`). +- The multi-version and SemVer-merge `makePackageRoot` sites (`prepare.cppm:7786`, `7886`) receive index packages from `loadVersionDep`, which are never members. +- `mcpp doctor` reads only non-inheritable keys. +- Non-member `path` dependencies acquire no workspace flags (e2e 321, and observed here). +- The root's `cxxflags`/`cflags` are not broadcast: a dependency unit carries its own flags only (measured). Only include directories are. + +--- + +## 4. Decisions + +| | Question | Decision | Basis | +|---|---|---|---| +| **D1** | Should a git-hosted workspace's `[workspace.build]` reach its git-consumed members? | **Yes.** Accepted in review. | P1. The flags are the member's statement about its own compilation, factored into its repository root. A member's own `[build]` already applies through git, and #650 adopted the package half for the same reason. The behaviour change is recorded in the CHANGELOG. | +| **D2** | What is the published form of a member? | **A normalised, self-contained manifest** (section 5.5). Publishing is refused only where normalisation cannot preserve meaning. | P6. The development form is valid only inside its workspace. Refusing every member whose effective manifest differs from its file would refuse every member that inherits anything, which is the purpose of a workspace. | +| **Q3** | Is the snapshot post-condition an internal error in release builds? | **Yes**, following `plan.cppm:1797`: `std::unexpected("internal error: ... (please report)")`. | P7. The defect it catches is silent, and the check costs one comparison per package. | +| **D3** | How does a member override inherited vectors? | **(a)** document the actual rule now; **(b)** make `defines` a keyed set as a follow-up. Include ordering and opt-out wait for evidence. | Section 5.7. | +| **D4** | What replaces the root include broadcast? | **Nothing implicit.** The root's include directories become private to the root, as #101 intended. A dependency that needs a consumer-supplied header receives it through an explicit, keyed mechanism, designed only when a package needs it. | P4, P5, and the measured absence of reliance. | + +--- + +## 5. Design + +### 5.1 W1: one pipeline for every member (F1, F2, F8) + +- In the path/git dependency branch (`prepare.cppm:8199-8209`), call `inherit_workspace_build` directly after `inherit_workspace_package`, under the same membership condition. This places it before `merge_conditional_config`, `report_flag_words_changes` and the fold, exactly as on the root. +- Remove the inheritance block from `makePackageRoot`, and take the manifest by `const&` again. The root is inherited once, at load. +- At the snapshot, check the post-condition `manifest.buildConfig.defines.empty()`. On violation, return `internal error: [build].defines reached the build graph unfolded for '' (please report)`. +- Rewrite the #539 comment in `makePackageRoot` so that it records why inheritance is not performed there. + +Alternatives rejected: folding again inside `makePackageRoot` fixes F1 but leaves F2, and it leaves inherited flags unreported by `report_flag_words_changes`. A special case that skips the root fixes F2 only, by adding the positional branch that P1 excludes. + +### 5.2 W2: one key table (F3) + +Replace the assignments, `kKnown` and the error text in `toml.cppm` with one table of rows `{ key, assign }`. Both the known-key check and the message are derived from the table. Add the row `ios_deployment_target`. `docs/07` refers to the keys instead of listing them. + +### 5.3 W3: git-hosted members (F4, D1) + +At the same load site, in the `gitMember` branch that already applies `inherit_workspace_package`, also apply `inherit_workspace_build(*dep_manifest, *rm, gitMemberCloneRoot)`. The repository root is the anchor for relative include directories. `docs/07` §6 states that both halves apply. + +### 5.4 W4: one effective-manifest loader (F5a, F6) + +Add `mcpp::project::load_effective_manifest(dir)`, which performs these steps: +1. Load `dir/mcpp.toml`. +2. Find the workspace root and decide membership with `is_workspace_member`. +3. Reload with `insideWorkspace` when the package is a member, and apply `inherit_workspace_config`. +4. Run `workspace_inheritance_error`. + +The member branch of `prepare_build` (`prepare.cppm:2373-2387`), `publish/pipeline.cppm`, `pack/route.cppm` and `toolchain/lifecycle.cppm` call it. Every other raw `manifest::load` of a project manifest is reviewed against it. The review list is in `src/` (`cli/cmd_build.cppm:49`, `cli/cmd_sbom.cppm:94`, `pm/commands.cppm:572`, `pm/index_management.cppm:98,174`, `doctor.cppm:706`, `build/execute.cppm:1325`, `build/test_targets.cppm:38`). Sites that read only non-inheritable keys stay as they are, and each is recorded as such. + +### 5.5 W5: the published form (F5b to F5f, D2) + +`mcpp publish` and `emit xpkg` produce the archive from a normalised manifest: + +1. **Inline the effective configuration.** Workspace-inherited `[package]` fields and `[workspace.build]` entries are written into the archived `mcpp.toml`. Vectors keep their inherited order, and scalars appear only where the member did not declare them. `x.workspace = true` dependencies are written with their resolved specification. +2. **Rewrite sibling edges.** A `path` edge to a workspace member is published as a version edge, using the `version` written on the edge. If the edge has no `version`, publishing is refused with a message that names the sibling's effective version and the line to add. The version is not inferred silently, because a bare version pin in mcpp is exact, and the constraint is the author's decision. A `path` edge to a non-member is refused, because the consumer cannot resolve it. +3. **Refuse what normalisation cannot carry.** An inherited include directory that resolves outside the member directory cannot exist in the archive, so publishing is refused and the directory is named. +4. **Emit the descriptor's `deps` from the normalised manifest**, so sibling edges appear there instead of being skipped. +5. **Keep the original** as `mcpp.toml.orig` in the archive, for audit, as Cargo does. +6. **Make it reviewable.** `mcpp publish --dry-run` writes the normalised manifest to `target/dist/` and prints its path. + +This changes what consumers read only for members of workspaces. A package that is not a workspace member normalises to its own file. + +### 5.6 W6: flag scoping (F7a, F7b, D4) + +- Remove the root include broadcast from the C, C++ and NASM channels of `flags.cppm`. The root's units receive their directories through `privateBuild`, which is measured to be complete, including directories declared by `build.mcpp`. +- Make cache soundness a stated property: a dependency's compile command must be identical across consumers that differ only in root-private settings. A unit test compares the dependency's command for two roots that differ in `include_dirs`, `private_include_dirs` and `include_dirs_after`. It would have caught F7b. +- **Rollout (P8).** The twenty in-repository scripts show no reliance on the broadcast, but the ecosystem has not been measured. Before merge, run the mcpp-index CI against the pull-request branch (`MCPP_SOURCE_REF`). For one release, when a dependency's compile fails with a missing header that exists in a root include directory, append a note that names the directory and states that consumer include paths no longer reach dependencies. Cached objects produced under the broadcast are not invalidated by a key change, so the cache layout version is incremented with this change. +- An explicit consumer-to-dependency header channel (the "configuration header" pattern used by FreeRTOS, lwIP and mbed TLS) is out of scope. If a package needs it, it is designed per dependency edge, and it enters that dependency's cache key. + +### 5.7 W7: override semantics (F9, D3) + +- **(a) Now, in the W1 pull request:** `docs/07` states the rule as it is. Last-wins flags are overridden by restating them. Include directories are not overridable. A redefined define produces a compiler diagnostic, and a define cannot be removed. +- **(b) Follow-up:** `defines` becomes a keyed set. A member entry `NAME=value` replaces the workspace entry with the same `NAME`, so one `-DNAME=...` word is emitted and no redefinition diagnostic arises. An entry `!NAME` removes the inherited entry. This extends to vectors the rule scalars already follow: the member wins where it declared. It needs its own criterion and a SPEC-004 §8 amendment, because `defines` entries are currently values without identity. +- Member-first include ordering and an inheritance opt-out are not adopted. CMake orders directory-level before target-level include directories, as mcpp does. An opt-out weakens the drift protection that implicit inheritance exists to provide. Both are reconsidered only with a concrete case. + +### 5.8 The rule, stated normatively + +Proposed as SPEC-004 §9, "Workspace inheritance and build-requirement scope": + +1. A workspace member **must** receive `[workspace.package]` and `[workspace.build]` exactly once, whether it is the selected root, a `path` dependency of another member, or a member of a git-hosted workspace. +2. Vectors are ordered workspace, then member, then matching `[target..build]`. Scalars are taken from the workspace only when the member did not declare the key. +3. Inheritance **must** complete before `defines` is folded and before the manifest is captured into the build graph. +4. Every command that reads a member's manifest **must** read the effective manifest. +5. A package's private build requirements, including its include directories, **must not** reach another package's units. +6. The published manifest of a member **must** be self-contained. + +--- + +## 6. Criteria + +Each criterion fails when its fix is removed. + +1. **W1, both positions and every per-package key exactly once.** e2e 321 is extended with a workspace that declares `defines`, `cflags`, `cxxflags` and `ldflags`, and a member that declares its own. Under `-p lib` and `-p app`, each workspace word occurs exactly once in the member's `.cpp` and `.c` entries and precedes the member's word. A C and a C++ `#error` guard make the denominator cover both channels. The negative leg (a non-member path dependency receives nothing) is retained. +2. **W1, post-condition.** A unit test passes a manifest with unfolded `defines` to the snapshot and receives the internal error. +3. **W2.** A unit test parses a workspace declaring every row of the key table, including `ios_deployment_target`, and asserts each inherited value. +4. **W3.** A `file://` repository fixture: the consumer build succeeds past an `#error` guard, and the flag occurs once. +5. **W4.** In a member that omits `version`, `emit xpkg` and `publish --dry-run --allow-dirty` succeed. In a member without `[toolchain]`, `toolchain list` marks the workspace's toolchain. Each has a control at the workspace root. +6. **W5.** From the section 3.6 fixture: the archived `mcpp.toml` contains the inherited `version`, `license` and `cxxflags` and a version edge to `util`. The descriptor's `deps` lists `util`. A consumer of the unpacked archive builds past the `#error` guard. An edge without `version` is refused with the sibling's version in the message. `mcpp.toml.orig` is present. +7. **W6.** (i) A root with a private `limits.h` containing `#error` builds a dependency, with `--cache off`, as a path and as an index dependency. (ii) The unit test of section 5.6: a dependency's compile command is identical under two roots that differ in include settings. (iii) The root's units carry each root directory exactly once. (iv) The ecosystem CI is green on the branch. +8. **W7(a).** The documentation change carries no criterion of its own. W7(b) receives one when it is designed. + +--- + +## 7. Delivery + +| PR | Content | Depends on | +|---|---|---| +| A | W1, the post-condition, W7(a), criteria 1 and 2, SPEC-004 §9 draft, CHANGELOG. Closes #690. | none | +| B | W2, criterion 3. | none | +| C | W3, criterion 4, `docs/07` §6. | A | +| D | W4, criterion 5. | A | +| E | W5, criterion 6, `docs/11`. | D | +| F | W6, criterion 7, cache layout version, one-release note, ecosystem CI. | none; independent of A to E | +| G | W7(b), with its own design record. | A | + +F is separated from the inheritance work because its risk is different. It changes what dependencies compile against, and it needs the ecosystem measurement before merge. B is separate from A so that its requirement keeps its own criterion. + +--- + +## 8. Mitigation for current releases + +Until PR A is released, write the workspace macro as flags on both channels: + +```toml +[workspace.build] +cxxflags = ["-DWORKSPACE_DEFINE=1"] +cflags = ["-DWORKSPACE_DEFINE=1"] # required when any member has C sources +``` + +Measured on 2026.9.21.3: `cxxflags` alone reaches the sibling's C++ units but not its C units, and with both lines both positions build. Until PR F is released, a root should not place headers whose names collide with system or dependency headers in its include directories. If a dependency behaves differently after such a header was present, remove that dependency's entry under `~/.mcpp/build-cache/v1/pkg/`. + +--- + +## 9. Open items + +1. The ecosystem measurement for W6 (section 5.6) has not been run. It is a merge condition of PR F. +2. The `~/.mcpp` glibc 2.44.3 payload on the measuring machine is corrupted (section 1). It is unrelated to this plan, but it should be repaired before local e2e results from that machine are relied upon. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 200fa112..6f184353 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -306 records. +308 records. ## By subject @@ -30,6 +30,7 @@ Records that declare one. Everything else is listed by date below. ### design +- [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — active - [MSVC toolset 的选择、#685、#687 与工具链管理规范：总体设计](2026-09-24-toolchain-selection-and-payload-trust-design.md) — active - [openkal 生态：能力的时刻模型，以及 C 环境方案空间的划分](2026-09-20-openkal-c-environment-ecosystem-design.md) — active - [#674:`presents = "posix"` 在 Windows 上兑现契约的下半段](2026-09-19-issue-674-cenv-posix-preinclude-design.md) — superseded @@ -53,6 +54,7 @@ Records that declare one. Everything else is listed by date below. ### plan +- [#690: implementation plan](2026-09-25-issue-690-implementation-plan.md) — active - [工具链选择与载荷可信度：实施计划](2026-09-24-toolchain-selection-implementation-plan.md) — landed - [openkal 生态：完整性收尾与验收方案](2026-09-21-openkal-ecosystem-completion-and-acceptance.md) — active - [C 环境生态方案：执行计划](2026-09-20-ecosystem-execution-plan.md) — active @@ -97,6 +99,8 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — active +- [#690: implementation plan](2026-09-25-issue-690-implementation-plan.md) — active - [工具链选择与载荷可信度：实施计划](2026-09-24-toolchain-selection-implementation-plan.md) — landed - [MSVC toolset 的选择、#685、#687 与工具链管理规范：总体设计](2026-09-24-toolchain-selection-and-payload-trust-design.md) — active - [#685、#687 与 Windows clang 的 MSVC STL：三个问题的归属，以及工具链载荷的规范化](2026-09-24-685-687-msvc-stl-and-toolchain-payloads.md) — landed From f461c8f9a91af142bbf32ba82b049ad6b8972e81 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:33:41 +0800 Subject: [PATCH 02/15] fix(workspace): one inheritance pipeline for every member, keyed defines, one key table (#690) A member reached as a dependency now inherits [workspace.package], x.workspace = true entries and [workspace.build] at its load site, before the conditional merge and the defines fold, as the root does. The snapshot in makePackageRoot no longer inherits and refuses unfolded defines with an internal error. A member of a git-hosted workspace inherits from its own repository. [build] defines is a set keyed by macro name, and !NAME removes an inherited name. The inheritable [build] subset is one exported table, which accepts ios_deployment_target. The e2e harness links payloads per version, so a version installed by a test no longer lands in the developer's registry. --- modules/manifest/src/toml.cppm | 61 +++--- modules/manifest/src/types.cppm | 35 +++ src/build/prepare.cppm | 198 ++++++++++++----- .../e2e/770_workspace_member_as_dependency.sh | 203 ++++++++++++++++++ tests/e2e/_inherit_toolchain.sh | 24 ++- tests/unit/test_workspace_inheritance.cpp | 167 ++++++++++++++ 6 files changed, 598 insertions(+), 90 deletions(-) create mode 100644 tests/e2e/770_workspace_member_as_dependency.sh create mode 100644 tests/unit/test_workspace_inheritance.cpp diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 9d976d69..ce75d978 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -3998,41 +3998,34 @@ std::expected parse_string(std::string_view content, } } - // [workspace.build] — the inheritable subset of [build]. + // [workspace.build] — the inheritable subset of [build], read through + // the one table that states it (`kWorkspaceBuildKeys`). The known-key + // check and the list in the error message come from the same table, + // so a key cannot be parsed by one and refused by the other. if (auto* wbuild = doc->get_table("workspace.build")) { auto& b = m.workspace.inherited.build; m.workspace.inherited.buildPresent = true; - if (auto v = doc->get_string_array("workspace.build.cflags")) b.cflags = *v; - if (auto v = doc->get_string_array("workspace.build.cxxflags")) b.cxxflags = *v; - if (auto v = doc->get_string_array("workspace.build.ldflags")) b.ldflags = *v; - if (auto v = doc->get_string_array("workspace.build.defines")) b.defines = *v; - if (auto v = doc->get_string_array("workspace.build.dialect_cxxflags")) - b.dialectCxxflags = *v; - if (auto v = doc->get_string_array("workspace.build.include_dirs")) - for (auto& d : *v) b.includeDirs.emplace_back(d); - if (auto v = doc->get_string_array("workspace.build.include_dirs_after")) - for (auto& d : *v) b.includeDirsAfter.emplace_back(d); - if (auto v = doc->get_string_array("workspace.build.private_include_dirs")) - for (auto& d : *v) b.privateIncludeDirs.emplace_back(d); - if (auto v = doc->get_string("workspace.build.c_standard")) b.cStandard = *v; - if (auto v = doc->get_string("workspace.build.linkage")) b.linkage = *v; - if (auto v = doc->get_string("workspace.build.target")) b.target = *v; - if (auto v = doc->get_string("workspace.build.cxx_runtime")) b.cxxRuntime = *v; - if (auto v = doc->get_string("workspace.build.dependency_linkage")) - b.dependencyLinkage = *v; - if (auto v = doc->get_string("workspace.build.macos_deployment_target")) - b.macosDeploymentTarget = *v; - if (auto v = doc->get_string("workspace.build.ios_deployment_target")) - b.iosDeploymentTarget = *v; - static constexpr std::string_view kKnown[] = { - "cflags", "cxxflags", "ldflags", "defines", "dialect_cxxflags", - "include_dirs", "include_dirs_after", "private_include_dirs", - "c_standard", "linkage", "target", "cxx_runtime", - "dependency_linkage", "macos_deployment_target", + for (auto const& row : kWorkspaceBuildKeys) { + const auto path = std::format("workspace.build.{}", row.key); + if (auto const* strings = std::get_if< + std::vector BuildConfig::*>(&row.field)) { + if (auto v = doc->get_string_array(path)) b.**strings = *v; + } else if (auto const* paths = std::get_if< + std::vector BuildConfig::*>(&row.field)) { + if (auto v = doc->get_string_array(path)) + for (auto& d : *v) (b.**paths).emplace_back(d); + } else if (auto const* scalar = std::get_if< + std::string BuildConfig::*>(&row.field)) { + if (auto v = doc->get_string(path)) b.**scalar = *v; + } + } + auto known = [](std::string_view key) { + return std::ranges::any_of(kWorkspaceBuildKeys, + [&](const WorkspaceBuildKey& row) { return row.key == key; }); }; for (auto& [key, ignored] : *wbuild) { (void)ignored; - if (std::ranges::find(kKnown, key) != std::end(kKnown)) continue; + if (known(key)) continue; // `allow_host_libs` is named explicitly because refusing it is // a decision and not an omission: it turns a correctness gate // off, and a workspace root that could set it once would @@ -4046,13 +4039,13 @@ std::expected parse_string(std::string_view content, "It disables the hermetic-link check for a specific " "artifact, so it belongs in that package's own [build] " "table where the person turning it off owns the result.")); + std::string supported; + for (auto const& row : kWorkspaceBuildKeys) + supported += std::format("{}{}", supported.empty() ? "" : ", ", + row.key); return std::unexpected(error(origin, std::format( "[workspace.build] has no key '{}' (or it is not " - "inheritable). Supported: cflags, cxxflags, ldflags, " - "defines, dialect_cxxflags, include_dirs, " - "include_dirs_after, private_include_dirs, c_standard, " - "linkage, target, cxx_runtime, dependency_linkage, " - "macos_deployment_target.", key))); + "inheritable). Supported: {}.", key, supported))); } } diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 79767567..af5cf1de 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1599,6 +1599,41 @@ struct WorkspaceInherited { // rather than implemented here. }; +// THE INHERITABLE SUBSET OF `[build]`, STATED ONCE. +// +// The parser reads `[workspace.build]` by iterating this table, derives its +// known-key check and its error text from it, and `mcpp publish` writes the +// inherited values back under the same keys. The set used to be written four +// times (the assignments, the known-key list, the error message and the +// usage chapter), and a key added to one of them was refused by another: +// `ios_deployment_target` was parsed and inherited but not known (#690, F3). +// A row pairs the TOML key, which is the same under `[build]` and +// `[workspace.build]`, with the `BuildConfig` member it fills. +struct WorkspaceBuildKey { + std::string_view key; + std::variant BuildConfig::*, + std::vector BuildConfig::*, + std::string BuildConfig::*> field; +}; + +inline constexpr std::array kWorkspaceBuildKeys{{ + {"cflags", &BuildConfig::cflags}, + {"cxxflags", &BuildConfig::cxxflags}, + {"ldflags", &BuildConfig::ldflags}, + {"defines", &BuildConfig::defines}, + {"dialect_cxxflags", &BuildConfig::dialectCxxflags}, + {"include_dirs", &BuildConfig::includeDirs}, + {"include_dirs_after", &BuildConfig::includeDirsAfter}, + {"private_include_dirs", &BuildConfig::privateIncludeDirs}, + {"c_standard", &BuildConfig::cStandard}, + {"linkage", &BuildConfig::linkage}, + {"target", &BuildConfig::target}, + {"cxx_runtime", &BuildConfig::cxxRuntime}, + {"dependency_linkage", &BuildConfig::dependencyLinkage}, + {"macos_deployment_target", &BuildConfig::macosDeploymentTarget}, + {"ios_deployment_target", &BuildConfig::iosDeploymentTarget}, +}}; + struct WorkspaceConfig { std::vector members; // relative paths to member dirs std::vector exclude; // paths to exclude diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 8d1d8f06..b2e41ae8 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -638,24 +638,78 @@ void report_flag_words_changes(const mcpp::manifest::Manifest& m) { } } +// The macro name a `defines` entry or a `-D` word defines: the text before +// the first `=`, or the whole text when there is no value. +std::string_view define_name(std::string_view entry) { + return entry.substr(0, entry.find('=')); +} + // Desugar `[build].defines` into `-D` on both C and C++ flag channels. // -// ORDER (both halves are load-bearing): this must run AFTER -// merge_conditional_config — `defines` is a BuildInputs member, so a -// matching `[target.'cfg(...)'.build] defines` has been appended by then and -// folds in the same pass, landing after the unconditional entries so GNU -// last-wins gives the conditional rule precedence — and BEFORE the manifest is +// ORDER (both halves are load-bearing): this must run AFTER every merge that +// contributes `defines` (workspace inheritance, then the package's own table, +// then a matching `[target.'cfg(...)'.build]`), and BEFORE the manifest is // snapshotted into packages[] / fingerprinted, because that snapshot (not the // manifest) is what the P1689 scan, the compile edges and compute_fingerprint -// actually read. +// actually read. `makePackageRoot` refuses a manifest whose `defines` are +// still unfolded. +// +// `defines` IS A SET KEYED BY MACRO NAME (SPEC-004 §8). A later entry for a +// name replaces the earlier one in place, so a member that restates an +// inherited `NAME=value` produces one `-DNAME=value` word instead of two +// words and a redefinition diagnostic; an entry `!NAME` removes the name. +// A list is not enough for this, because the compiler resolves a repeated +// `-D` by warning (an error under `-Werror`) and a `-U` written in `cxxflags` +// precedes every folded `-D` and so cannot remove one. +// +// The key covers every `-D` word already in the flag lists as well: +// those written in `cflags`/`cxxflags` of the same tables, and those folded +// by an earlier call (the layer-conditional pass calls this again with only +// its own entries). A name this call defines or removes supersedes them, so +// the package's compile lines carry at most one definition per name. // // Idempotent: clearing the vector after folding makes repeated calls harmless. // Both `cflags` and `cxxflags` get the macro; assembly units pick it up for // free via the -D/-U/-I subset the ninja backend filters out of packageCflags. // A define is a value, so it enters the flag list as one word // (`flag_element`): `N="x"` reaches the compiler as `-DN="x"` on every host. -void fold_build_defines_into_flags(mcpp::manifest::BuildConfig& bc) { +export void fold_build_defines_into_flags(mcpp::manifest::BuildConfig& bc) { + if (bc.defines.empty()) return; + + std::vector resolved; // entries, first-seen order + std::vector named; // every name this call touches + auto touch = [&](std::string_view name) { + if (std::ranges::find(named, name) == named.end()) + named.emplace_back(name); + }; for (auto const& d : bc.defines) { + if (d.starts_with('!')) { + const auto name = std::string_view(d).substr(1); + std::erase_if(resolved, [&](const std::string& e) { + return define_name(e) == name; + }); + touch(name); + continue; + } + const auto name = define_name(d); + touch(name); + auto it = std::ranges::find_if(resolved, [&](const std::string& e) { + return define_name(e) == name; + }); + if (it != resolved.end()) *it = d; + else resolved.push_back(d); + } + + auto superseded = [&](const std::string& element) { + auto words = mcpp::manifest::flag_words(element); + if (words.size() != 1 || !words.front().starts_with("-D")) return false; + const auto name = define_name(std::string_view(words.front()).substr(2)); + return std::ranges::find(named, name) != named.end(); + }; + std::erase_if(bc.cflags, superseded); + std::erase_if(bc.cxxflags, superseded); + + for (auto const& d : resolved) { const auto element = mcpp::manifest::flag_element("-D" + d); bc.cflags.push_back(element); bc.cxxflags.push_back(element); @@ -663,6 +717,23 @@ void fold_build_defines_into_flags(mcpp::manifest::BuildConfig& bc) { bc.defines.clear(); } +// The post-condition of the normalisation pipeline, as the snapshot checks it: +// every `defines` entry has been folded into the flag lists. A non-empty list +// here means a merge ran after the fold, and the entries would otherwise be +// dropped without a diagnostic (#690). Returns the internal-error text, or +// nothing when the manifest may be captured. +export std::optional +unfolded_defines_error(const mcpp::manifest::Manifest& m) { + auto const& d = m.buildConfig.defines; + if (d.empty()) return std::nullopt; + return std::format( + "internal error: [build].defines of package '{}' reached the build " + "graph unfolded ({} entr{}, first '{}'); a merge ran after " + "fold_build_defines_into_flags (please report)", + m.package.name.empty() ? std::string("(root)") : m.package.name, + d.size(), d.size() == 1 ? "y" : "ies", d.front()); +} + // ── The SECOND conditional pass: predicates that name a target-side layer ──── // // #540/#494. `docs/14` documents a package adapting to the C library it was @@ -6678,37 +6749,29 @@ prepare_build(bool print_fingerprint, auto makePackageRoot = [&](const std::filesystem::path& packageRoot, - const mcpp::manifest::Manifest& manifestIn) + const mcpp::manifest::Manifest& manifest) + -> std::expected { - // `[workspace.build]` APPLIES TO EVERY MEMBER, INCLUDING ONE REACHED AS - // ANOTHER MEMBER'S `path` DEPENDENCY — WHICH IS THE ORDINARY SHAPE. - // - // Inheritance runs where the command's own manifest is loaded, so - // `mcpp build -p appb` gave `appb` the workspace flags and gave `liba` - // none, even though `liba` is a member of the same workspace and is - // being compiled by the same command. Measured before this: `-DWS_FLAG` - // on the consumer's TUs and not on the sibling's. - // - // `[workspace.package] standard` did not have the problem, because the - // standard is imposed graph-wide from the root for BMI-compatibility - // reasons — which is exactly why the gap was invisible until a - // `[build]` flag was inheritable too. - // - // Applied HERE because this is the one funnel both dependency-assembly - // sites go through, and because the include directories a few lines - // below are captured from the manifest at this moment: a later mutation - // would reach the flags and silently not the include dirs. - // - // Only for MEMBERS. An index or git dependency is not part of the - // workspace and must not acquire its flags. - mcpp::manifest::Manifest manifest = manifestIn; - if (wsManifest && !runtimeWorkspaceRoot.empty() - && mcpp::project::is_workspace_member(*wsManifest, - runtimeWorkspaceRoot, - packageRoot)) { - mcpp::project::inherit_workspace_build(manifest, *wsManifest, - runtimeWorkspaceRoot); - } + // THE SNAPSHOT READS A NORMALISED MANIFEST; IT DOES NOT NORMALISE ONE. + // + // Every merge that feeds a package's build inputs (workspace + // inheritance, the conditional `[target..build]` sections) runs + // at the package's LOAD site, and `fold_build_defines_into_flags` runs + // after all of them. This lambda only captures the result. + // + // `[workspace.build]` inheritance used to run here (#539). The root + // had already inherited at load time, so it received the workspace + // entries twice; a member reached as a sibling's `path` dependency + // inherited after its `defines` had been folded, so the workspace + // `defines` never reached its compile lines (#690). Both follow from + // performing a merge at the snapshot, and both are removed by + // performing it at the load site, where the root already did. + // + // The post-condition below is what keeps it removed: a merge placed + // after the fold leaves `defines` non-empty here, and the build stops + // with an internal error instead of dropping the macros in silence. + if (auto unfolded = unfolded_defines_error(manifest)) + return std::unexpected(*unfolded); mcpp::modgraph::PackageRoot pkg; pkg.root = packageRoot; @@ -6771,7 +6834,11 @@ prepare_build(bool print_fingerprint, return pkg; }; - packages[0] = makePackageRoot(*root, *m); + { + auto rootPackage = makePackageRoot(*root, *m); + if (!rootPackage) return std::unexpected(rootPackage.error()); + packages[0] = std::move(*rootPackage); + } auto recordDependencyEdge = [&](std::size_t consumerDepIndex, @@ -7783,7 +7850,9 @@ prepare_build(bool print_fingerprint, .sourceKind = "version", }); const auto depPackageIndex = packages.size(); - packages.push_back(makePackageRoot(secStage, *dep_manifests.back())); + auto secPackage = makePackageRoot(secStage, *dep_manifests.back()); + if (!secPackage) return std::unexpected(secPackage.error()); + packages.push_back(std::move(*secPackage)); recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec, item.buildOnly, name); auto linkFlagsAdded = propagateLinkFlags(secStage, *dep_manifests.back()); @@ -7882,8 +7951,10 @@ prepare_build(bool print_fingerprint, // in dep_manifests; packages = [main, dep_0, dep_1, …], so // packages[depIndex+1] is the same dep. *dep_manifests[it->second.depIndex] = std::move(newManifest); - packages[it->second.depIndex + 1] = + auto mergedPackage = makePackageRoot(newRoot, *dep_manifests[it->second.depIndex]); + if (!mergedPackage) return std::unexpected(mergedPackage.error()); + packages[it->second.depIndex + 1] = std::move(*mergedPackage); recordDependencyEdge(item.consumerDepIndex, it->second.depIndex + 1, spec, item.buildOnly, name); @@ -8197,24 +8268,39 @@ prepare_build(bool print_fingerprint, name, dep_root.string(), dm.error().format())); } dep_manifest = std::move(*dm); - // The metadata half of the inheritance. The `[build]` half runs in - // `makePackageRoot`, where the include directories are captured; - // splitting them is what keeps each one at the point its consumer - // reads it. + // A MEMBER REACHED AS A DEPENDENCY INHERITS HERE, AT ITS LOAD SITE, + // EXACTLY AS THE ROOT INHERITS AT ITS OWN. + // + // Three parts of what a member receives from its workspace matter + // to a dependency: `[workspace.package]` (a member may omit + // `version`), `x.workspace = true` dependency entries (without the + // merge the entry reaches resolution with no version and no path, + // and is reported as an unreadable index entry), and + // `[workspace.build]`. All three run before the conditional merge + // and the `defines` fold below, which is the order the root + // follows; the snapshot in `makePackageRoot` only captures the + // result (#690). The remaining parts of `inherit_workspace_config` + // (`[toolchain]`, `[target.]`, `[indices]`) are decided by + // the root for the whole graph and are not applied to a dependency. + // + // A member of a git-hosted workspace inherits from ITS repository, + // anchored at the clone, so that the same commit compiles the same + // way in its own checkout and in a consumer's graph. + auto inheritAsMember = [&](const mcpp::manifest::Manifest& ws, + const std::filesystem::path& wsRoot) + -> std::optional { + mcpp::project::inherit_workspace_package(*dep_manifest, ws); + mcpp::project::merge_workspace_deps(*dep_manifest, ws, wsRoot); + mcpp::project::inherit_workspace_build(*dep_manifest, ws, wsRoot); + return mcpp::project::workspace_inheritance_error( + *dep_manifest, dep_root); + }; if (depIsMember) { - mcpp::project::inherit_workspace_package( - *dep_manifest, *wsManifest); - if (auto bad = mcpp::project::workspace_inheritance_error( - *dep_manifest, dep_root)) + if (auto bad = inheritAsMember(*wsManifest, runtimeWorkspaceRoot)) return std::unexpected(*bad); } else if (!gitMember.empty()) { - // A repository member inherits its repository's - // `[workspace.package]`, as it does when the repository is - // built from its own checkout. if (auto rm = mcpp::manifest::load(gitMemberCloneRoot / "mcpp.toml")) { - mcpp::project::inherit_workspace_package(*dep_manifest, *rm); - if (auto bad = mcpp::project::workspace_inheritance_error( - *dep_manifest, dep_root)) + if (auto bad = inheritAsMember(*rm, gitMemberCloneRoot)) return std::unexpected(*bad); } } @@ -8358,7 +8444,9 @@ prepare_build(bool print_fingerprint, : std::string{}, }); const auto depPackageIndex = packages.size(); - packages.push_back(makePackageRoot(dep_root, *dep_manifests.back())); + auto depPackage = makePackageRoot(dep_root, *dep_manifests.back()); + if (!depPackage) return std::unexpected(depPackage.error()); + packages.push_back(std::move(*depPackage)); recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec, item.buildOnly, name); diff --git a/tests/e2e/770_workspace_member_as_dependency.sh b/tests/e2e/770_workspace_member_as_dependency.sh new file mode 100644 index 00000000..c1d58f61 --- /dev/null +++ b/tests/e2e/770_workspace_member_as_dependency.sh @@ -0,0 +1,203 @@ +#!/usr/bin/env bash +# requires: gcc python3 +# 770 -- a workspace member is compiled the same way whichever position it +# holds in the graph, and however it was fetched (#690). +# +# A. `[workspace.build]` reaches a member exactly once, as the selected root +# (`-p lib`) and as a sibling's `path` dependency (`-p app`): every +# workspace word occurs once in the member's C++ and C entries, and +# precedes the member's own word. Before #690 the sibling position lost +# `defines` (0 occurrences) and the root position carried `cflags` and +# `cxxflags` twice. +# B. A member reached as a dependency resolves its own +# `x.workspace = true` entries. Before #690 the entry reached resolution +# with neither version nor path. +# C. `defines` is a set keyed by macro name: a member that restates an +# inherited name produces one `-D` word with its own value, and `!NAME` +# removes an inherited name. +# D. A member of a git-hosted workspace, consumed through `git`, receives its +# repository's `[workspace.build]`, as its own checkout does. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +# count : occurrences of in that entry. +count() { + python3 - "$1" "$2" "$3" <<'EOF' +import json, sys +cdb, suffix, word = sys.argv[1], sys.argv[2], sys.argv[3] +for e in json.load(open(cdb)): + if e["file"].replace("\\", "/").endswith(suffix): + print(e["arguments"].count(word)) + sys.exit(0) +print("missing") +EOF +} +# before : the first precedes the first . +before() { + python3 - "$1" "$2" "$3" "$4" <<'EOF' +import json, sys +cdb, suffix, a, b = sys.argv[1:5] +for e in json.load(open(cdb)): + if e["file"].replace("\\", "/").endswith(suffix): + args = e["arguments"] + print("yes" if a in args and b in args and args.index(a) < args.index(b) else "no") + sys.exit(0) +print("missing") +EOF +} + +# ── A, B, C: one workspace, three members ─────────────────────────────────── +mkdir -p "$TMP/ws" && cd "$TMP/ws" +mkdir -p lib app/src util +cat > mcpp.toml <<'EOF' +[workspace] +members = ["lib", "app", "util"] + +[workspace.package] +version = "0.1.0" + +[workspace.dependencies] +util = { path = "util" } + +[workspace.build] +defines = ["WS_DEF=1", "LEVEL=1", "WS_ONLY=1"] +cflags = ["-DWS_CFLAG=1"] +cxxflags = ["-DWS_CXXFLAG=1"] +EOF +cat > util/mcpp.toml <<'EOF' +[package] +name = "util" + +[targets.util] +kind = "lib" + +[build] +sources = ["u.cpp"] +EOF +echo 'int util_v() { return 5; }' > util/u.cpp +cat > lib/mcpp.toml <<'EOF' +[package] +name = "lib" + +[dependencies] +util.workspace = true + +[targets.lib] +kind = "lib" + +[build] +sources = ["lib.cpp", "c.c"] +defines = ["MEMBER_DEF=1", "LEVEL=2", "!WS_ONLY"] +cflags = ["-DMEMBER_CFLAG=1"] +cxxflags = ["-DMEMBER_CXXFLAG=1"] +EOF +cat > lib/lib.cpp <<'EOF' +#if !defined(WS_DEF) || !defined(WS_CXXFLAG) || !defined(MEMBER_DEF) +#error "lib.cpp: a workspace or member definition is missing" +#endif +#if LEVEL != 2 +#error "lib.cpp: the member's LEVEL must replace the workspace's" +#endif +#ifdef WS_ONLY +#error "lib.cpp: !WS_ONLY must remove the inherited name" +#endif +int util_v(); +extern "C" int lib_c(); +int lib_v() { return util_v() + lib_c(); } +EOF +cat > lib/c.c <<'EOF' +#if !defined(WS_DEF) || !defined(WS_CFLAG) || !defined(MEMBER_DEF) +#error "c.c: a workspace or member definition is missing" +#endif +int lib_c(void) { return LEVEL; } +EOF +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" + +[dependencies] +lib = { path = "../lib" } + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int lib_v();\nint main() { return lib_v() == 7 ? 0 : 1; }\n' > app/src/main.cpp + +# Each position is asserted right after its own build, so the root position's +# counts are read even when the sibling position does not build. +for position in lib app; do + "$MCPP" build -p "$position" > "build-$position.log" 2>&1 \ + || fail "A/B: -p $position" "build-$position.log" + cdb="$position/compile_commands.json" + [ -f "$cdb" ] || fail "A: no $cdb" + for word in -DWS_DEF=1 -DWS_CXXFLAG=1 -DMEMBER_CXXFLAG=1 -DMEMBER_DEF=1 -DLEVEL=2; do + n=$(count "$cdb" lib/lib.cpp "$word") + [ "$n" = 1 ] || fail "A: $word occurs $n times in lib.cpp (-p $position)" + done + for word in -DWS_DEF=1 -DWS_CFLAG=1 -DMEMBER_CFLAG=1 -DMEMBER_DEF=1 -DLEVEL=2; do + n=$(count "$cdb" lib/c.c "$word") + [ "$n" = 1 ] || fail "A: $word occurs $n times in c.c (-p $position)" + done + for word in -DLEVEL=1 -DWS_ONLY=1; do + n=$(count "$cdb" lib/lib.cpp "$word") + [ "$n" = 0 ] || fail "C: $word occurs $n times in lib.cpp (-p $position)" + done + [ "$(before "$cdb" lib/lib.cpp -DWS_CXXFLAG=1 -DMEMBER_CXXFLAG=1)" = yes ] \ + || fail "A: the workspace cxxflags must precede the member's (-p $position)" + [ "$(before "$cdb" lib/c.c -DWS_CFLAG=1 -DMEMBER_CFLAG=1)" = yes ] \ + || fail "A: the workspace cflags must precede the member's (-p $position)" +done +"$MCPP" run -p app > run.log 2>&1 || fail "A/B: the program did not return 0" run.log +echo "ok: A, B, C -- one definition per name in both positions, sibling workspace dependency resolved" + +# ── D: a member of a git-hosted workspace ─────────────────────────────────── +mkdir -p "$TMP/repo/glib" && cd "$TMP/repo" +cat > mcpp.toml <<'EOF' +[workspace] +members = ["glib"] + +[workspace.package] +version = "0.2.0" + +[workspace.build] +cxxflags = ["-DREPO_FLAG=1"] +EOF +cat > glib/mcpp.toml <<'EOF' +[package] +namespace = "probe770" +name = "glib" + +[targets.glib] +kind = "lib" + +[build] +sources = ["g.cpp"] +EOF +printf '#ifndef REPO_FLAG\n#error "the repository [workspace.build] did not reach its git-consumed member"\n#endif\nint g_v() { return REPO_FLAG; }\n' > glib/g.cpp +git init -q -b main . && git add -A \ + && git -c user.email=e2e@mcpp -c user.name=e2e commit -qm init + +mkdir -p "$TMP/consumer/src" && cd "$TMP/consumer" +cat > mcpp.toml < src/main.cpp +"$MCPP" build > git.log 2>&1 || fail "D: consumer of a git-hosted member" git.log +n=$(count compile_commands.json glib/g.cpp -DREPO_FLAG=1) +[ "$n" = 1 ] || fail "D: -DREPO_FLAG=1 occurs $n times in the git member's g.cpp" +echo "ok: D -- the git-consumed member compiles as its repository does" + +echo "PASS: 770_workspace_member_as_dependency" diff --git a/tests/e2e/_inherit_toolchain.sh b/tests/e2e/_inherit_toolchain.sh index 8d7a4e3c..ba3eb9bd 100644 --- a/tests/e2e/_inherit_toolchain.sh +++ b/tests/e2e/_inherit_toolchain.sh @@ -19,16 +19,38 @@ if [[ ! -d "$USER_MCPP" && -n "${USERPROFILE:-}" ]]; then USER_MCPP="$USERPROFILE/.mcpp" fi +# Links the developer's payloads into the test's home ONE VERSION AT A TIME. +# +# The package directory (`xpkgs/xim-x-glibc`) is created as a real directory in +# the test's home and only its version directories are links. Linking the +# package directory itself made every version a test INSTALLED land in the +# developer's registry: the install wrote `xim-x-glibc/2.44.3` through the link, +# with linker scripts naming the test's temporary home, which was deleted when +# the test ended. Measured 2026-09-25: `31_transitive_deps.sh` recreated +# `~/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44.3/lib/libc.so` pointing at +# `/tmp/tmp.*/mcpp-home/...` (mcpp#293, second shape). A version that exists in +# the developer's registry is still shared by link, so an in-place rewrite of an +# existing payload remains the first shape of #293. link_xpkg_payloads() { local source_dir="$1" local target_dir="$MCPP_HOME/registry/data/xpkgs" [[ -d "$source_dir" ]] || return 0 mkdir -p "$target_dir" - local entry base + local entry base version vbase shopt -s nullglob for entry in "$source_dir"/*; do base="$(basename "$entry")" + if [[ -d "$entry" && ! -L "$target_dir/$base" ]]; then + mkdir -p "$target_dir/$base" + for version in "$entry"/*; do + vbase="$(basename "$version")" + [[ -e "$target_dir/$base/$vbase" ]] && continue + ln -sf "$version" "$target_dir/$base/$vbase" 2>/dev/null \ + || cp -r "$version" "$target_dir/$base/$vbase" + done + continue + fi [[ -e "$target_dir/$base" ]] && continue ln -sf "$entry" "$target_dir/$base" 2>/dev/null \ || cp -r "$entry" "$target_dir/$base" diff --git a/tests/unit/test_workspace_inheritance.cpp b/tests/unit/test_workspace_inheritance.cpp new file mode 100644 index 00000000..8c1e2b49 --- /dev/null +++ b/tests/unit/test_workspace_inheritance.cpp @@ -0,0 +1,167 @@ +#include + +import std; +import mcpp.manifest; +import mcpp.project; +import mcpp.build.prepare; + +// #690. The unit-level halves of the workspace-inheritance repair: the +// `[workspace.build]` key table (one statement of the inheritable subset), the +// keyed `defines` fold, and the snapshot's post-condition. The end-to-end +// halves (both graph positions, git-hosted members) are +// tests/e2e/321_workspace_inheritance.sh and +// tests/e2e/741_workspace_member_as_dependency.sh. + +namespace { + +// A value for every row of `kWorkspaceBuildKeys`. The table and this map must +// have the same keys: a row added without a value here fails +// `EveryTableRowIsParsedAndInherited` by name. +const std::map kSampleScalar = { + {"c_standard", "c17"}, + {"linkage", "static"}, + {"target", "x86_64-linux-musl"}, + {"cxx_runtime", "self-contained"}, + {"dependency_linkage", "static"}, + {"macos_deployment_target", "13.0"}, + {"ios_deployment_target", "15.0"}, +}; + +std::string workspace_declaring_every_key() { + std::string src = + "[workspace]\nmembers = [\"m\"]\n\n[workspace.package]\nversion = \"0.1.0\"\n\n" + "[workspace.build]\n"; + for (auto const& row : mcpp::manifest::kWorkspaceBuildKeys) { + if (std::holds_alternative(row.field)) { + auto it = kSampleScalar.find(std::string(row.key)); + src += std::format("{} = \"{}\"\n", row.key, + it == kSampleScalar.end() ? "missing-sample" : it->second); + } else { + src += std::format("{} = [\"ws_{}\"]\n", row.key, row.key); + } + } + return src; +} + +std::vector as_strings(const std::vector& v) { + std::vector out; + for (auto const& p : v) out.push_back(p.generic_string()); + return out; +} + +} // namespace + +// F3: a key the parser assigns and inherits must also be a key it accepts. +// `ios_deployment_target` was the counterexample. +TEST(WorkspaceInheritance, EveryTableRowIsParsedAndInherited) { + auto ws = mcpp::manifest::parse_string(workspace_declaring_every_key()); + ASSERT_TRUE(ws.has_value()) << ws.error().format(); + ASSERT_TRUE(ws->workspace.inherited.buildPresent); + + auto member = mcpp::manifest::parse_string( + "[package]\nname = \"m\"\n", "m/mcpp.toml", {.insideWorkspace = true}); + ASSERT_TRUE(member.has_value()) << member.error().format(); + const std::filesystem::path wsRoot = "/ws"; + mcpp::project::inherit_workspace_build(*member, *ws, wsRoot); + + std::size_t scalars = 0; + for (auto const& row : mcpp::manifest::kWorkspaceBuildKeys) { + SCOPED_TRACE(std::string(row.key)); + auto const& parsed = ws->workspace.inherited.build; + auto const& inherited = member->buildConfig; + if (auto const* f = std::get_if + mcpp::manifest::BuildConfig::*>(&row.field)) { + const std::vector want{std::format("ws_{}", row.key)}; + EXPECT_EQ(parsed.**f, want); + EXPECT_EQ(inherited.**f, want); + } else if (auto const* f = std::get_if + mcpp::manifest::BuildConfig::*>(&row.field)) { + EXPECT_EQ(as_strings(parsed.**f), + std::vector{std::format("ws_{}", row.key)}); + // Relative include directories are anchored at the workspace root. + EXPECT_EQ(as_strings(inherited.**f), + std::vector{ + (wsRoot / std::format("ws_{}", row.key)).generic_string()}); + } else if (auto const* f = std::get_if(&row.field)) { + ++scalars; + auto it = kSampleScalar.find(std::string(row.key)); + ASSERT_NE(it, kSampleScalar.end()) << "no sample value for a new scalar row"; + EXPECT_EQ(parsed.**f, it->second); + EXPECT_EQ(inherited.**f, it->second); + } + } + EXPECT_EQ(scalars, kSampleScalar.size()); +} + +TEST(WorkspaceInheritance, UnknownKeyErrorListsTheTable) { + auto ws = mcpp::manifest::parse_string( + "[workspace]\nmembers = [\"m\"]\n\n[workspace.build]\nno_such_key = []\n"); + ASSERT_FALSE(ws.has_value()); + const auto text = ws.error().format(); + for (auto const& row : mcpp::manifest::kWorkspaceBuildKeys) + EXPECT_NE(text.find(row.key), std::string::npos) << row.key; +} + +// W7: `defines` is a set keyed by macro name. +TEST(DefinesFold, LaterEntryReplacesEarlierInPlace) { + mcpp::manifest::BuildConfig bc; + bc.defines = {"X=1", "Y", "X=2"}; + mcpp::build::fold_build_defines_into_flags(bc); + EXPECT_EQ(bc.cxxflags, (std::vector{"-DX=2", "-DY"})); + EXPECT_EQ(bc.cflags, (std::vector{"-DX=2", "-DY"})); + EXPECT_TRUE(bc.defines.empty()); +} + +TEST(DefinesFold, BangNameRemovesTheName) { + mcpp::manifest::BuildConfig bc; + bc.defines = {"X=1", "!X", "Z"}; + mcpp::build::fold_build_defines_into_flags(bc); + EXPECT_EQ(bc.cxxflags, (std::vector{"-DZ"})); +} + +TEST(DefinesFold, EntrySupersedesADashDWordInTheFlagLists) { + mcpp::manifest::BuildConfig bc; + bc.cxxflags = {"-DX=1", "-O2", "-DXY=1"}; + bc.cflags = {"-DX"}; + bc.defines = {"X=3"}; + mcpp::build::fold_build_defines_into_flags(bc); + // `-DXY` is another name and stays; `-DX=1` and `-DX` are superseded. + EXPECT_EQ(bc.cxxflags, (std::vector{"-O2", "-DXY=1", "-DX=3"})); + EXPECT_EQ(bc.cflags, (std::vector{"-DX=3"})); +} + +// The layer-conditional pass folds a second time with only its own entries. +TEST(DefinesFold, SecondPassRemovesAWordTheFirstPassFolded) { + mcpp::manifest::BuildConfig bc; + bc.defines = {"X=1", "W"}; + mcpp::build::fold_build_defines_into_flags(bc); + bc.defines = {"!X"}; + mcpp::build::fold_build_defines_into_flags(bc); + EXPECT_EQ(bc.cxxflags, (std::vector{"-DW"})); +} + +TEST(DefinesFold, ValueWithSpaceStaysOneWordAndKeyed) { + mcpp::manifest::BuildConfig bc; + bc.defines = {"N=\"a b\""}; + mcpp::build::fold_build_defines_into_flags(bc); + ASSERT_EQ(bc.cxxflags.size(), 1u); + bc.defines = {"N=2"}; + mcpp::build::fold_build_defines_into_flags(bc); + EXPECT_EQ(bc.cxxflags, (std::vector{"-DN=2"})); +} + +// W1: the snapshot refuses a manifest whose `defines` were not folded. +TEST(SnapshotPostcondition, UnfoldedDefinesAreAnInternalError) { + mcpp::manifest::Manifest m; + m.package.name = "lib"; + EXPECT_FALSE(mcpp::build::unfolded_defines_error(m).has_value()); + m.buildConfig.defines = {"WORKSPACE_DEFINE=1"}; + auto err = mcpp::build::unfolded_defines_error(m); + ASSERT_TRUE(err.has_value()); + EXPECT_NE(err->find("internal error"), std::string::npos); + EXPECT_NE(err->find("'lib'"), std::string::npos); + EXPECT_NE(err->find("WORKSPACE_DEFINE=1"), std::string::npos); + mcpp::build::fold_build_defines_into_flags(m.buildConfig); + EXPECT_FALSE(mcpp::build::unfolded_defines_error(m).has_value()); +} From 567aa2a550123ab5f70a30ec857df039dbbd45a8 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:34:11 +0800 Subject: [PATCH 03/15] fix(build): a consumer's include directories stay in the consumer (#690 W6) The root's [build] include_dirs and include_dirs_after were written into the file-level $cxxflags/$cflags/$asmflags/$nasmflags of build.ninja, which every unit in the graph reads. A root header named like a system header shadowed it inside a dependency, private_include_dirs included, and because no cache-key axis contained the broadcast, a dependency object in the global cache could be compiled against another project's root headers (measured: cJSON_Compare answered 1 in an unrelated project). - flags.cppm: no include directory is broadcast; every unit keeps its own package's directories through $local_includes (C, C++, GAS and NASM). - cache_key.cppm: kCacheEpoch 3 -> 4, orphaning entries written while the broadcast existed. - ninja_backend.cppm / execute.cppm: consumer_include_scope_advice names the consumer directory that holds a header a dependency now fails to find, on the full path and, through a sidecar beside build.ninja, on the fast path. - tests: unit tests for the file-level channel, the unit channel's absolutisation, the advice and the sidecar; e2e 765. --- src/build/cache_key.cppm | 16 +- src/build/execute.cppm | 6 + src/build/flags.cppm | 112 +++------ src/build/ninja_backend.cppm | 144 +++++++++++ ...include_directory_stays_in_the_consumer.sh | 225 ++++++++++++++++++ tests/unit/test_ninja_backend.cpp | 168 +++++++------ 6 files changed, 527 insertions(+), 144 deletions(-) create mode 100755 tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh diff --git a/src/build/cache_key.cppm b/src/build/cache_key.cppm index bdaff087..e641b551 100644 --- a/src/build/cache_key.cppm +++ b/src/build/cache_key.cppm @@ -92,7 +92,21 @@ export namespace mcpp::build::cache_key { // compiled WITH the substitution. Bumping this orphans the entire cache — // one cold rebuild — rather than trust a key equality that cannot tell a // pre-fix entry from a post-fix one. -inline constexpr int kCacheEpoch = 3; +// 4 (#690, design record F7): the root's `[build] include_dirs` and +// `include_dirs_after`, `private_include_dirs` included, used to be written +// into the file-level compile flags of build.ninja, which every dependency +// unit reads, while no axis of this key contained them. A cached dependency +// object could therefore have been compiled against ANOTHER project's root +// headers, and the key could not say so. Measured: with the entry for +// `compat.cjson` 1.7.19 moved aside, a project whose root carried a private +// `float.h` redefining `DBL_EPSILON` populated the entry, and an unrelated +// project with no include directories then received that object and printed +// `cJSON_Compare(1.0, 1.2) = 1`. The broadcast is gone (flags.cppm), so a +// dependency's compile line no longer depends on its consumer; the entries +// written while it existed are the epoch-3 shape again: entries whose key +// never described them. Bumping orphans the cache once, one cold rebuild, +// rather than trust any entry written before the fix. +inline constexpr int kCacheEpoch = 4; // Axes A/B/C — identical for every package in one build, computed once. struct BuildAxes { diff --git a/src/build/execute.cppm b/src/build/execute.cppm index e03bf692..e12b1ade 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1258,6 +1258,12 @@ std::optional run_ninja_fast(const std::string& ninjaProgram, if (auto advice = mcpp::build::graph_c_library_isolation_advice(out); !advice.empty()) std::fputs(advice.c_str(), stderr); + // #690: the consumer-include note, from the list the plan wrote beside + // build.ninja (`write_consumer_include_sidecar`). + if (auto advice = mcpp::build::consumer_include_scope_advice( + out, mcpp::build::read_consumer_include_sidecar(ninjaPath.parent_path())); + !advice.empty()) + std::fputs(advice.c_str(), stderr); // THE SAME ADVICE THE PLAN PATH GIVES, FROM THE LIST THE PLAN WROTE // DOWN. This path has no `BuildPlan` by construction, so the C // library's `[c-abi-absent]` table reaches it through a file beside diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 4afe81d7..17854cbe 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -580,53 +580,31 @@ CompileFlags compute_flags(const BuildPlan& plan) { std::string pic_flag = (plan.needsPic && !isMsvcDialect && !peTarget) ? " -fPIC" : ""; - // Include dirs — this is the TYPED PATH channel (bare paths from the - // manifest; the dialect prefix is applied here at emission), not the - // FLAG-STRING channel that `normalize_include_flags` serves (cflags/ - // cxxflags, where the -I/-iquote/... prefix is already embedded in the - // string by the scanner). `normalize_include_flags`'s prefix table only - // knows GNU spellings, so routing dialect-prefixed tokens through it - // silently no-ops under MSVC (`/Iinclude` matches nothing and is never - // rewritten against plan.projectRoot — but ninja runs with cwd = output - // dir, so a relative include dir stops resolving). Absolutize the path - // directly instead (dialect-agnostic), then prepend the prefix, then - // ninja-$-escape and shell-quote per token (#234) so an include dir - // whose name contains a space can't silently split into two shell words - // once ninja hands the resolved command line to the shell. - // The one place this file turns a manifest include entry into a path. - // make_preferred: a multi-segment TOML entry like `generated/inc` keeps - // its `/` on MSVC, and the bare `projectRoot / inc` join would be MIXED — - // reaching both the ninja command line and the CDB's arguments (via - // f.cxx → split_flags). Same rule as every other manifest-path ingestion - // point (#390); no-op on POSIX. ONE lambda because the same join is needed - // four times in this function — {include_dirs, include_dirs_after} × {the - // C/C++ token list, the NASM one} — and re-deriving it per site is how the - // two channels drifted apart in the first place. - auto abs_native = [&](const std::filesystem::path& inc) { - auto p = inc.has_root_path() ? inc : (plan.projectRoot / inc); - p.make_preferred(); - return p; - }; - - std::vector includeTokens; - for (auto& inc : plan.manifest.buildConfig.includeDirs) { - includeTokens.push_back(include_token(d, abs_native(inc))); - } - // #249: `[build] include_dirs_after` — searched AFTER the toolchain's - // system dirs via -idirafter (gcc+clang), so entries can't shadow - // standard headers. cl.exe has no -idirafter; under the msvc dialect - // they degrade to regular /I appended at the END of the include list - // (documented degradation; clang-MSVC uses the gnu dialect). - const bool msvcInclude = d.includePrefix == std::string_view("/I"); - for (auto& inc : plan.manifest.buildConfig.includeDirsAfter) { - includeTokens.push_back( - include_token(d, abs_native(inc), msvcInclude ? "/I" : "-idirafter")); - } - std::string include_flags; - for (auto& t : includeTokens) { - include_flags += ' '; - include_flags += t; // already prefixed, escaped and quoted - } + // NO INCLUDE DIRECTORY IS BROADCAST FROM HERE. + // + // The strings assembled in this function are the FILE-LEVEL `$cxxflags`, + // `$cflags`, `$asmflags` and `$nasmflags` of build.ninja, and every edge in + // the graph reads them: the root's units, every dependency's units and the + // std module alike. The root's `[build] include_dirs` and + // `include_dirs_after` used to be appended here, a leftover from v0.0.1 + // that the usage-requirements model (#101) never removed. Under that model + // a package's include directories are its PRIVATE build requirement, and + // each unit already receives its own package's directories through its + // per-unit `$local_includes` (`CompileUnit::localIncludeDirs`, filled from + // `privateBuild`). The broadcast therefore had one effect of its own: + // it put the root's directories, `private_include_dirs` included, on the + // compile line of every dependency. + // + // Measured before this change (#690, design record F7): a root header + // named `limits.h` stopped `compat.cjson`'s `cJSON.c` from compiling, and + // because the dependency cache key never contained the broadcast, an + // object compiled against one project's root headers was served to an + // unrelated project (`cJSON_Compare(1.0, 1.2)` answered 1). The root's own + // units carried each directory twice. The rule now holds in both + // directions: a package's private build requirements reach only its own + // units, and usage requirements travel from a dependency to its consumers, + // never from a consumer into a dependency. See `kCacheEpoch` 4 in + // cache_key.cppm for the entries written while the broadcast existed. // Sysroot / payload paths — resolved ONCE by the toolchain link model // (mcpp.toolchain.linkmodel, the single source of truth shared with @@ -1091,52 +1069,40 @@ CompileFlags compute_flags(const BuildPlan& plan) { // plan.dialectFlags rides right behind -std= (issue #210): module-graph- // global dialect flags reach every TU (deps included) via this global // cxxflags string, exactly like the standard flag itself. - f.cxx = std::format("{}{}{}{}{}{}{}{}{}{}{}{}", cxx_std_flag, plan.dialectFlags, + f.cxx = std::format("{}{}{}{}{}{}{}{}{}{}{}", cxx_std_flag, plan.dialectFlags, msvc_base, module_flag, std_module_flag, std_compat_module_flag, prebuilt_module_flag, - opt_flag, pic_flag, compile_toolchain_flags, b_flag, include_flags); + opt_flag, pic_flag, compile_toolchain_flags, b_flag); // MSVC compiles C with cl.exe too; /std: for C uses cN spellings — skip // the C standard flag there (cl defaults are fine for the C entry TUs). f.cc = isMsvcDialect - ? std::format("{}{}{}{}{}", msvc_base, opt_flag, compile_toolchain_flags, - b_flag, include_flags) - : std::format("{}{}{}{}{}{}{}", d.stdPrefix, c_std, opt_flag, pic_flag, - compile_toolchain_flags, b_flag, include_flags); + ? std::format("{}{}{}{}", msvc_base, opt_flag, compile_toolchain_flags, + b_flag) + : std::format("{}{}{}{}{}{}", d.stdPrefix, c_std, opt_flag, pic_flag, + compile_toolchain_flags, b_flag); // GAS assembly (.S/.s via the C driver): the asm-safe subset — no -std // (C-only) and no -O (meaningless), but PIC stays (.S sources gate on // __PIC__), -g is fine, and the toolchain-location flags must come along // (hermetic link model: never fall back to a host `as`). MSVC dialect has // no GAS path — prepare hard-errors before these flags are consumed. - f.as = std::format("{}{}{}{}{}", + f.as = std::format("{}{}{}{}", prof.debug ? " -g" : "", pic_flag, - compile_toolchain_flags, b_flag, include_flags); + compile_toolchain_flags, b_flag); // NASM (.asm): fixed GNU-ish spelling of its own — include dirs are // re-spelt with -I regardless of dialect (nasm ≥2.14 inserts a missing // path separator itself); DWARF debug info exists on ELF only. if (!plan.nasmPath.empty()) { - // Same abs_native join as the C/C++ channel above — one decision, one - // implementation. Two knock-on effects, both wanted: the entry is now - // spelt with native separators (#390), and the "already rooted?" test - // becomes has_root_path() instead of is_absolute(), so a root-relative - // `/x` entry is left alone here exactly as it is for the C/C++ include - // list. The two predicates only differ on Windows, and only for that - // spelling — where NASM disagreeing with the compiler about the SAME - // `include_dirs` key was the bug, not the feature. - std::string nasm_includes; - for (auto& inc : plan.manifest.buildConfig.includeDirs) { - nasm_includes += " -I" + escape_path(abs_native(inc)); - } - // #249: nasm has no system header dirs to defer to — after-dirs - // degrade to plain -I appended at the end. - for (auto& inc : plan.manifest.buildConfig.includeDirsAfter) { - nasm_includes += " -I" + escape_path(abs_native(inc)); - } + // Include directories reach a NASM unit through its per-unit + // `$local_includes`, spelt `-I` for every entry (see + // `local_include_flags` in ninja_backend.cppm), for the same reason + // the C/C++ channel above carries none: a file-level list would reach + // every package's `.asm` units. std::string nasm_debug; if (prof.debug && plan.nasmFormat.starts_with("elf")) nasm_debug = " -g -F dwarf"; - f.nasm = nasm_debug + nasm_includes; + f.nasm = nasm_debug; } // Link flags diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 6226cb6b..a43a14c4 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -167,6 +167,37 @@ std::string graph_c_library_isolation_advice(std::string_view output, std::string_view cAbiName = {}, std::string_view cAbiCoordinate = {}); +// #690 (design record F7): the note for a dependency that compiled only because +// its consumer's include directories used to reach it. +// +// The root's `[build] include_dirs` were broadcast to every unit in the graph +// until mcpp 2026.9.25.1; a dependency that included a header it never declared +// compiled as long as the consumer happened to provide one. That dependency now +// fails with an ordinary missing-header error, which on its own reads as a +// broken package. The note names the directory in which the header does exist +// and the rule that changed. +// +// Same shape as the advice above: text-matched against RAW ninja output and +// appended, never rewriting what the compiler said. It fires only when both +// hold: the compiler reported a missing header (gcc `fatal error: X: No such +// file or directory`, clang `fatal error: 'X' file not found`, cl.exe `C1083: +// Cannot open include file: 'X'`), and a regular file named X exists under one +// of `rootIncludeDirs` (absolute paths: the root's `include_dirs` and +// `include_dirs_after`; `private_include_dirs` is a subset of the former). +std::string consumer_include_scope_advice( + std::string_view output, + const std::vector& rootIncludeDirs); + +// The root's include directories as the advice above needs them: expanded and +// absolute. Written beside build.ninja so the fast path, which has no plan, +// gives the same note (see `write_c_abi_absent_sidecar` for why both channels +// must). +std::vector root_include_dirs_of(const BuildPlan& plan); +void write_consumer_include_sidecar(const std::filesystem::path& outputDir, + const std::vector& dirs); +std::vector +read_consumer_include_sidecar(const std::filesystem::path& outputDir); + } // namespace mcpp::build namespace mcpp::build { @@ -748,6 +779,116 @@ read_c_abi_absent_sidecar(const std::filesystem::path& outputDir) { return out; } +namespace { +constexpr std::string_view kConsumerIncludeSidecar = ".mcpp-consumer-include-dirs"; + +// The header named by one compiler diagnostic line, or empty. Three spellings, +// one per driver family; anything else is not a missing-header report. +std::string missing_header_in(std::string_view line) { + if (auto at = line.find("' file not found"); + at != std::string_view::npos && at > 0) { + auto open = line.rfind('\'', at - 1); + if (open != std::string_view::npos && open < at) + return std::string(line.substr(open + 1, at - open - 1)); + return {}; + } + constexpr std::string_view kCl = "Cannot open include file: '"; + if (auto at = line.find(kCl); at != std::string_view::npos) { + auto from = at + kCl.size(); + auto close = line.find('\'', from); + if (close != std::string_view::npos) + return std::string(line.substr(from, close - from)); + return {}; + } + constexpr std::string_view kGcc = "fatal error: "; + constexpr std::string_view kEnoent = ": No such file or directory"; + auto at = line.find(kGcc); + auto end = line.rfind(kEnoent); + if (at != std::string_view::npos && end != std::string_view::npos + && end > at + kGcc.size()) + return std::string(line.substr(at + kGcc.size(), + end - at - kGcc.size())); + return {}; +} +} // namespace + +std::string consumer_include_scope_advice( + std::string_view output, + const std::vector& rootIncludeDirs) { + if (rootIncludeDirs.empty()) return {}; + std::vector> found; + std::size_t pos = 0; + while (pos <= output.size()) { + auto nl = output.find('\n', pos); + auto line = output.substr(pos, nl == std::string_view::npos + ? std::string_view::npos : nl - pos); + pos = nl == std::string_view::npos ? output.size() + 1 : nl + 1; + auto header = missing_header_in(line); + if (header.empty()) continue; + if (std::ranges::any_of(found, [&](auto const& f) { return f.first == header; })) + continue; + for (auto const& dir : rootIncludeDirs) { + std::error_code ec; + if (std::filesystem::is_regular_file(dir / header, ec)) { + found.emplace_back(header, dir); + break; + } + } + } + if (found.empty()) return {}; + std::string named; + for (auto const& [header, dir] : found) + named += std::format("\n '{}' in {}", header, dir.string()); + return std::format( + "\n" + "note: a header reported missing above exists in this project's own " + "include directories:{}\n" + " Since mcpp 2026.9.25.1 a consumer's include directories no longer " + "reach the\n" + " units of its dependencies: a package's `include_dirs` are its own " + "private build\n" + " requirement. The dependency has to reach the header itself, " + "through its own\n" + " `[build] include_dirs` or through a dependency that provides " + "it.\n", + named); +} + +std::vector root_include_dirs_of(const BuildPlan& plan) { + std::vector dirs; + auto add = [&](const std::vector& entries) { + for (auto const& inc : entries) + for (auto& d : expand_manifest_include_entry(plan.projectRoot, inc)) + dirs.push_back(std::move(d)); + }; + add(plan.manifest.buildConfig.includeDirs); + add(plan.manifest.buildConfig.includeDirsAfter); + return dirs; +} + +void write_consumer_include_sidecar(const std::filesystem::path& outputDir, + const std::vector& dirs) { + const auto path = outputDir / kConsumerIncludeSidecar; + std::error_code ec; + if (dirs.empty()) { std::filesystem::remove(path, ec); return; } + std::ofstream f(path, std::ios::binary | std::ios::trunc); + if (!f) return; + for (auto const& d : dirs) f << d.string() << '\n'; +} + +std::vector +read_consumer_include_sidecar(const std::filesystem::path& outputDir) { + std::vector dirs; + std::ifstream f(outputDir / kConsumerIncludeSidecar, std::ios::binary); + if (!f) return dirs; + std::string line; + while (std::getline(f, line)) { + if (!line.empty() && line.back() == '\r') line.pop_back(); + if (!line.empty()) dirs.emplace_back(line); + } + return dirs; +} + std::string c_abi_absent_facility_advice( std::string_view output, std::string_view cAbiName, const std::vector& absent) { @@ -3173,6 +3314,7 @@ std::expected NinjaBackend::build(const BuildPlan& plan plan.outputDir, plan.targetSide.cAbi.interfaceName, plan.targetSide.cAbiDecl ? plan.targetSide.cAbiDecl->absent : std::vector{}); + write_consumer_include_sidecar(plan.outputDir, root_include_dirs_of(plan)); auto manifest = emit_ninja_string(plan); stage("emit-ninja"); @@ -3537,6 +3679,8 @@ std::expected NinjaBackend::build(const BuildPlan& plan // and gets the degraded-but-still-correct form. diagnostics += graph_c_library_isolation_advice( out, plan.targetSide.cAbi.interfaceName, plan.targetSide.cAbi.impl); + // #690: read from the plan here and from the sidecar on the fast path. + diagnostics += consumer_include_scope_advice(out, root_include_dirs_of(plan)); if (plan.targetSide.cAbiDecl) diagnostics += c_abi_absent_facility_advice( out, plan.targetSide.cAbi.interfaceName, diff --git a/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh b/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh new file mode 100755 index 00000000..17809e76 --- /dev/null +++ b/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh @@ -0,0 +1,225 @@ +#!/usr/bin/env bash +# requires: python3 +# 765_a_consumer_include_directory_stays_in_the_consumer.sh +# +# #690, design record F7. A package's `[build] include_dirs` (and +# `private_include_dirs`, a subset of them) are its PRIVATE build requirement: +# they reach the package's own units and nothing else. Until 2026.9.25.1 the +# root's directories were written into the file-level compile flags of +# build.ninja, which every unit in the graph reads, so a root header named like +# a system header shadowed it inside a dependency, and a dependency object in +# the global cache could have been compiled against another project's root +# headers (the key never contained them). +# +# Four assertions, each on the artefact rather than on build success alone: +# (a) a root PRIVATE `limits.h` that is an `#error` does not reach the +# dependency's C or C++ unit: the build succeeds; +# (b) compile_commands.json: no dependency entry carries a root directory; +# the root's own entry carries each root directory exactly once; the +# dependency's PUBLIC include directory still reaches the root; +# (c) the dependency's include/define/feature words are identical under two +# roots that differ only in their include settings; +# (d) a dependency that compiled only because the root provided its header +# now fails, and the output names the root directory that has it, on the +# full path and again on the fast path. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +cd "$TMP" +mkdir -p dep/include/dep dep/src app/src app/appinc app/appprivinc app2/src app2/other + +cat > dep/include/dep/api.h <<'EOF' +#pragma once +#ifdef __cplusplus +extern "C" { +#endif +int dep_c_value(void); +#ifdef __cplusplus +} +#endif +int dep_cxx_value(); +EOF +cat > dep/src/dep.cpp <<'EOF' +#include +#include +int dep_cxx_value() { return INT_MAX > 0 ? 3 : 0; } +EOF +cat > dep/src/dep_c.c <<'EOF' +#include +#include +int dep_c_value(void) { return CHAR_BIT == 8 ? 4 : 0; } +EOF +cat > dep/mcpp.toml <<'EOF' +[package] +name = "dep" +version = "0.1.0" + +[build] +include_dirs = ["include"] +sources = ["src/dep.cpp", "src/dep_c.c"] + +[targets.dep] +kind = "lib" +EOF + +# The root: two include directories, one of them private, and a private header +# that must never be seen by anyone but the root itself. +echo '#define APP_CFG 7' > app/appinc/appcfg.h +echo '#define APP_PRIV 1' > app/appprivinc/apppriv.h +printf '#error ROOT PRIVATE HEADER REACHED A DEPENDENCY\n' > app/appprivinc/limits.h +cat > app/src/main.cpp <<'EOF' +#include +#include "appcfg.h" +#include "apppriv.h" +int main() { return dep_cxx_value() + dep_c_value() == APP_CFG + APP_PRIV - 1 ? 0 : 1; } +EOF +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +dep = { path = "../dep" } + +[build] +include_dirs = ["appinc", "appprivinc"] +private_include_dirs = ["appprivinc"] + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +# A second root that differs only in its include settings. +echo '#define OTHER 1' > app2/other/other.h +cat > app2/src/main.cpp <<'EOF' +#include +#include "other.h" +int main() { return dep_cxx_value() + dep_c_value() + OTHER == 8 ? 0 : 1; } +EOF +cat > app2/mcpp.toml <<'EOF' +[package] +name = "app2" +version = "0.1.0" + +[dependencies] +dep = { path = "../dep" } + +[build] +include_dirs = ["other"] + +[targets.app2] +kind = "bin" +main = "src/main.cpp" +EOF + +# ── (a) ────────────────────────────────────────────────────────────────────── +(cd app && "$MCPP" build > build.log 2>&1) || { + cat app/build.log + echo "FAIL (a): the root's private include directory reached a dependency unit" + exit 1 +} +(cd app2 && "$MCPP" build > build.log 2>&1) || { cat app2/build.log; echo "FAIL: app2 build"; exit 1; } + +# ── (b) and (c) ────────────────────────────────────────────────────────────── +python3 - "$TMP" <<'PY' +import json, os, sys +tmp = sys.argv[1] +def entries(root): + return json.load(open(os.path.join(tmp, root, "compile_commands.json"))) +def args(e): + return e.get("arguments") or e["command"].split() +def norm(s): + return s.replace("\\", "/") +def count(a, needle): + return sum(1 for x in a if needle in norm(x)) +fails = [] +app = entries("app") +dep_units = [e for e in app if norm(e["file"]).split("/")[-1] in ("dep.cpp", "dep_c.c")] +main_units = [e for e in app if norm(e["file"]).endswith("app/src/main.cpp")] +if len(dep_units) != 2: fails.append(f"(b) expected 2 dependency entries, found {len(dep_units)}") +if len(main_units) != 1: fails.append(f"(b) expected 1 root entry, found {len(main_units)}") +for e in dep_units: + for d in ("app/appinc", "app/appprivinc"): + if count(args(e), d): + fails.append(f"(b) {norm(e['file'])} carries the root directory {d}") +for e in main_units: + a = args(e) + for d in ("app/appinc", "app/appprivinc"): + if count(a, d) != 1: + fails.append(f"(b) root entry carries {d} {count(a, d)} times, expected once") + if count(a, "dep/include") < 1: + fails.append("(b) the dependency's public include directory does not reach the root") +def words(root): + out = {} + for e in entries(root): + f = norm(e["file"]).split("/")[-1] + if f not in ("dep.cpp", "dep_c.c"): continue + out[f] = sorted(norm(x) for x in args(e) + if x.startswith(("-I", "-D", "-f", "-idirafter", "/I", "/D"))) + return out +w1, w2 = words("app"), words("app2") +if not w1 or w1 != w2: + fails.append(f"(c) the dependency's words differ between two roots:\n app : {w1}\n app2: {w2}") +for f in fails: print("FAIL", f) +sys.exit(1 if fails else 0) +PY + +# ── (d) ────────────────────────────────────────────────────────────────────── +mkdir -p leaner/src app3/src app3/appinc +echo '#define FROM_CONSUMER 1' > app3/appinc/needs_consumer.h +cat > leaner/src/leaner.cpp <<'EOF' +#include +int leaner_value() { return FROM_CONSUMER; } +EOF +cat > leaner/mcpp.toml <<'EOF' +[package] +name = "leaner" +version = "0.1.0" + +[build] +sources = ["src/leaner.cpp"] + +[targets.leaner] +kind = "lib" +EOF +cat > app3/src/main.cpp <<'EOF' +int leaner_value(); +int main() { return leaner_value() == 1 ? 0 : 1; } +EOF +cat > app3/mcpp.toml <<'EOF' +[package] +name = "app3" +version = "0.1.0" + +[dependencies] +leaner = { path = "../leaner" } + +[build] +include_dirs = ["appinc"] + +[targets.app3] +kind = "bin" +main = "src/main.cpp" +EOF +cd app3 +for pass in full fast; do + if "$MCPP" build > "d-$pass.log" 2>&1; then + cat "d-$pass.log" + echo "FAIL (d, $pass): a dependency compiled against its consumer's header" + exit 1 + fi + grep -q "needs_consumer.h" "d-$pass.log" || { + cat "d-$pass.log"; echo "FAIL (d, $pass): failed for an unexpected reason"; exit 1; } + grep -q "no longer" "d-$pass.log" && grep -q "appinc" "d-$pass.log" || { + cat "d-$pass.log" + echo "FAIL (d, $pass): the advice naming the consumer's directory is missing" + exit 1 + } +done + +echo "OK: a consumer's include directories stay in the consumer" diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index cb659bec..ab370a97 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -180,48 +180,46 @@ TEST(NinjaBackend, CompileCommandsUsesSameCppStandard) { << cdb; } -TEST(NinjaBackend, CxxFlagsIncludeBuildIncludeDirs) { - auto plan = minimal_plan(); - plan.manifest.buildConfig.includeDirs = {"include", "third_party/imgui"}; - - auto flags = compute_flags(plan); - - EXPECT_NE(flags.cxx.find(escaped_include_flag(plan.projectRoot / "include")), - std::string::npos) - << flags.cxx; - // #390: a multi-segment entry is normalized to NATIVE separators before - // the -I token is built (a mixed `...\third_party/imgui` used to reach - // the CDB through f.cxx). Build the expected path natively too. - auto imgui = plan.projectRoot / "third_party" / "imgui"; - EXPECT_NE(flags.cxx.find(escaped_include_flag(imgui)), - std::string::npos) - << flags.cxx; -} - -// #390: the NASM include list is built from the SAME `[build] include_dirs` -// key as the C/C++ one, so it must absolutize and spell entries identically — -// it used to re-derive the join on its own (and with a different "already -// rooted?" predicate). Nothing here is nasm-specific except the channel: the -// point is that one manifest key cannot produce two different paths. -TEST(NinjaBackend, NasmIncludeDirsMatchTheCxxChannelSpelling) { +// #690 (design record F7): the root's `[build] include_dirs` and +// `include_dirs_after` are its PRIVATE build requirement. They reach the root's +// units through their per-unit `$local_includes` and must not appear in the +// FILE-LEVEL flag strings, which every unit in the graph reads (every +// dependency's units and the std module included). Before the fix both keys +// were appended to all four strings, so a root header named like a system or +// dependency header shadowed it inside the dependency, and a dependency cache +// entry could be compiled against another project's root headers. +TEST(NinjaBackend, FileLevelFlagsCarryNoRootIncludeDirectory) { auto plan = minimal_plan(); plan.nasmPath = "/usr/bin/nasm"; - plan.manifest.buildConfig.includeDirs = {"third_party/imgui"}; + plan.manifest.buildConfig.includeDirs = {"include", "third_party/imgui"}; plan.manifest.buildConfig.includeDirsAfter = {"generated/inc"}; auto flags = compute_flags(plan); - auto native = [](std::filesystem::path p) { p.make_preferred(); return p; }; - auto imgui = native(plan.projectRoot / "third_party" / "imgui"); - auto gen = native(plan.projectRoot / "generated" / "inc"); + for (auto const* channel : {&flags.cxx, &flags.cc, &flags.as, &flags.nasm}) { + for (auto const* dir : {"include", "imgui", "generated"}) { + EXPECT_EQ(channel->find(dir), std::string::npos) + << "root include directory '" << dir + << "' reached a file-level flag string: " << *channel; + } + } +} - // Absolutized against projectRoot, natively spelt, and -I for BOTH keys - // (nasm has no system-header chain to defer to, so after-dirs degrade). - EXPECT_NE(flags.nasm.find(escaped_include_flag(imgui)), std::string::npos) - << flags.nasm; - EXPECT_NE(flags.nasm.find(escaped_include_flag(gen)), std::string::npos) - << flags.nasm; - EXPECT_EQ(flags.nasm.find("-idirafter"), std::string::npos) << flags.nasm; +// What the file-level channel used to guarantee now holds on the per-unit +// one: a relative `[build] include_dirs` entry is absolutised against the +// project root and natively spelt before it becomes a unit's include +// directory (#390; the Cluster A review fix of #226/#234, whose failure shape +// was a relative `/Iinclude` under the MSVC dialect that stopped resolving +// once ninja ran from the output directory). +TEST(NinjaBackend, RootIncludeEntriesAreAbsolutisedForTheUnitChannel) { + auto plan = minimal_plan(); + auto dirs = mcpp::build::expand_manifest_include_entry( + plan.projectRoot, "third_party/imgui"); + ASSERT_EQ(dirs.size(), 1u); + auto expected = plan.projectRoot / "third_party" / "imgui"; + expected.make_preferred(); + EXPECT_EQ(dirs.front(), expected); + EXPECT_TRUE(dirs.front().has_root_path()) << dirs.front(); } // #249: a compile unit's localIncludeDirsAfter emit as -idirafter into the @@ -397,41 +395,6 @@ TEST(NinjaBackend, NasmUnitEmitsIncludeDirsAfterAsPlainDashI) { EXPECT_EQ(line.find("-idirafter"), std::string::npos) << line; } -// Cluster A review fix (#226/#234 follow-up): `[build] include_dirs` is a -// TYPED PATH channel — bare paths from the manifest, dialect prefix applied -// at emission (-I under GNU, /I under MSVC) — not the FLAG-STRING channel -// that normalize_include_flags serves (cflags/cxxflags, where the prefix is -// already embedded in the string by the scanner). Routing dialect-prefixed -// include tokens through normalize_include_flags (whose prefix table only -// knows GNU spellings: -I/-iquote/-isystem/-idirafter/-iprefix/-L) silently -// no-ops under MSVC: "/Iinclude" matches no table entry and is never -// rewritten against plan.projectRoot, so it survives as a *relative* path — -// but ninja runs with cwd = the output dir, so the include stops resolving. -// The fix absolutizes the path directly (dialect-agnostic) before -// prepending the dialect prefix. This test would FAIL before the fix -// (emitting the literal, unrewritten "/Iinclude") and passes after. -TEST(NinjaBackend, MsvcIncludeDirsAreAbsolutizedNotGnuNormalized) { - // The MSVC-dialect logic under test is host-independent; run it on POSIX - // where the test's temp projectRoot has no drive letter. On Windows the - // runner's `C:\...` temp path gets its `:` ninja-escaped (`C$:`), which - // would need escape-aware matching unrelated to what this test verifies. - if constexpr (mcpp::platform::is_windows) - GTEST_SKIP() << "MSVC-dialect path check runs on POSIX (avoids Windows drive-colon ninja escaping)"; - - auto plan = minimal_plan(); - plan.toolchain.compiler = mcpp::toolchain::CompilerId::MSVC; - plan.toolchain.binaryPath = "cl.exe"; - plan.toolchain.targetTriple = "x86_64-pc-windows-msvc"; - plan.manifest.buildConfig.includeDirs = {"include"}; - - auto flags = compute_flags(plan); - - auto expected = "/I" + (plan.projectRoot / "include").string(); - EXPECT_NE(flags.cxx.find(expected), std::string::npos) << flags.cxx; - // The un-rewritten, still-relative token must never appear. - EXPECT_EQ(flags.cxx.find("/Iinclude"), std::string::npos) << flags.cxx; -} - // ── assembly sources (.S/.s → asm_object via $cc, .asm → nasm_object) ──────── TEST(NinjaBackend, GasSourceUsesAsmObjectRule) { @@ -1689,6 +1652,71 @@ TEST(GraphCLibraryIsolationAdvice, DegradesWithoutNamingAnyPackage) { EXPECT_NE(advice.find("this target's C library"), std::string::npos); } +// ── consumer_include_scope_advice (#690, design record F7) ────────────────── +// +// A dependency that compiled only because its consumer's include directories +// used to be broadcast to it now fails with an ordinary missing header. The +// note fires on the compiler's report AND a file of that name in one of the +// root's include directories; either alone is not this shape. + +namespace { +std::filesystem::path consumer_include_fixture() { + auto root = std::filesystem::temp_directory_path() + / std::format("mcpp-consumer-include-{}", + std::chrono::steady_clock::now().time_since_epoch().count()); + std::filesystem::create_directories(root / "appinc" / "cfg"); + std::ofstream(root / "appinc" / "cfg7.h") << "#define WHO 1\n"; + std::ofstream(root / "appinc" / "cfg" / "nested.h") << "#define N 1\n"; + return root; +} +} // namespace + +TEST(ConsumerIncludeScopeAdvice, NamesTheConsumerDirectoryForEachDriverSpelling) { + const auto root = consumer_include_fixture(); + const std::vector dirs{root / "absent", root / "appinc"}; + const std::string gcc = + "FAILED: obj/dep/src/dep.c.o\n" + "../dep/src/dep.c:1:10: fatal error: cfg7.h: No such file or directory\n"; + const std::string clang = + "../dep/src/dep.cpp:1:10: fatal error: 'cfg/nested.h' file not found\n"; + const std::string cl = + "dep.cpp(1): fatal error C1083: Cannot open include file: 'cfg7.h': " + "No such file or directory\n"; + for (auto const& out : {gcc, clang, cl}) { + auto advice = mcpp::build::consumer_include_scope_advice(out, dirs); + ASSERT_FALSE(advice.empty()) << out; + EXPECT_NE(advice.find((root / "appinc").string()), std::string::npos) << advice; + EXPECT_NE(advice.find("no longer"), std::string::npos) << advice; + } + std::filesystem::remove_all(root); +} + +TEST(ConsumerIncludeScopeAdvice, SilentWhenTheHeaderIsNotTheConsumersOrNoHeaderIsMissing) { + const auto root = consumer_include_fixture(); + const std::vector dirs{root / "appinc"}; + // A missing header the consumer does not have either: an ordinary error. + EXPECT_TRUE(mcpp::build::consumer_include_scope_advice( + "dep.c:1:10: fatal error: other.h: No such file or directory\n", dirs).empty()); + // A failure that is not a missing header, although cfg7.h is the consumer's. + EXPECT_TRUE(mcpp::build::consumer_include_scope_advice( + "dep.c:3:1: error: unknown type name 'cfg7'\n", dirs).empty()); + // No root include directories at all (the fast path before any plan wrote + // the sidecar). + EXPECT_TRUE(mcpp::build::consumer_include_scope_advice( + "dep.c:1:10: fatal error: cfg7.h: No such file or directory\n", {}).empty()); + std::filesystem::remove_all(root); +} + +TEST(ConsumerIncludeScopeAdvice, SidecarRoundTrips) { + const auto root = consumer_include_fixture(); + const std::vector dirs{root / "appinc", root / "with space"}; + mcpp::build::write_consumer_include_sidecar(root, dirs); + EXPECT_EQ(mcpp::build::read_consumer_include_sidecar(root), dirs); + mcpp::build::write_consumer_include_sidecar(root, {}); + EXPECT_TRUE(mcpp::build::read_consumer_include_sidecar(root).empty()); + std::filesystem::remove_all(root); +} + TEST(GraphCLibraryIsolationAdvice, DoesNotRewriteTheCompilersOwnLine) { // "Reuse the channel, don't rewrite compiler output": the advice is // APPENDED text, and the original diagnostic line survives byte for byte From f39d1ab349a922bc3668a8e6cd51aae1b5ba010a Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:39:35 +0800 Subject: [PATCH 04/15] docs: workspace inheritance in every position, keyed defines, include scope; SPEC-004 1.6 (#690) --- ...workspace-build-inheritance-consistency.md | 6 ++- docs/04-mcpp-toml.md | 36 ++++++++++++- docs/07-workspace.md | 36 ++++++++++++- docs/specs/manifest-semantics.md | 52 ++++++++++++++++--- docs/zh/04-mcpp-toml.md | 30 ++++++++++- docs/zh/07-workspace.md | 31 ++++++++++- .../e2e/770_workspace_member_as_dependency.sh | 2 +- 7 files changed, 178 insertions(+), 15 deletions(-) diff --git a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md index 8f65ee57..79cd8a55 100644 --- a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md +++ b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md @@ -73,6 +73,8 @@ The decisions below are derived from these rules. Each rule names its source in **F2 (measured): the selected root member inherits twice.** The root inherits at load (`prepare.cppm:2365`/`2384`). `packages[0] = makePackageRoot(*root, *m)` (`prepare.cppm:6774`) finds it is a member and inherits again. The measured compile line is `-DFLAGLEVEL=1 -fno-exceptions -DFLAGLEVEL=1 -fno-exceptions -DFLAGLEVEL=2 -fexceptions`, which is the workspace entries twice followed by the member's once. The second copy is prepended as well, so the outcome of any override is unchanged. The cost falls on flags that are not idempotent, such as `-include x.h` without a guard, or options that accumulate. +**F10 (measured during implementation): a member reached as a dependency does not resolve its own `x.workspace = true` entries.** A sibling `lib` whose manifest says `util.workspace = true` builds under `-p lib`. Under `-p app` it fails with `dependency 'util' has SemVer constraint '' but no readable index entry for it`. The dependency load site applied only `inherit_workspace_package`, so the entry reached resolution with neither version nor path. This is the same placement defect as F1, for the third part of what a member inherits. + **F8 (reasoned): the F1 dependency's cache key records a define its compile does not carry.** `cache_key.cppm:534` reads `pkg.manifest.buildConfig.defines`, which retains the unfolded entries. Root cause: the build half of inheritance runs inside the snapshot, after the fold. @@ -152,7 +154,7 @@ Measured on a git repository whose workspace declares `version`, `license` and ` | **D1** | Should a git-hosted workspace's `[workspace.build]` reach its git-consumed members? | **Yes.** Accepted in review. | P1. The flags are the member's statement about its own compilation, factored into its repository root. A member's own `[build]` already applies through git, and #650 adopted the package half for the same reason. The behaviour change is recorded in the CHANGELOG. | | **D2** | What is the published form of a member? | **A normalised, self-contained manifest** (section 5.5). Publishing is refused only where normalisation cannot preserve meaning. | P6. The development form is valid only inside its workspace. Refusing every member whose effective manifest differs from its file would refuse every member that inherits anything, which is the purpose of a workspace. | | **Q3** | Is the snapshot post-condition an internal error in release builds? | **Yes**, following `plan.cppm:1797`: `std::unexpected("internal error: ... (please report)")`. | P7. The defect it catches is silent, and the check costs one comparison per package. | -| **D3** | How does a member override inherited vectors? | **(a)** document the actual rule now; **(b)** make `defines` a keyed set as a follow-up. Include ordering and opt-out wait for evidence. | Section 5.7. | +| **D3** | How does a member override inherited vectors? | **(a)** document the actual rule; **(b)** make `defines` a keyed set. Both are delivered in the #690 pull request (review of 2026-09-25 asked for one pull request per repository). Include ordering and opt-out wait for evidence. | Section 5.7. | | **D4** | What replaces the root include broadcast? | **Nothing implicit.** The root's include directories become private to the root, as #101 intended. A dependency that needs a consumer-supplied header receives it through an explicit, keyed mechanism, designed only when a package needs it. | P4, P5, and the measured absence of reliance. | --- @@ -242,6 +244,8 @@ Each criterion fails when its fix is removed. ## 7. Delivery +Superseded in review (2026-09-25): every row below is delivered in one pull request, and each workstream keeps its own criterion inside it. The order and dependencies are in [the implementation plan](2026-09-25-issue-690-implementation-plan.md). + | PR | Content | Depends on | |---|---|---| | A | W1, the post-condition, W7(a), criteria 1 and 2, SPEC-004 §9 draft, CHANGELOG. Closes #690. | none | diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index e58c5d7f..c30e0421 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -400,14 +400,14 @@ the package/feature boundary, not on an individual target. sources = ["src/**/*.cppm", "src/**/*.cpp"] # Source globs (default: src/**/*.{cppm,cpp,cc,c,S,s,asm}) module_extensions = [".ixx"] # Extra extensions used by module INTERFACES (§ below) build_program_timeout = 1800 # Seconds a build.mcpp may run; 0 = no limit (§ below) -include_dirs = ["include", "third_party/include"] # Header search paths +include_dirs = ["include", "third_party/include"] # Header search paths of this package (§ below) include_dirs_after = ["*"] # Header dirs searched AFTER system dirs (-idirafter) private_include_dirs = ["vendor/src/include"] # Of `include_dirs`, the ones a consumer must NOT get c_standard = "c11" # Standard for C source files (default c11) cflags = ["-DFOO=1"] # Extra C compile flags cxxflags = ["-DBAR=2"] # Extra C++ compile flags (do not put -std=... here) ldflags = ["-lfoo"] # Extra link flags -defines = ["BIZ=1", "QUX"] # Preprocessor macros for every TU (desugars to -D; reaches module scans) +defines = ["BIZ=1", "QUX"] # Preprocessor macros for every TU of this package; keyed by name (§ below) cxx_runtime = "self-contained" # C++ runtime contract (§ below); static_stdlib is the old spelling target = "x86_64-linux-musl" # Default build target when no --target is passed # (≙ cargo build.target; e.g. "ship fully-static") @@ -462,6 +462,38 @@ element whose words differ from the arguments a release before 2026.9.17.1 passed on the same host, and names both. A build that repeats the plan does not repeat the warning. +#### What reaches which unit: `defines` and include directories *(mcpp 2026.9.25.1+)* + +`defines` is a set keyed by macro name. Its entries are read in the order the +package receives them: `[workspace.build]` (for a workspace member), the +package's own `[build]`, then each matching `[target..build]`. A later +entry for a name replaces the earlier one, and an entry `!NAME` removes the name. +Each name reaches the compiler as one `-DNAME` word, on the C and the C++ units +alike: + +```toml +[build] +defines = ["LEVEL=1", "TRACE"] + +[target.'cfg(os = "windows")'.build] +defines = ["LEVEL=2", "!TRACE"] # on Windows: -DLEVEL=2, and no TRACE +``` + +A `defines` entry also replaces a `-DNAME` word for the same name that the +package writes in `cflags` or `cxxflags`. An older mcpp passes `!NAME` to the +compiler as `-D!NAME`, which is an error, so a package that uses it needs mcpp +2026.9.25.1 or later. + +`include_dirs` and `include_dirs_after` are searched by this package's own +units. A package's consumers also receive them, except the entries named in +`private_include_dirs`. Its dependencies never receive them: a dependency is +compiled against its own include directories and the ones its own dependencies +publish, whichever project builds it. A dependency that includes a header must +reach it through its own `include_dirs` or through one of its dependencies. +When a dependency's compile reports a missing header that exists in the +consumer's include directories, mcpp names that directory after the compiler's +message. + #### `dependency_linkage` — static or shared is the consumer's decision ```toml diff --git a/docs/07-workspace.md b/docs/07-workspace.md index c8dcf220..d0547fb5 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -195,7 +195,8 @@ name = "core" | kind | rule | |---|---| | scalars (`standard`, `version`, `license`, `c_standard`, `linkage`, …) | the member wins **when it declared the key**; otherwise the workspace value applies | -| vectors (`cxxflags`, `ldflags`, `defines`, `dialect_cxxflags`, `include_dirs`, …) | append, **workspace first** — so a member's own flag comes later on the command line, where it wins | +| vectors (`cxxflags`, `cflags`, `ldflags`, `dialect_cxxflags`, `include_dirs`, …) | append, **workspace first** | +| `defines` | a set keyed by macro name: a member entry for an inherited name replaces it, and `!NAME` removes it (2026.9.25.1+) | | `[workspace.dependencies]` | explicit opt-in per dependency, `x.workspace = true` (§3) | "Declared" means the key was written, not that its value differs from the @@ -211,6 +212,34 @@ Dependencies keep their explicit opt-in because a dependency is an edge in the resolution graph: inheriting one implicitly would change what a member resolves without its own manifest naming it. +**What an appended vector overrides.** The member's words follow the +workspace's on the command line. A flag the compiler resolves last-wins is +therefore overridden by restating it: `-fexceptions` after `-fno-exceptions`, +`-Wno-x` after `-Wx`, `-O2` after `-O0`. Include directories are searched in +order, so a header in a workspace `include_dirs` directory is found before a +header of the same name in the member's. A macro is overridden through +`defines`, which emits one `-DNAME` word per name: + +```toml +# workspace root +[workspace.build] +defines = ["LOG_LEVEL=1", "TRACE"] + +# member +[build] +defines = ["LOG_LEVEL=3", "!TRACE"] # compiles with -DLOG_LEVEL=3 and no TRACE +``` + +A `defines` entry also replaces a `-DNAME` word for the same name written in +`cflags` or `cxxflags` of the same package. `!NAME` requires mcpp 2026.9.25.1 or +later; an older mcpp passes it to the compiler as `-D!NAME`, which is an error. + +**Every member receives the inherited values exactly once, in every position** +(2026.9.25.1+): as the package a command builds (`-p `, or a command run +inside the member), as another member's `path` dependency, and as a member of a +git-hosted workspace consumed through `git` (§6). A member reached as a dependency +also resolves its own `x.workspace = true` entries. + **`version` may be omitted by a member** when `[workspace.package]` supplies it. It remains required overall — a member with neither is refused, naming both the member and the workspace key that would have supplied it. @@ -376,7 +405,10 @@ A project outside the workspace reaches a member of a git-hosted workspace by the member's identity: `myproject.http = { git = "...", rev = "..." }` selects `libs/http` among the root manifest's `members`, at the same commit, and the member inherits `[workspace.package]` as it does here (mcpp 2026.9.16.1+; see -[05 — Dependencies](05-dependencies.md)). +[05 — Dependencies](05-dependencies.md)). It also inherits its repository's +`[workspace.build]` and resolves its `x.workspace = true` entries against that +repository's `[workspace.dependencies]` (2026.9.25.1+), so the same commit +compiles the same way in its own checkout and in a consumer's graph. ## 7. Relationship to C++ Modules diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 88ae3770..6cfd92f2 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -5,11 +5,11 @@ | **规范编号** | SPEC-004 | | **标题** | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | | **状态** | **草案(Draft)** | -| **版本** | 1.5 | -| **最后修改** | 2026-09-17 | +| **版本** | 1.6 | +| **最后修改** | 2026-09-25 | | **最低实现版本** | 条件化形状:mcpp **2026.8.29.1**(`[target..build-dependencies]` 起齐备);目标轴:mcpp **2026.9.6.4** | | **作者/维护** | mcpp-community | -| **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md` | +| **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md`
`.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md` | | **相关使用文档** | [docs/04 —— mcpp.toml 字段参考](../04-mcpp-toml.md) | ## 规范用语 @@ -33,7 +33,7 @@ 一个条目按什么解析、键怎么命名。它不列举字段——字段在 docs/05。 它回答的是一个新字段或新 section 该长什么样,以及一份 manifest 为什么这样组织。 -§8 另陈述编译 flag 列表中一个元素代表哪些参数。 +§8 另陈述编译 flag 列表中一个元素代表哪些参数，§9 陈述工作空间继承与构建需求的作用域。 **边界。** 本规范不覆盖字段的准入条件,那由 docs/05 附录 A(Schema Ownership Principle)规定,本规范不重复它,只在 §6 引用并补充一条。 @@ -334,6 +334,16 @@ feature-deps feature-xlings ← 限定词是门 规则本身由读回性质陈述:任意词经实现的拼写读回为它自身,经宿主引号读回也为它自身, 且 POSIX 宿主上由 `/bin/sh` 实测(`modules/manifest/tests/test_flag_words.cpp`、 `tests/unit/test_compile_commands.cpp`)。 +11. §8 `defines` 集合语义与 §9 第 1 至 3 条的判据:同一成员分别作为命令构建的包与作为兄弟 + 成员的 `path` 依赖时,每个工作空间词在 C 与 C++ 编译单元中各恰好出现一次且先于成员 + 自己的词;成员重写的宏名只出现成员的值,`!NAME` 移除的宏名不出现;作为依赖的成员 + 解析自己的 `x.workspace = true` 条目;通过 `git` 引用的仓库成员收到其仓库的 + `[workspace.build]`(`tests/e2e/770_workspace_member_as_dependency.sh`)。第 3 条的 + 内部错误与第 4 条的键表由单元测试陈述(`tests/unit/test_workspace_inheritance.cpp`)。 +12. §9 第 6 条的判据:根包私有目录中的 `limits.h` 不到达依赖的 C 与 C++ 编译单元;依赖 + 的编译命令在两个仅 include 设置不同的根包下相同;根包自己的编译单元中每个根包目录 + 恰好出现一次;依赖因此找不到头文件时,报错之后指出消费者目录 + (`tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh`)。 ## 8. 编译 flag 列表的元素 @@ -356,14 +366,43 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 不适用。自 mcpp#234 起,每个版本在宿主读取之前都把这样的元素整体加引号,该例外保持其含义不变。 `defines` 的一个条目 `X` 是一个值:它代表一个词 `-D` 与 `X` 的拼接,**禁止**按上述规则 -读取。实现向这三个列表插入一个词 `w` 时,**必须**使用一个按上述规则读回恰为 `w` 的拼写。 +读取。 + +`defines` 是按宏名构成的集合(mcpp 2026.9.25.1 起)。条目的宏名是 `=` 之前的文本, +无 `=` 时为整个条目。条目按包接收它们的顺序读取:`[workspace.build]`、包自己的 +`[build]`、各个命中的 `[target..build]`。实现**必须**满足:同一宏名的后一个 +条目在原位替换前一个;条目 `!NAME` 移除宏名 `NAME`;一个包的每个编译单元对每个宏名 +至多收到一个 `-D` 词;`defines` 条目取代同一个包的 `cflags`、`cxxflags` 中读作单个词 +且宏名相同的 `-D` 词。实现向这三个列表插入一个词 `w` 时,**必须**使用一个按上述规则读回恰为 `w` 的拼写。 实现**必须**把每个词原样交给编译器,与宿主的命令行读取规则(POSIX `sh`、MSVCRT)无关; `compile_commands.json` 与构建数据库(SPEC-005 R3.7)列出的参数**必须**是这些词。 `ldflags`、`dialect_cxxflags` 与 `std-module-flags` 不在本节范围内。 -**状态:已实现(mcpp 2026.9.17.1)。** +**状态:已实现(mcpp 2026.9.17.1;`defines` 的集合语义 mcpp 2026.9.25.1)。** + +## 9. 工作空间继承与构建需求的作用域 + +1. 工作空间成员**必须**恰好接收一次 `[workspace.package]`、`[workspace.build]` 与 + `x.workspace = true` 条目的继承,无论它是命令构建的包、另一个成员的 `path` 依赖, + 还是通过 `git` 引用的、托管在 git 上的工作空间的成员。后一种情况按该成员所在仓库的 + 工作空间根继承,相对路径以该仓库的根为锚点。 +2. 向量按工作空间、成员、命中的 `[target..build]` 的顺序追加;`defines` 按 + §8 的集合语义合并。标量仅在成员未**声明**该键时取工作空间的值。 +3. 继承**必须**在 `defines` 展开之前、在清单被固定进构建图之前完成。实现**必须**拒绝 + 把含有未展开 `defines` 的清单固定进构建图,并报告内部错误。 +4. 可继承的 `[build]` 键集合只陈述一次。解析、已知键检查与报错文本**必须**取自同一 + 陈述。 +5. 读取成员清单的每一条命令**必须**读取继承后的有效清单。 +6. 一个包的私有构建需求,包括它的 `include_dirs` 与 `include_dirs_after`,**禁止**到达 + 另一个包的编译单元。使用需求只从依赖流向它的消费者,**禁止**从消费者流入依赖。 + 一个依赖的编译命令**必须**与构建它的工程无关;依赖缓存键**必须**包含到达该命令的 + 全部输入。 +7. 工作空间成员的发布形态**必须**自包含:发布的清单写出继承来的值,兄弟成员之间的 + `path` 边以版本边发布,无法以版本表达的 `path` 边**必须**被拒绝发布。 + +**状态:已实现(mcpp 2026.9.25.1)。** ## 变更记录 @@ -375,3 +414,4 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 | 1.3 | 2026-09-14 | 条件依赖声明替换同一身份的无条件声明,`targets.` 成为可条件化的 section,不读取的 section 必须报出(mcpp 2026.9.14.2):新增 §3.1.1 与 §7 第 8 条判据。 | | 1.4 | 2026-09-15 | 库目标的默认链接形态 `linkage`(mcpp 2026.9.15.2):§3.1.1 补默认值的语义、优先顺序与拒绝条件;§7 补第 9 条判据。 | | 1.5 | 2026-09-17 | 编译 flag 列表元素的读法(mcpp 2026.9.17.1,#655):新增 §8 与 §7 第 10 条判据。 | +| 1.6 | 2026-09-25 | 工作空间继承与构建需求的作用域(mcpp 2026.9.25.1,#690):§8 补 `defines` 的集合语义;新增 §9 与 §7 第 11、12 条判据。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 6b9e4d28..556db48b 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -402,14 +402,14 @@ mcpp 刻意不在一次构建里用两种方式编译同一份共享源码：一 sources = ["src/**/*.cppm", "src/**/*.cpp"] # Source globs (default: src/**/*.{cppm,cpp,cc,c,S,s,asm}) module_extensions = [".ixx"] # Extra extensions used by module INTERFACES (§ below) build_program_timeout = 1800 # Seconds a build.mcpp may run; 0 = no limit (§ below) -include_dirs = ["include", "third_party/include"] # Header search paths +include_dirs = ["include", "third_party/include"] # 本包的头文件搜索路径（见下文） include_dirs_after = ["*"] # Header dirs searched AFTER system dirs (-idirafter) private_include_dirs = ["vendor/src/include"] # Of `include_dirs`, the ones a consumer must NOT get c_standard = "c11" # Standard for C source files (default c11) cflags = ["-DFOO=1"] # Extra C compile flags cxxflags = ["-DBAR=2"] # Extra C++ compile flags (do not put -std=... here) ldflags = ["-lfoo"] # Extra link flags -defines = ["BIZ=1", "QUX"] # Preprocessor macros for every TU (desugars to -D; reaches module scans) +defines = ["BIZ=1", "QUX"] # 本包每个编译单元的预处理宏；按宏名构成集合（见下文） cxx_runtime = "self-contained" # C++ runtime contract (§ below); static_stdlib is the old spelling target = "x86_64-linux-musl" # Default build target when no --target is passed # (≙ cargo build.target; e.g. "ship fully-static") @@ -462,6 +462,32 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | 同一宿主上传出的参数不同，mcpp 会在 `build/flag-words` 下发出警告，并 点名两者。重复同一份 plan 的构建不会重复这条警告。 +#### 什么到达哪个编译单元：`defines` 与头文件目录 *(mcpp 2026.9.25.1+)* + +`defines` 是按宏名构成的集合。条目按包接收它们的顺序读取：`[workspace.build]` +（对工作空间成员而言）、包自己的 `[build]`，然后是每个命中的 +`[target..build]`。同一宏名的后一个条目替换前一个，条目 `!NAME` 移除该 +宏名。每个宏名以一个 `-DNAME` 词到达编译器，C 与 C++ 编译单元相同： + +```toml +[build] +defines = ["LEVEL=1", "TRACE"] + +[target.'cfg(os = "windows")'.build] +defines = ["LEVEL=2", "!TRACE"] # Windows 上：-DLEVEL=2，且不定义 TRACE +``` + +`defines` 条目同样替换本包在 `cflags` 或 `cxxflags` 中写下的同名 `-DNAME` 词。 +更早的 mcpp 会把 `!NAME` 作为 `-D!NAME` 交给编译器，那是一个错误，因此使用它的包 +需要 mcpp 2026.9.25.1 或更新版本。 + +`include_dirs` 与 `include_dirs_after` 由本包自己的编译单元搜索。本包的消费者也会 +收到它们，`private_include_dirs` 中列出的条目除外。本包的依赖永远收不到它们：无论 +由哪个工程构建，依赖都只对照自己的头文件目录以及它自己的依赖所公开的目录编译。 +依赖若包含某个头文件，必须通过自己的 `include_dirs` 或它的某个依赖找到它。当依赖的 +编译报告缺少某个头文件、而该文件存在于消费者的头文件目录中时，mcpp 会在编译器的 +报错之后指出那个目录。 + #### `dependency_linkage` —— 静态还是动态由消费者决定 ```toml diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index 128c4c43..876a7d20 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -198,7 +198,8 @@ name = "core" | 种类 | 规则 | |---|---| | 标量（`standard`、`version`、`license`、`c_standard`、`linkage` 等） | 成员**声明了该键**时成员胜出；否则取工作空间的值 | -| 向量（`cxxflags`、`ldflags`、`defines`、`dialect_cxxflags`、`include_dirs` 等） | 追加，**工作空间在前**——因而成员自己的标志排在命令行更后面，后者胜出 | +| 向量（`cxxflags`、`cflags`、`ldflags`、`dialect_cxxflags`、`include_dirs` 等） | 追加，**工作空间在前** | +| `defines` | 按宏名构成的集合：成员对某个继承来的宏名写出的条目替换它，`!NAME` 移除它（2026.9.25.1+） | | `[workspace.dependencies]` | 逐依赖显式选择加入，`x.workspace = true`（§3） | "声明了"指的是这个键被写过，而不是它的值与默认值不同。成员在 @@ -211,6 +212,31 @@ c++23；什么都不写的成员得到 c++26。这两种情况值相同而意图 显式选择加入，因为依赖是解析图上的一条边：隐式继承一条边，会在成员自己的 manifest 只字未提的情况下改变它解析到什么。 +**追加的向量能覆盖什么。** 成员的词在命令行上排在工作空间的词之后。编译器按 +"后者胜出"处理的标志因此可以通过重写来覆盖：`-fno-exceptions` 之后的 +`-fexceptions`、`-Wx` 之后的 `-Wno-x`、`-O0` 之后的 `-O2`。头文件目录按顺序搜索， +所以工作空间 `include_dirs` 目录中的头文件，先于成员目录中的同名头文件被找到。 +宏通过 `defines` 覆盖，每个宏名只产生一个 `-DNAME` 词： + +```toml +# 工作空间根 +[workspace.build] +defines = ["LOG_LEVEL=1", "TRACE"] + +# 成员 +[build] +defines = ["LOG_LEVEL=3", "!TRACE"] # 以 -DLOG_LEVEL=3 编译，且不定义 TRACE +``` + +`defines` 条目同样替换同一个包的 `cflags` 或 `cxxflags` 中同名的 `-DNAME` 词。 +`!NAME` 要求 mcpp 2026.9.25.1 或更新版本；更早的 mcpp 会把它作为 `-D!NAME` 交给 +编译器，那是一个错误。 + +**每个成员在每种位置上都恰好接收一次继承值**（2026.9.25.1+）：作为命令所构建的包 +（`-p `，或在成员目录内执行的命令）、作为另一个成员的 `path` 依赖，以及作为 +通过 `git` 引用的、托管在 git 上的工作空间的成员（§6）。作为依赖出现的成员同样会解析 +它自己的 `x.workspace = true` 条目。 + **成员可以省略 `version`**，只要 `[workspace.package]` 提供了它。这个字段整体上 仍是必需的——两边都没有时会被拒绝，同时指出成员文件和本该提供它的那个 workspace 键。 @@ -363,6 +389,9 @@ myproject/ `myproject.http = { git = "...", rev = "..." }` 会在根 manifest 的 `members` 中选中 `libs/http`，取同一个提交，而该成员会像在工作空间内部一样继承 `[workspace.package]`（mcpp 2026.9.16.1+；见 [05 —— 依赖](05-dependencies.md)）。 +它还继承所在仓库的 `[workspace.build]`，并按该仓库的 `[workspace.dependencies]` +解析自己的 `x.workspace = true` 条目（2026.9.25.1+），因此同一个提交在它自己的 +检出中与在使用方的依赖图中以相同方式编译。 ## 7. 与 C++ 模块的关系 diff --git a/tests/e2e/770_workspace_member_as_dependency.sh b/tests/e2e/770_workspace_member_as_dependency.sh index c1d58f61..7f1f040b 100644 --- a/tests/e2e/770_workspace_member_as_dependency.sh +++ b/tests/e2e/770_workspace_member_as_dependency.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# requires: gcc python3 +# requires: python3 # 770 -- a workspace member is compiled the same way whichever position it # holds in the graph, and however it was fetched (#690). # From 5e76f33e706f8e48a282c87f903849632a261326 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:40:24 +0800 Subject: [PATCH 05/15] docs: noun-phrase heading for the defines and include scope section --- docs/04-mcpp-toml.md | 2 +- docs/zh/04-mcpp-toml.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index c30e0421..f084455e 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -462,7 +462,7 @@ element whose words differ from the arguments a release before 2026.9.17.1 passed on the same host, and names both. A build that repeats the plan does not repeat the warning. -#### What reaches which unit: `defines` and include directories *(mcpp 2026.9.25.1+)* +#### The scope of `defines` and include directories *(mcpp 2026.9.25.1+)* `defines` is a set keyed by macro name. Its entries are read in the order the package receives them: `[workspace.build]` (for a workspace member), the diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 556db48b..dc820266 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -462,7 +462,7 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | 同一宿主上传出的参数不同，mcpp 会在 `build/flag-words` 下发出警告，并 点名两者。重复同一份 plan 的构建不会重复这条警告。 -#### 什么到达哪个编译单元：`defines` 与头文件目录 *(mcpp 2026.9.25.1+)* +#### `defines` 与头文件目录的作用范围 *(mcpp 2026.9.25.1+)* `defines` 是按宏名构成的集合。条目按包接收它们的顺序读取：`[workspace.build]` （对工作空间成员而言）、包自己的 `[build]`，然后是每个命中的 From 45f266c002558c31c04530b828d8fbaa75b28165 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:43:57 +0800 Subject: [PATCH 06/15] docs: #690 sandbox verification script --- .agents/docs/2026-09-25-issue-690-verify.sh | 163 ++++++++++++++++++++ 1 file changed, 163 insertions(+) create mode 100755 .agents/docs/2026-09-25-issue-690-verify.sh diff --git a/.agents/docs/2026-09-25-issue-690-verify.sh b/.agents/docs/2026-09-25-issue-690-verify.sh new file mode 100755 index 00000000..4324857e --- /dev/null +++ b/.agents/docs/2026-09-25-issue-690-verify.sh @@ -0,0 +1,163 @@ +#!/usr/bin/env bash +# Sandbox verification of mcpp 2026.9.25.1 (#690), run against the PUBLISHED +# release inside an xlings sandbox: +# +# B64=$(base64 -w0 .agents/docs/2026-09-25-issue-690-verify.sh) +# xlings subos new v690 2>/dev/null || true +# xlings subos use v690 --sandbox --cmd "echo $B64 | base64 -d > /tmp/v.sh && VER=2026.9.25.1 bash /tmp/v.sh" +# +# VER selects the release under test. Running it with VER=2026.9.24.1 is the +# control: every section marked CHANGE must fail there, and every other section +# must pass on both. +# +# Every probe directory is removed at the start of its section, because the +# sandbox's $HOME persists between runs of the same subos. A section that does +# not run is reported as SKIP and counted separately from a pass. +set -u +VER="${VER:-2026.9.25.1}" +W="$HOME/v690" +fails=0; passes=0; skips=0 +pass() { echo "PASS $1"; passes=$((passes+1)); } +fail() { echo "FAIL $1"; [ -n "${2:-}" ] && [ -f "$2" ] && tail -15 "$2"; fails=$((fails+1)); } +skip() { echo "SKIP $1"; skips=$((skips+1)); } + +# ── 0. the release under test, from the published channel ────────────────── +# MCPP_OVERRIDE runs the script against a local binary, for debugging the +# script itself; the verification proper always installs from the index. +if [ -n "${MCPP_OVERRIDE:-}" ]; then + MCPP="$MCPP_OVERRIDE" +else + xlings config --mirror CN >/dev/null 2>&1 || true + xlings update >/dev/null 2>&1 || true + xlings install "mcpp@$VER" -y > /tmp/v690-install.log 2>&1 || true + MCPP="$HOME/.xlings/data/xpkgs/xim-x-mcpp/$VER/bin/mcpp" +fi +if [ ! -x "$MCPP" ]; then + echo "FATAL: mcpp $VER is not installable from the index"; tail -20 /tmp/v690-install.log; exit 2 +fi +got=$("$MCPP" --version 2>&1 | head -1) +case "$got" in *"$VER"*) pass "0 installed: $got";; *) fail "0 version: $got";; esac +"$MCPP" self config --mirror CN >/dev/null 2>&1 || true + +count() { # count + python3 - "$1" "$2" "$3" <<'EOF' +import json, sys +cdb, suffix, word = sys.argv[1:4] +for e in json.load(open(cdb)): + if e["file"].replace("\\", "/").endswith(suffix): + print(e["arguments"].count(word)); sys.exit(0) +print("missing") +EOF +} + +# ── 1. CHANGE: #690 as reported: workspace defines reach a sibling ───────── +rm -rf "$W/s1"; mkdir -p "$W/s1/lib" "$W/s1/app/src"; cd "$W/s1" +printf '[workspace]\nmembers = ["lib", "app"]\n\n[workspace.package]\nversion = "0.1.0"\n\n[workspace.build]\ndefines = ["WORKSPACE_DEFINE=1"]\n' > mcpp.toml +printf '[package]\nnamespace = "probe"\nname = "lib"\n\n[targets.probe_lib]\nkind = "lib"\n\n[build]\nsources = ["lib.cpp"]\n' > lib/mcpp.toml +printf '#ifndef WORKSPACE_DEFINE\n#error WORKSPACE_DEFINE missing in library dependency\n#endif\nint probe_lib() { return WORKSPACE_DEFINE; }\n' > lib/lib.cpp +printf '[package]\nnamespace = "probe"\nname = "app"\n\n[dependencies]\n"probe.lib" = { path = "../lib" }\n\n[targets.probe_app]\nkind = "bin"\nmain = "src/main.cpp"\n' > app/mcpp.toml +printf 'int probe_lib();\nint main() { return probe_lib() != 1; }\n' > app/src/main.cpp +if "$MCPP" build -p app > s1.log 2>&1 && [ "$(count app/compile_commands.json lib/lib.cpp -DWORKSPACE_DEFINE=1)" = 1 ]; then + pass "1 CHANGE: the sibling receives the workspace define exactly once" +else fail "1 CHANGE: sibling position" s1.log; fi +if "$MCPP" build -p lib > s1r.log 2>&1 && [ "$(count lib/compile_commands.json lib/lib.cpp -DWORKSPACE_DEFINE=1)" = 1 ]; then + pass "1 the root position receives it exactly once" +else fail "1 root position" s1r.log; fi + +# ── 2. CHANGE: keyed defines and a sibling's workspace dependency ────────── +rm -rf "$W/s2"; mkdir -p "$W/s2/lib" "$W/s2/app/src" "$W/s2/util"; cd "$W/s2" +printf '[workspace]\nmembers = ["lib", "app", "util"]\n\n[workspace.package]\nversion = "0.1.0"\n\n[workspace.dependencies]\nutil = { path = "util" }\n\n[workspace.build]\ndefines = ["LEVEL=1", "TRACE"]\n' > mcpp.toml +printf '[package]\nname = "util"\n\n[targets.util]\nkind = "lib"\n\n[build]\nsources = ["u.cpp"]\n' > util/mcpp.toml +echo 'int util_v() { return 5; }' > util/u.cpp +printf '[package]\nname = "lib"\n\n[dependencies]\nutil.workspace = true\n\n[targets.lib]\nkind = "lib"\n\n[build]\nsources = ["l.cpp"]\ndefines = ["LEVEL=2", "!TRACE"]\n' > lib/mcpp.toml +printf '#if LEVEL != 2\n#error LEVEL\n#endif\n#ifdef TRACE\n#error TRACE\n#endif\nint util_v();\nint lib_v() { return util_v() + LEVEL; }\n' > lib/l.cpp +printf '[package]\nname = "app"\n\n[dependencies]\nlib = { path = "../lib" }\n\n[targets.app]\nkind = "bin"\nmain = "src/main.cpp"\n' > app/mcpp.toml +printf 'int lib_v();\nint main() { return lib_v() == 7 ? 0 : 1; }\n' > app/src/main.cpp +if "$MCPP" run -p app > s2.log 2>&1 && [ "$(count app/compile_commands.json lib/l.cpp -DLEVEL=1)" = 0 ]; then + pass "2 CHANGE: LEVEL replaced, TRACE removed, util.workspace resolved for a sibling" +else fail "2 CHANGE: keyed defines / sibling workspace dependency" s2.log; fi + +# ── 3. CHANGE: a git-hosted member receives its repository's [workspace.build] +rm -rf "$W/s3"; mkdir -p "$W/s3/repo/glib" "$W/s3/consumer/src"; cd "$W/s3/repo" +printf '[workspace]\nmembers = ["glib"]\n\n[workspace.package]\nversion = "0.2.0"\n\n[workspace.build]\ncxxflags = ["-DREPO_FLAG=1"]\n' > mcpp.toml +printf '[package]\nnamespace = "v690"\nname = "glib"\n\n[targets.glib]\nkind = "lib"\n\n[build]\nsources = ["g.cpp"]\n' > glib/mcpp.toml +printf '#ifndef REPO_FLAG\n#error REPO_FLAG\n#endif\nint g_v() { return REPO_FLAG; }\n' > glib/g.cpp +if command -v git >/dev/null 2>&1; then + git init -q -b main . && git add -A && git -c user.email=v@v -c user.name=v commit -qm init + cd "$W/s3/consumer" + printf '[package]\nname = "consumer"\nversion = "0.1.0"\n\n[dependencies]\n"v690.glib" = { git = "file://%s/s3/repo", branch = "main" }\n\n[targets.consumer]\nkind = "bin"\nmain = "src/main.cpp"\n' "$W" > mcpp.toml + printf 'int g_v();\nint main() { return g_v() == 1 ? 0 : 1; }\n' > src/main.cpp + if "$MCPP" run > s3.log 2>&1; then pass "3 CHANGE: git-consumed member compiled with REPO_FLAG" + else fail "3 CHANGE: git-hosted member" s3.log; fi +else skip "3 git is not available in the sandbox"; fi + +# ── 4. CHANGE: a consumer's private header does not reach a dependency ───── +rm -rf "$W/s4"; mkdir -p "$W/s4/app/src" "$W/s4/app/priv" "$W/s4/dep/inc"; cd "$W/s4" +printf '#error ROOT PRIVATE HEADER REACHED A DEPENDENCY\n' > app/priv/limits.h +echo '#define DEP_HDR 3' > dep/inc/dep.h +printf '[package]\nname = "dep"\nversion = "0.1.0"\n\n[targets.dep]\nkind = "lib"\n\n[build]\ninclude_dirs = ["inc"]\nsources = ["d.c"]\n' > dep/mcpp.toml +printf '#include \n#include \nint dep_v(void) { return DEP_HDR + (INT_MAX > 0); }\n' > dep/d.c +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[dependencies]\ndep = { path = "../dep" }\n\n[build]\ninclude_dirs = ["priv"]\nprivate_include_dirs = ["priv"]\n\n[targets.app]\nkind = "bin"\nmain = "src/main.cpp"\n' > app/mcpp.toml +printf 'extern "C" int dep_v(void);\nint main() { return dep_v() == 4 ? 0 : 1; }\n' > app/src/main.cpp +cd app +if "$MCPP" run > s4.log 2>&1; then pass "4 CHANGE: the dependency compiled against its own headers" +else fail "4 CHANGE: root private header reached the dependency" s4.log; fi + +# ── 5. CHANGE: the index dependency cache is not poisoned across projects ── +rm -rf "$W/s5"; mkdir -p "$W/s5/a/src" "$W/s5/a/priv" "$W/s5/b/src"; cd "$W/s5" +printf '#include_next \n#undef DBL_EPSILON\n#define DBL_EPSILON 0.5\n' > a/priv/float.h +main_src='#include \n#include \nint main() {\n cJSON* x = cJSON_CreateNumber(1.0); cJSON* y = cJSON_CreateNumber(1.2);\n std::printf("compare=%%d\\n", (int)cJSON_Compare(x, y, 1));\n cJSON_Delete(x); cJSON_Delete(y);\n}\n' +printf "$main_src" > a/src/main.cpp; printf "$main_src" > b/src/main.cpp +printf '[package]\nname = "a"\nversion = "0.1.0"\n\n[dependencies.compat]\ncjson = "1.7.19"\n\n[build]\ninclude_dirs = ["priv"]\nprivate_include_dirs = ["priv"]\n\n[targets.a]\nkind = "bin"\nmain = "src/main.cpp"\n' > a/mcpp.toml +printf '[package]\nname = "b"\nversion = "0.1.0"\n\n[dependencies.compat]\ncjson = "1.7.19"\n\n[targets.b]\nkind = "bin"\nmain = "src/main.cpp"\n' > b/mcpp.toml +# 5a compiles cJSON unconditionally (--cache off), so the reading does not +# depend on what the cache already holds. 5b then lets A populate the cache and +# reads B: in a fresh sandbox home the cache is empty, so A compiles cJSON and +# B is served A's object. +( cd a && "$MCPP" build --cache off > ../s5a0.log 2>&1 && "$MCPP" run --cache off > ../s5a.log 2>&1 ) +if grep -q "compare=0" s5a.log; then + pass "5a CHANGE: cJSON compiled against its own float.h, not the consumer's" +else fail "5a CHANGE: consumer header reached cJSON ($(grep -o 'compare=[0-9]' s5a.log))" s5a0.log; fi +( cd a && "$MCPP" run > ../s5ac.log 2>&1 ); ( cd b && "$MCPP" run > ../s5b.log 2>&1 ) +if grep -q "compare=0" s5b.log; then + pass "5b CHANGE: an unrelated project is not served a poisoned object" +else fail "5b CHANGE: B=$(grep -o 'compare=[0-9]' s5b.log)" s5b.log; fi + +# ── 6. CHANGE: the published form of a workspace member is self-contained ── +rm -rf "$W/s6"; mkdir -p "$W/s6/lib" "$W/s6/util"; cd "$W/s6" +if command -v git >/dev/null 2>&1; then + printf '[workspace]\nmembers = ["lib", "util"]\n\n[workspace.package]\nversion = "0.3.0"\nlicense = "MIT"\nrepo = "https://example.invalid/v690"\n\n[workspace.build]\ncxxflags = ["-DWS_FLAG=1"]\n' > mcpp.toml + printf '[package]\nnamespace = "v690"\nname = "util"\n\n[targets.util]\nkind = "lib"\n\n[build]\nsources = ["u.cpp"]\n' > util/mcpp.toml + echo 'int util_v() { return 2; }' > util/u.cpp + printf '[package]\nnamespace = "v690"\nname = "lib"\n\n[dependencies]\n"v690.util" = { path = "../util", version = "0.3.0" }\n\n[targets.lib]\nkind = "lib"\n\n[build]\nsources = ["lib.cpp"]\n' > lib/mcpp.toml + printf '#ifndef WS_FLAG\n#error WS_FLAG\n#endif\nint lib_v() { return WS_FLAG; }\n' > lib/lib.cpp + git init -q -b main . && git add -A && git -c user.email=v@v -c user.name=v commit -qm init + ( cd lib && "$MCPP" publish --dry-run --allow-dirty > ../s6.log 2>&1 ) + tarball=$(ls lib/target/dist/*.tar.gz 2>/dev/null | head -1) + if [ -n "$tarball" ]; then + toml=$(tar xzf "$tarball" -O "lib-0.3.0/mcpp.toml" 2>/dev/null) + if echo "$toml" | grep -q 'WS_FLAG' && echo "$toml" | grep -q '0.3.0' \ + && ! echo "$toml" | grep -q 'path' \ + && tar tzf "$tarball" | grep -q 'mcpp.toml.orig'; then + pass "6 CHANGE: the archive carries the inherited values and a version edge" + else fail "6 CHANGE: archived manifest is not normalised" s6.log; echo "$toml"; fi + else fail "6 CHANGE: publish --dry-run produced no archive" s6.log; fi +else skip "6 git is not available in the sandbox"; fi + +# ── 7. index packages still build (unchanged behaviour) ──────────────────── +for pkg in zlib fmt; do + rm -rf "$W/s7-$pkg"; mkdir -p "$W/s7-$pkg/src"; cd "$W/s7-$pkg" + case $pkg in + zlib) dep='zlib = "1.3.2"'; src='#include \nint main() { return zlibVersion()[0] == 49 ? 0 : 1; }\n';; + fmt) dep='fmt = "12.2.0"'; src='import std;\nimport fmt;\nint main() { return fmt::format("{}", 42) == std::string("42") ? 0 : 1; }\n';; + esac + table=compat; [ "$pkg" = fmt ] && table=fmtlib + printf '[package]\nname = "s7%s"\nversion = "0.1.0"\n\n[dependencies.%s]\n%s\n\n[targets.s7%s]\nkind = "bin"\nmain = "src/main.cpp"\n' "$pkg" "$table" "$dep" "$pkg" > mcpp.toml + printf "$src" > src/main.cpp + if "$MCPP" run > s7.log 2>&1; then pass "7 $table.$pkg builds and runs" + else fail "7 $table.$pkg" s7.log; fi +done + +echo +echo "summary: mcpp $VER passes=$passes fails=$fails skips=$skips" +[ "$fails" -eq 0 ] From e6155099a938d4487dc17cdb4fa9360a568f6ff2 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Fri, 25 Sep 2026 05:42:15 +0800 Subject: [PATCH 07/15] publish: the effective manifest, a normalised published form, and a reproducible archive (#690 W4, W5) --- modules/libs/src/toml.cppm | 186 +++++++ src/pack/route.cppm | 8 +- src/pm/publisher.cppm | 120 +++++ src/project.cppm | 43 ++ src/publish/normalize.cppm | 466 ++++++++++++++++++ src/publish/pipeline.cppm | 69 ++- src/toolchain/lifecycle.cppm | 7 +- ...70_a_published_member_is_self_contained.sh | 221 +++++++++ ...e_the_build_read_the_effective_manifest.sh | 90 ++++ tests/unit/test_toml_serialize.cpp | 147 ++++++ 10 files changed, 1346 insertions(+), 11 deletions(-) create mode 100644 src/publish/normalize.cppm create mode 100755 tests/e2e/770_a_published_member_is_self_contained.sh create mode 100755 tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh create mode 100644 tests/unit/test_toml_serialize.cpp diff --git a/modules/libs/src/toml.cppm b/modules/libs/src/toml.cppm index a5384c84..43511045 100644 --- a/modules/libs/src/toml.cppm +++ b/modules/libs/src/toml.cppm @@ -87,6 +87,15 @@ public: bool has_explicit_table(std::string_view path) const { return explicitTables_.contains(path); } + // The dotted paths written as `[header]` or `[[header]]`. `serialize` + // reads this set so that a table written as a header is written back as + // one, and a table created by a dotted key or an inline table is not. + const std::set>& explicit_tables() const { + return explicitTables_; + } + std::set>& explicit_tables() { + return explicitTables_; + } private: Table root_; @@ -99,6 +108,19 @@ std::expected parse_file(const std::filesystem::path& p); // Serialization helpers (for emitting mcpp.lock and xpkg generation) std::string escape_string(std::string_view raw); +// Write a document back as TOML text that `parse` reads into the same tree and +// the same set of explicit tables. +// +// THE EXPLICIT SET IS PART OF THE DOCUMENT'S MEANING, NOT ITS FORMATTING. The +// manifest reader asks `has_explicit_table` to decide whether a dependency +// subtable is a namespace (`[dependencies.acme]`) or a dotted selector, so a +// writer that turned an inline table into a header, or the reverse, would +// change what a re-read manifest declares. A table is therefore written as a +// `[header]` exactly when its path is explicit, as `[[header]]` elements when +// an array's path is explicit, and otherwise inline or through dotted keys. +// Comments and key order are not preserved: the tree carries neither. +std::string serialize(const Document& doc); + } // namespace mcpp::libs::toml // ===================================================================== @@ -592,6 +614,170 @@ std::expected parse_file(const std::filesystem::path& p) { return parse(ss.str()); } +namespace detail { + +inline bool is_bare_key(std::string_view k) { + if (k.empty()) return false; + for (char c : k) + if (!(std::isalnum(static_cast(c)) || c == '_' || c == '-')) + return false; + return true; +} + +inline std::string render_key(std::string_view k) { + return is_bare_key(k) ? std::string(k) : escape_string(k); +} + +inline std::string render_value(const Value& v); + +inline std::string render_inline_table(const Table& t) { + std::string out = "{"; + bool first = true; + for (auto const& [k, v] : t) { + out += first ? " " : ", "; + first = false; + out += render_key(k) + " = " + render_value(v); + } + out += first ? "}" : " }"; + return out; +} + +inline std::string render_value(const Value& v) { + switch (v.kind()) { + case Value::Kind::String: return escape_string(v.as_string()); + case Value::Kind::Int: return std::to_string(v.as_int()); + case Value::Kind::Bool: return v.as_bool() ? "true" : "false"; + case Value::Kind::Table: return render_inline_table(v.as_table()); + case Value::Kind::Array: { + std::string out = "["; + bool first = true; + for (auto const& e : v.as_array()) { + if (!first) out += ", "; + first = false; + out += render_value(e); + } + return out + "]"; + } + case Value::Kind::Null: break; + } + return "\"\""; +} + +inline std::string child_path(std::string_view parent, std::string_view key) { + return parent.empty() ? std::string(key) : std::format("{}.{}", parent, key); +} + +// The dotted path a header line spells, each segment rendered as a key. +inline std::string render_header(const std::vector& segs) { + std::string out; + for (auto const& s : segs) { + if (!out.empty()) out += '.'; + out += render_key(s); + } + return out; +} + +// Whether any table below `path` (inclusive) must be written as a header. +inline bool needs_header(const std::set>& ex, + const std::string& path, const Value& v) { + if (ex.contains(path)) return true; + if (!v.is_table()) return false; + for (auto const& [k, c] : v.as_table()) + if (needs_header(ex, child_path(path, k), c)) return true; + return false; +} + +struct Writer { + const std::set>& ex; + std::string out; + + struct Deferred { std::vector segs; const Value* v; bool aot; }; + + static std::string dotted(const std::vector& segs) { + std::string p; + for (auto const& s : segs) p = p.empty() ? s : p + "." + s; + return p; + } + + // Emit the body of the section whose table is `t` at dotted `path` + // (segments `segs`): its values, then dotted keys for implicit + // containers, then the headers of its explicit descendants. + void section(const Table& t, const std::string& path, + const std::vector& segs) { + std::vector later; + body(t, path, segs, {}, later); + flush(later); + } + + void flush(const std::vector& later) { + for (auto const& d : later) { + if (d.aot) { + for (auto const& e : d.v->as_array()) { + out += std::format("\n[[{}]]\n", render_header(d.segs)); + std::vector nested; + body(e.as_table(), dotted(d.segs), d.segs, {}, nested); + flush(nested); + } + } else { + out += std::format("\n[{}]\n", render_header(d.segs)); + section(d.v->as_table(), dotted(d.segs), d.segs); + } + } + } + + void body(const Table& t, const std::string& path, + const std::vector& segs, + const std::vector& dottedPrefix, + std::vector& later) { + auto prefix = [&](std::string_view k) { + std::string p; + for (auto const& s : dottedPrefix) p += render_key(s) + "."; + return p + render_key(k); + }; + // `package` first at the root: a manifest opens with the package it + // declares. Key order carries no meaning in TOML, so this is only + // the order a reader expects. + std::vector*> order; + for (auto const& e : t) order.push_back(&e); + if (path.empty()) + std::ranges::stable_partition(order, + [](auto const* e) { return e->first == "package"; }); + for (auto const* e : order) { + auto const& k = e->first; + auto const& v = e->second; + const auto cp = child_path(path, k); + auto childSegs = segs; + childSegs.push_back(k); + if (v.is_array() && ex.contains(cp)) { + later.push_back(Deferred{ childSegs, &v, true }); + continue; + } + if (!v.is_table() || !needs_header(ex, cp, v)) { + out += prefix(k) + " = " + render_value(v) + "\n"; + continue; + } + if (ex.contains(cp)) { + later.push_back(Deferred{ childSegs, &v, false }); + continue; + } + // An implicit container with an explicit descendant: its own + // values are written as dotted keys from this section, which + // keeps the container implicit. + auto nextPrefix = dottedPrefix; + nextPrefix.push_back(k); + body(v.as_table(), cp, childSegs, nextPrefix, later); + } + } +}; + +} // namespace detail + +std::string serialize(const Document& doc) { + detail::Writer w{ doc.explicit_tables(), {} }; + w.section(doc.root(), "", {}); + return w.out; +} + std::string escape_string(std::string_view raw) { std::string out = "\""; for (char c : raw) { diff --git a/src/pack/route.cppm b/src/pack/route.cppm index 1436f09c..6cb3810b 100644 --- a/src/pack/route.cppm +++ b/src/pack/route.cppm @@ -61,8 +61,12 @@ namespace mcpp::pack { std::expected route_pack_target(std::string_view requested) { auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) return std::unexpected("no mcpp.toml in current dir or parents"); - auto m = mcpp::manifest::load(*root / "mcpp.toml"); - if (!m) return std::unexpected(m.error().format()); + // The effective manifest (#690): a workspace member that leaves `version` + // to `[workspace.package]` is refused by a raw load before any routing + // question is asked. + auto eff = mcpp::project::load_effective_manifest(*root); + if (!eff) return std::unexpected(eff.error()); + auto m = std::optional(std::move(eff->manifest)); auto is_library = [](const mcpp::manifest::Target& t) { return t.kind == mcpp::manifest::Target::Library diff --git a/src/pm/publisher.cppm b/src/pm/publisher.cppm index 6f698e6c..f61e7c9a 100644 --- a/src/pm/publisher.cppm +++ b/src/pm/publisher.cppm @@ -65,6 +65,26 @@ std::string make_release_tarball(const std::filesystem::path& root, std::string_view version, const std::filesystem::path& output); +// A file whose archived content differs from the committed one, or that the +// commit does not contain. `path` is relative to the package root. +struct ArchiveOverlay { + std::filesystem::path path; + std::string content; +}; + +// The same archive with `overlays` applied, built from git objects so that it +// stays a function of the commit: the overlays are written as blobs into a +// temporary index read from HEAD, the index becomes a commit whose author, +// committer and dates are HEAD's, and that commit is archived exactly as +// `HEAD` is by the overload above. Two runs on one commit produce identical +// bytes, on every host, with no archiver other than git. With no overlays +// this is the overload above. +std::string make_release_tarball(const std::filesystem::path& root, + std::string_view name, + std::string_view version, + const std::filesystem::path& output, + const std::vector& overlays); + // Convenience: build a real ReleaseInfo for v0.0.3-style local publish // where all three platforms point at the same source tarball. Caller has // already produced the tarball + sha256 by other means. @@ -398,6 +418,106 @@ std::string make_release_tarball(const std::filesystem::path& root, return {}; } +std::string make_release_tarball(const std::filesystem::path& root, + std::string_view name, + std::string_view version, + const std::filesystem::path& output, + const std::vector& overlays) +{ + if (overlays.empty()) return make_release_tarball(root, name, version, output); + namespace proc = mcpp::platform::process; + using Env = std::vector>; + const std::string dir = root.string(); + auto trim = [](std::string s) { + while (!s.empty() && (s.back() == '\n' || s.back() == '\r')) s.pop_back(); + return s; + }; + auto git_value = [&](std::vector args, const Env& env = {}) + -> std::expected { + std::vector argv{ "git", "-C", dir }; + argv.insert(argv.end(), args.begin(), args.end()); + // Standard output only: a warning on standard error must not become + // part of an object name. A failure is rerun below with both streams + // so the message can say why. + auto r = proc::capture_stdout(argv, env); + if (r.exit_code != 0) { + r = proc::capture_exec(argv, env); + std::string shown; + for (auto const& a : args) shown += " " + a; + return std::unexpected(std::format("git{} failed (rc={}): {}", + shown, r.exit_code, trim(r.output))); + } + return trim(r.output); + }; + + std::error_code ec; + std::filesystem::create_directories(output.parent_path(), ec); + auto prefixInRepo = git_value({ "rev-parse", "--show-prefix" }); + if (!prefixInRepo) return prefixInRepo.error(); + // HEAD's identity and dates, so that the commit made below is the same + // object on every run. + auto who = git_value({ "log", "-1", "--date=raw", + "--format=%an%n%ae%n%ad%n%cn%n%ce%n%cd", "HEAD" }); + if (!who) return who.error(); + std::vector fields; + { + std::istringstream in(*who); + std::string line; + while (std::getline(in, line)) { + if (!line.empty() && line.back() == '\r') line.pop_back(); + fields.push_back(line); + } + } + if (fields.size() != 6) + return std::format("git log -1 HEAD returned {} fields, expected 6", fields.size()); + + const auto scratch = output.parent_path() + / std::format(".mcpp-publish-{}", std::hash{}(output.string())); + std::filesystem::remove_all(scratch, ec); + std::filesystem::create_directories(scratch, ec); + struct Cleanup { + std::filesystem::path p; + ~Cleanup() { std::error_code e; std::filesystem::remove_all(p, e); } + } cleanup{ scratch }; + const Env indexEnv{ { "GIT_INDEX_FILE", (scratch / "index").string() } }; + + if (auto r = git_value({ "read-tree", "HEAD" }, indexEnv); !r) return r.error(); + std::size_t n = 0; + for (auto const& o : overlays) { + auto file = scratch / std::format("blob-{}", n++); + { + std::ofstream os(file, std::ios::binary); + os << o.content; + if (!os) return std::format("cannot write '{}'", file.string()); + } + auto blob = git_value({ "hash-object", "-w", file.string() }); + if (!blob) return blob.error(); + auto entry = *prefixInRepo + o.path.generic_string(); + if (auto r = git_value({ "update-index", "--add", "--cacheinfo", + std::format("100644,{},{}", *blob, entry) }, indexEnv); !r) + return r.error(); + } + auto tree = git_value({ "write-tree" }, indexEnv); + if (!tree) return tree.error(); + const Env commitEnv{ + { "GIT_AUTHOR_NAME", fields[0] }, { "GIT_AUTHOR_EMAIL", fields[1] }, + { "GIT_AUTHOR_DATE", fields[2] }, { "GIT_COMMITTER_NAME", fields[3] }, + { "GIT_COMMITTER_EMAIL", fields[4] }, { "GIT_COMMITTER_DATE", fields[5] }, + }; + auto commit = git_value({ "commit-tree", *tree, "-p", "HEAD", "-m", + "mcpp publish: normalized manifest" }, commitEnv); + if (!commit) return commit.error(); + // Run from the package directory: `git archive` then archives that + // subtree only, as it does for HEAD in the overload above. + auto archived = git_value({ "archive", "--format=tar.gz", + std::format("--prefix={}-{}/", name, version), + "-o", output.string(), *commit }); + if (!archived) return archived.error(); + if (!std::filesystem::exists(output)) + return std::format("git archive exited 0 but no tarball at '{}'", output.string()); + return {}; +} + ReleaseInfo make_release_info(std::string_view version, std::string_view url, std::string_view sha256) diff --git a/src/project.cppm b/src/project.cppm index 8cfc4004..ec8b14fd 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -315,6 +315,49 @@ export std::optional workspace_inheritance_error( return std::nullopt; } +// THE EFFECTIVE MANIFEST OF A PROJECT DIRECTORY, FOR EVERY READER OUTSIDE +// `prepare_build`. +// +// `prepare_build` applies workspace inheritance where it loads the manifest a +// command names. Every other command that reads a project manifest (publish, +// pack routing, `emit xpkg`, `toolchain list`) used to call `manifest::load` +// directly and therefore saw the raw file: a member that omits `version` +// because `[workspace.package]` supplies it was refused, and a member without +// `[toolchain]` was reported against the global default while `mcpp build` in +// the same directory resolved the workspace's toolchain (#690, F5a and F6). +// +// The rule is the one `prepare_build` follows: a directory that its workspace +// lists as a member is loaded with `insideWorkspace` and receives +// `inherit_workspace_config` anchored at the workspace root, and the +// required-field check runs after inheritance. A directory that is not a +// member, including a workspace root that carries its own `[package]`, is +// loaded as written. +export struct EffectiveManifest { + mcpp::manifest::Manifest manifest; // after inheritance + std::optional workspace; // set when `member` + std::filesystem::path workspaceRoot; // empty unless `member` + bool member = false; +}; + +export std::expected +load_effective_manifest(const std::filesystem::path& dir) { + const auto manifestPath = dir / "mcpp.toml"; + const auto wsRoot = find_workspace_root(std::filesystem::absolute(dir)); + if (wsRoot.empty()) { + auto m = mcpp::manifest::load(manifestPath); + if (!m) return std::unexpected(m.error().format()); + return EffectiveManifest{ std::move(*m), std::nullopt, {}, false }; + } + auto m = mcpp::manifest::load(manifestPath, {.insideWorkspace = true}); + if (!m) return std::unexpected(m.error().format()); + auto ws = mcpp::manifest::load(wsRoot / "mcpp.toml"); + if (!ws) return std::unexpected(ws.error().format()); + inherit_workspace_config(*m, *ws, wsRoot); + if (auto bad = workspace_inheritance_error(*m, dir)) + return std::unexpected(*bad); + return EffectiveManifest{ std::move(*m), std::move(*ws), wsRoot, true }; +} + // Resolve which member directory a workspace command acts on, for the // single-member case. Shares the match rule (basename OR member path) with // prepare_build's member switch, so `build -p X` and `test -p X` agree. diff --git a/src/publish/normalize.cppm b/src/publish/normalize.cppm new file mode 100644 index 00000000..3cca029f --- /dev/null +++ b/src/publish/normalize.cppm @@ -0,0 +1,466 @@ +// mcpp.publish.normalize — the manifest a published archive carries. +// +// A workspace member's `mcpp.toml` is valid only inside its workspace: it may +// omit `version` and `license` because `[workspace.package]` supplies them, its +// compile flags may come from `[workspace.build]`, and it may reach a sibling +// through `path = "../util"`. The archive `mcpp publish` produces contains the +// member directory alone, and a consumer reads that file as written. Before +// #690 the archive therefore carried a manifest that no consumer could build: +// the inherited configuration was absent and the sibling edge pointed outside +// the archive (design record 2026-09-25, section 3.6). +// +// THE PUBLISHED MANIFEST IS NORMALISED, NOT REWRITTEN. The raw TOML tree of the +// member is the starting point, and only three kinds of edit are made to it: +// +// 1. Inherited values are written back where the member did not declare +// them. The values are read from the EFFECTIVE manifest that +// `load_effective_manifest` produced, so the inheritance rule is applied +// by the one function that applies it for a build and is not restated +// here. +// 2. A dependency written `workspace = true` receives the source the +// workspace resolved for it, read from the same effective manifest. +// 3. A `path` edge that leaves the package directory is replaced by its +// `version`, or refused when it has none. A `path` edge that stays inside +// the directory is part of the archive and is kept. +// +// Anything else in the file, including keys this version does not know, passes +// through unchanged. A manifest that needs none of the edits is published +// byte for byte as it is on disk. + +export module mcpp.publish.normalize; + +import std; +import mcpp.libs.toml; +import mcpp.manifest; +import mcpp.project; + +export namespace mcpp::publish { + +struct NormalizedManifest { + std::string text; // the manifest the archive carries + std::string original; // the file as it is on disk + bool changed = false; + // `text` read back by the manifest parser. Its dependency maps are the + // ones a consumer of the archive resolves, so the descriptor's `deps` are + // emitted from them. + mcpp::manifest::Manifest manifest; +}; + +std::expected +normalize_for_publish(const std::filesystem::path& packageDir, + const mcpp::project::EffectiveManifest& effective); + +} // namespace mcpp::publish + +namespace mcpp::publish { + +namespace { + +namespace t = mcpp::libs::toml; + +// `[workspace.build]` keys, each paired with the field of `BuildConfig` that +// carries it. Vectors of flags, vectors of directories and scalars are written +// back by three different rules, so the table records which rule applies. +struct StringVectorKey { + std::string_view key; + std::vector mcpp::manifest::BuildConfig::* field; +}; +struct PathVectorKey { + std::string_view key; + std::vector mcpp::manifest::BuildConfig::* field; +}; +struct ScalarKey { + std::string_view key; + std::string mcpp::manifest::BuildConfig::* field; +}; + +using BC = mcpp::manifest::BuildConfig; + +const StringVectorKey kStringVectors[] = { + {"cflags", &BC::cflags}, + {"cxxflags", &BC::cxxflags}, + {"ldflags", &BC::ldflags}, + {"defines", &BC::defines}, + {"dialect_cxxflags", &BC::dialectCxxflags}, +}; +const PathVectorKey kPathVectors[] = { + {"include_dirs", &BC::includeDirs}, + {"include_dirs_after", &BC::includeDirsAfter}, + {"private_include_dirs", &BC::privateIncludeDirs}, +}; +const ScalarKey kScalars[] = { + {"c_standard", &BC::cStandard}, + {"linkage", &BC::linkage}, + {"target", &BC::target}, + {"cxx_runtime", &BC::cxxRuntime}, + {"dependency_linkage", &BC::dependencyLinkage}, + {"macos_deployment_target", &BC::macosDeploymentTarget}, + {"ios_deployment_target", &BC::iosDeploymentTarget}, +}; + +t::Value string_array(const std::vector& v) { + t::Array a; + for (auto const& s : v) a.emplace_back(s); + return t::Value{std::move(a)}; +} + +// The table at `key` inside `parent`, created (and recorded as a header) when +// absent. Returns nullptr when the key holds something other than a table. +t::Table* table_at(t::Table& parent, const std::string& key, + const std::string& dottedPath, + std::set>& explicitTables) { + auto it = parent.find(key); + if (it == parent.end()) { + parent[key] = t::Value{t::Table{}}; + explicitTables.insert(dottedPath); + return &parent[key].as_table(); + } + if (!it->second.is_table()) return nullptr; + return &it->second.as_table(); +} + +// Same discriminator the manifest reader uses: a dependency table that names a +// source is a specification; any other table is a namespace or a selector. +bool names_a_source(const t::Table& sub) { + for (auto const& [k, v] : sub) + if (k == "path" || k == "version" || k == "git" || k == "workspace") + return true; + return false; +} + +std::string join(const std::vector& segs) { + std::string out; + for (auto const& s : segs) out += (out.empty() ? "" : ".") + s; + return out; +} + +// Is `target` inside `dir`? Both are compared lexically: the archive is built +// from the tracked tree, whose paths are what a consumer receives. +bool inside(const std::filesystem::path& dir, const std::filesystem::path& target) { + auto rel = target.lexically_normal().lexically_relative(dir.lexically_normal()); + if (rel.empty()) return false; + auto first = *rel.begin(); + return first != ".."; +} + +struct Walk { + const std::filesystem::path& packageDir; + const mcpp::project::EffectiveManifest& effective; + const t::Document& raw; + const std::filesystem::path manifestPath; + bool changed = false; + + // The map key the manifest reader assigns to the dependency at `segs` + // inside `section`. Obtained by letting the reader parse a document that + // declares only this dependency, so the key rules (namespace tables, + // dotted selectors, quoted legacy keys) are not restated here. + std::expected + reader_key(std::string_view section, const std::vector& segs, + const t::Value& leaf) const { + t::Table root; + root["package"] = t::Value{t::Table{ + {"name", t::Value{std::string("normalize")}}, + {"version", t::Value{std::string("0.0.0")}}}}; + std::set> ex{ "package", std::string(section) }; + // Build the pruned subtree from the leaf upward. + t::Value node = leaf; + for (std::size_t i = segs.size(); i-- > 1;) { + t::Table parent; + parent[segs[i]] = node; + node = t::Value{std::move(parent)}; + } + t::Table sectionTable; + sectionTable[segs[0]] = node; + root[std::string(section)] = t::Value{std::move(sectionTable)}; + std::string prefix(section); + for (std::size_t i = 0; i + 1 < segs.size(); ++i) { + prefix += "." + segs[i]; + if (raw.has_explicit_table(prefix)) ex.insert(prefix); + } + auto text = t::serialize(t::Document{std::move(root), std::move(ex)}); + auto m = mcpp::manifest::parse_string(text, manifestPath); + if (!m) return std::unexpected(m.error().format()); + const auto& map = section == "dependencies" ? m->dependencies + : section == "build-dependencies" ? m->buildDependencies + : m->devDependencies; + if (map.size() != 1) + return std::unexpected(std::format( + "internal error: the dependency at [{}] {} did not read back as " + "exactly one entry (please report)", section, join(segs))); + return map.begin()->first; + } + + const std::map& + effective_map(std::string_view section) const { + const auto& m = effective.manifest; + return section == "dependencies" ? m.dependencies + : section == "build-dependencies" ? m.buildDependencies + : m.devDependencies; + } + + // Replace `workspace = true` by the source the workspace resolved. + std::expected + resolve_workspace(t::Table& spec, std::string_view section, + const std::vector& segs, bool conditional) { + auto it = spec.find("workspace"); + if (it == spec.end() || !it->second.is_bool() || !it->second.as_bool()) + return {}; + if (conditional) + return std::unexpected(std::format( + "{}: [{}] '{}' inherits from [workspace.dependencies] inside a " + "conditional section, and a published manifest cannot carry " + "that inheritance. State its source (version, path or git) in " + "the member's manifest.", + manifestPath.string(), section, join(segs))); + if (!effective.member) return {}; + auto key = reader_key(section, segs, t::Value{spec}); + if (!key) return std::unexpected(key.error()); + const auto& map = effective_map(section); + auto found = map.find(*key); + if (found == map.end() || found->second.inheritWorkspace) + return std::unexpected(std::format( + "{}: [{}] '{}' is declared `workspace = true`, and " + "[workspace.dependencies] of '{}' does not name it.", + manifestPath.string(), section, join(segs), + (effective.workspaceRoot / "mcpp.toml").string())); + const auto& s = found->second; + for (auto k : {"workspace", "version", "path", "git", "rev", "tag", "branch"}) + spec.erase(std::string(k)); + if (!s.version.empty()) spec["version"] = t::Value{s.version}; + if (!s.path.empty()) spec["path"] = t::Value{s.path}; + if (!s.git.empty()) { + spec["git"] = t::Value{s.git}; + if (!s.gitRev.empty()) + spec[s.gitRefKind.empty() ? std::string("rev") : s.gitRefKind] = + t::Value{s.gitRev}; + } + changed = true; + return {}; + } + + // Apply the path rule. Returns true when the entry is to be removed. + std::expected + apply_path_rule(t::Table& spec, std::string_view section, + const std::vector& segs, bool dev) { + auto it = spec.find("path"); + if (it == spec.end() || !it->second.is_string()) return false; + std::filesystem::path p(it->second.as_string()); + auto target = p.is_absolute() ? p : packageDir / p; + if (inside(packageDir, target)) { + if (p.is_absolute()) { + // Only reachable through `workspace = true`: the workspace + // anchors the path, and an absolute path cannot be published. + spec["path"] = t::Value{target.lexically_normal() + .lexically_relative(packageDir.lexically_normal()).generic_string()}; + changed = true; + } + return false; + } + if (spec.contains("version")) { + spec.erase(std::string("path")); + changed = true; + return false; + } + if (dev) { + // A consumer never resolves a development dependency, and one + // that exists only as a path has no published form. + changed = true; + return true; + } + const auto name = join(segs); + if (effective.member && effective.workspace + && mcpp::project::is_workspace_member(*effective.workspace, + effective.workspaceRoot, target)) { + std::string version = ""; + if (auto sib = mcpp::project::load_effective_manifest(target); + sib && !sib->manifest.package.version.empty()) + version = sib->manifest.package.version; + return std::unexpected(std::format( + "{}: [{}] '{}' reaches the workspace member '{}' by path, and " + "the published archive contains only this package's directory. " + "State the version the member is published under beside the " + "path:\n\n {} = {{ path = \"{}\", version = \"{}\" }}\n", + manifestPath.string(), section, name, + target.lexically_normal().string(), name, it->second.as_string(), + version)); + } + return std::unexpected(std::format( + "{}: [{}] '{}' is a path dependency outside this package ('{}'), " + "which a consumer of the published archive cannot reach. Depend on " + "a published version instead, or state `version` beside `path`.", + manifestPath.string(), section, name, it->second.as_string())); + } + + // Walk one dependency table. `segs` is the key path below the section. + std::expected + deps(t::Table& table, std::string_view section, std::vector segs, + bool conditional, bool dev) { + std::vector remove; + for (auto& [k, v] : table) { + auto here = segs; + here.push_back(k); + if (!v.is_table()) continue; // a version string + auto& sub = v.as_table(); + if (!names_a_source(sub)) { + if (auto r = deps(sub, section, here, conditional, dev); !r) return r; + continue; + } + if (auto r = resolve_workspace(sub, section, here, conditional); !r) + return std::unexpected(r.error()); + auto drop = apply_path_rule(sub, section, here, dev); + if (!drop) return std::unexpected(drop.error()); + if (*drop) remove.push_back(k); + } + for (auto const& k : remove) table.erase(k); + return {}; + } +}; + +} // namespace + +std::expected +normalize_for_publish(const std::filesystem::path& packageDir, + const mcpp::project::EffectiveManifest& effective) { + const auto manifestPath = packageDir / "mcpp.toml"; + NormalizedManifest out; + { + std::ifstream is(manifestPath, std::ios::binary); + if (!is) return std::unexpected(std::format("cannot open '{}'", manifestPath.string())); + std::stringstream ss; + ss << is.rdbuf(); + out.original = ss.str(); + } + auto doc = t::parse(out.original); + if (!doc) return std::unexpected(std::format("{}:{}:{}: {}", manifestPath.string(), + doc.error().where.line, doc.error().where.column, doc.error().message)); + auto& root = doc->root(); + auto& explicitTables = doc->explicit_tables(); + bool changed = false; + + if (effective.member && effective.workspace) { + const auto& ws = *effective.workspace; + const auto& inh = ws.workspace.inherited; + const auto& eff = effective.manifest; + + // [package]: the fields the member left to the workspace. + auto* pkg = table_at(root, "package", "package", explicitTables); + if (!pkg) return std::unexpected(std::format( + "{}: `package` is not a table", manifestPath.string())); + auto put_string = [&](std::string_view key, const std::string& value) { + if (value.empty() || pkg->contains(key)) return; + (*pkg)[std::string(key)] = t::Value{value}; + changed = true; + }; + put_string("version", eff.package.version); + put_string("license", eff.package.license); + put_string("description", eff.package.description); + put_string("repo", eff.package.repo); + if (!eff.package.authors.empty() && !pkg->contains("authors")) { + (*pkg)["authors"] = string_array(eff.package.authors); + changed = true; + } + // `standard` may also be declared under the deprecated `[language]`, + // so declaredness is read from the parser rather than from the tree. + auto rawManifest = mcpp::manifest::parse_string(out.original, manifestPath, + {.insideWorkspace = true}); + if (!rawManifest) return std::unexpected(rawManifest.error().format()); + if (!rawManifest->package.standardDeclared && inh.standardDeclared) { + (*pkg)["standard"] = t::Value{eff.package.standard}; + changed = true; + } + + // [build]: the keys of [workspace.build], with the member's own + // entries after the workspace's, as `inherit_workspace_build` ordered + // them. + if (inh.buildPresent) { + const auto& w = inh.build; + const auto& b = eff.buildConfig; + t::Table* build = nullptr; + auto build_table = [&]() -> t::Table* { + if (!build) build = table_at(root, "build", "build", explicitTables); + return build; + }; + for (auto const& k : kStringVectors) { + if ((w.*k.field).empty()) continue; + auto* bt = build_table(); + if (!bt) return std::unexpected(std::format( + "{}: `build` is not a table", manifestPath.string())); + (*bt)[std::string(k.key)] = string_array(b.*k.field); + changed = true; + } + for (auto const& k : kPathVectors) { + const auto n = (w.*k.field).size(); + if (n == 0) continue; + std::vector dirs; + const auto& all = b.*k.field; + for (std::size_t i = 0; i < all.size(); ++i) { + if (i >= n) { dirs.push_back(all[i].string()); continue; } + if (!inside(packageDir, all[i])) + return std::unexpected(std::format( + "{}: [workspace.build] {} entry '{}' resolves to '{}', " + "outside this package's directory, and the published " + "archive contains only that directory. Move the headers " + "into the package, or declare the directory in the " + "package's own [build] for its own build only.", + manifestPath.string(), k.key, (w.*k.field)[i].string(), + all[i].lexically_normal().string())); + dirs.push_back(all[i].lexically_normal() + .lexically_relative(packageDir.lexically_normal()).generic_string()); + } + auto* bt = build_table(); + if (!bt) return std::unexpected(std::format( + "{}: `build` is not a table", manifestPath.string())); + (*bt)[std::string(k.key)] = string_array(dirs); + changed = true; + } + for (auto const& k : kScalars) { + if ((w.*k.field).empty()) continue; + auto* bt = build_table(); + if (!bt) return std::unexpected(std::format( + "{}: `build` is not a table", manifestPath.string())); + if (bt->contains(k.key)) continue; + (*bt)[std::string(k.key)] = t::Value{b.*k.field}; + changed = true; + } + } + } + + // Dependencies: the unconditional sections and every conditional one. + Walk walk{ packageDir, effective, *doc, manifestPath }; + for (auto section : {"dependencies", "build-dependencies", "dev-dependencies"}) { + auto it = root.find(section); + if (it == root.end() || !it->second.is_table()) continue; + if (auto r = walk.deps(it->second.as_table(), section, {}, false, + std::string_view(section) == "dev-dependencies"); !r) + return std::unexpected(r.error()); + } + if (auto it = root.find("target"); it != root.end() && it->second.is_table()) { + for (auto& [selector, row] : it->second.as_table()) { + if (!row.is_table()) continue; + for (auto section : {"dependencies", "build-dependencies", "dev-dependencies"}) { + auto st = row.as_table().find(section); + if (st == row.as_table().end() || !st->second.is_table()) continue; + if (auto r = walk.deps(st->second.as_table(), section, {}, true, + std::string_view(section) == "dev-dependencies"); !r) + return std::unexpected(r.error()); + } + } + } + changed = changed || walk.changed; + + out.changed = changed; + out.text = changed + ? "# Normalised by `mcpp publish` from mcpp.toml.orig: the values this\n" + "# package inherited from its workspace are written out, and every\n" + "# dependency names a source a consumer can reach.\n" + + t::serialize(*doc) + : out.original; + auto reread = mcpp::manifest::parse_string(out.text, manifestPath); + if (!reread) return std::unexpected(std::format( + "internal error: the normalised manifest of '{}' does not parse " + "(please report): {}", manifestPath.string(), reread.error().format())); + out.manifest = std::move(*reread); + return out; +} + +} // namespace mcpp::publish diff --git a/src/publish/pipeline.cppm b/src/publish/pipeline.cppm index d1f7ca46..ee177dc3 100644 --- a/src/publish/pipeline.cppm +++ b/src/publish/pipeline.cppm @@ -1,6 +1,7 @@ // mcpp.publish.pipeline — the publish pipeline (tarball + sha256 + xpkg.lua + // next-step instructions) and xpkg emission to file/stdout. -// Bodies moved verbatim from the CLI layer. Zero behavior change. +// Both read the effective manifest of the project and emit from the +// normalised one (#690, mcpp.publish.normalize). module; #include @@ -13,11 +14,32 @@ import mcpp.manifest; import mcpp.modgraph.scanner; import mcpp.platform; import mcpp.project; +import mcpp.publish.normalize; import mcpp.publish.xpkg_emit; import mcpp.ui; namespace mcpp::publish { +namespace { + +// The manifest a descriptor is generated from: the effective manifest (what +// the package is), with the dependency maps of the normalised manifest (what a +// consumer of the published archive resolves). A sibling reached by +// `path` plus `version` is a version dependency there, so the descriptor lists +// it instead of dropping it with every other path edge. +mcpp::manifest::Manifest descriptor_manifest(const mcpp::project::EffectiveManifest& eff, + const NormalizedManifest& normalized) { + auto m = eff.manifest; + if (normalized.changed) { + m.dependencies = normalized.manifest.dependencies; + m.buildDependencies = normalized.manifest.buildDependencies; + m.devDependencies = normalized.manifest.devDependencies; + } + return m; +} + +} // namespace + // `mcpp emit xpkg [-V version] [-o output] [--namespace NS]`. export int emit_xpkg_to(std::string version, const std::filesystem::path& output, std::string namespaceOverride = {}) { @@ -26,8 +48,12 @@ export int emit_xpkg_to(std::string version, const std::filesystem::path& output std::println(stderr, "error: no mcpp.toml found"); return 2; } - auto m = mcpp::manifest::load(*root / "mcpp.toml"); - if (!m) { std::println(stderr, "error: {}", m.error().format()); return 2; } + // The effective manifest (#690): a workspace member reads as it builds. + auto eff = mcpp::project::load_effective_manifest(*root); + if (!eff) { std::println(stderr, "error: {}", eff.error()); return 2; } + auto normalized = normalize_for_publish(*root, *eff); + if (!normalized) { std::println(stderr, "error: {}", normalized.error()); return 2; } + auto m = std::optional(descriptor_manifest(*eff, *normalized)); auto scan = mcpp::modgraph::scan_package(*root, *m); if (!scan.errors.empty()) { for (auto& e : scan.errors) std::println(stderr, "error: {}", e.format()); @@ -83,11 +109,20 @@ export int publish_package(bool dry_run, bool allow_dirty) { } } - auto m = mcpp::manifest::load(*root / "mcpp.toml"); - if (!m) { - mcpp::ui::error(std::format("manifest parse: {}", m.error().format())); + // The effective manifest (#690), and the normalised form the archive + // carries: a workspace member's own file is valid only inside its + // workspace, and the archive contains the member alone. + auto eff = mcpp::project::load_effective_manifest(*root); + if (!eff) { + mcpp::ui::error(std::format("manifest parse: {}", eff.error())); + return 2; + } + auto normalized = normalize_for_publish(*root, *eff); + if (!normalized) { + mcpp::ui::error(normalized.error()); return 2; } + auto m = std::optional(descriptor_manifest(*eff, *normalized)); auto scan = mcpp::modgraph::scan_package(*root, *m); if (!scan.errors.empty()) { for (auto& e : scan.errors) mcpp::ui::error(e.format()); @@ -105,9 +140,27 @@ export int publish_package(bool dry_run, bool allow_dirty) { auto tarball = distDir / std::format("{}-{}.tar.gz", pkg.name, pkg.version); auto xpkgPath = distDir / std::format("{}.lua", pkg.name); - // 1. Pack source via `git archive` (respects .gitignore). + // 1. Pack source via `git archive` (respects .gitignore). A normalised + // manifest replaces `mcpp.toml` and the file as written is kept beside + // it as `mcpp.toml.orig`; an unchanged one leaves the archive exactly as + // `git archive HEAD` produces it. + std::vector overlays; + if (normalized->changed) { + overlays.push_back({ "mcpp.toml", normalized->text }); + overlays.push_back({ "mcpp.toml.orig", normalized->original }); + auto manifestCopy = distDir / std::format("{}-{}.mcpp.toml", pkg.name, pkg.version); + std::ofstream os(manifestCopy, std::ios::binary); + os << normalized->text; + if (!os) { + mcpp::ui::error(std::format("cannot write '{}'", manifestCopy.string())); + return 1; + } + mcpp::ui::status("Manifest", + std::format("{} (normalised; the archive carries it as mcpp.toml)", + manifestCopy.string())); + } if (auto err = mcpp::publish::make_release_tarball( - *root, pkg.name, pkg.version, tarball); + *root, pkg.name, pkg.version, tarball, overlays); !err.empty()) { mcpp::ui::error(std::format("tarball: {}", err)); diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index 29b77250..dd6091b0 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -15,6 +15,7 @@ import mcpp.fetcher.progress; import mcpp.manifest; import mcpp.platform; import mcpp.platform.axis; +import mcpp.project; import mcpp.toolchain.detect; import mcpp.toolchain.msvc; import mcpp.toolchain.registry; @@ -413,7 +414,11 @@ EffectiveDefault effective_default_toolchain(const mcpp::config::GlobalConfig& c std::error_code ec; auto mpath = std::filesystem::current_path(ec) / "mcpp.toml"; if (!ec && std::filesystem::exists(mpath, ec)) { - if (auto m = mcpp::manifest::load(mpath)) { + // The effective manifest (#690): a workspace member without its own + // `[toolchain]` builds with the workspace's, and the listing must name + // the toolchain the build resolves rather than the global default. + if (auto e = mcpp::project::load_effective_manifest(mpath.parent_path())) { + auto const* m = &e->manifest; if (auto t = m->toolchain.for_platform(mcpp::platform::name); t && !t->empty()) return { *t, true }; diff --git a/tests/e2e/770_a_published_member_is_self_contained.sh b/tests/e2e/770_a_published_member_is_self_contained.sh new file mode 100755 index 00000000..fceec873 --- /dev/null +++ b/tests/e2e/770_a_published_member_is_self_contained.sh @@ -0,0 +1,221 @@ +#!/usr/bin/env bash +# requires: gcc +# 770_a_published_member_is_self_contained.sh — the published form of a +# workspace member (#690, design 2026-09-25 sections 3.6 and 5.5). +# +# A member's own mcpp.toml is valid only inside its workspace: it omits the +# `version` and `license` that `[workspace.package]` supplies, its flags come +# partly from `[workspace.build]`, and it reaches a sibling by `path`. The +# archive `mcpp publish` produces contains the member directory alone, and a +# consumer reads the manifest in it as written. Before #690 that manifest was +# the raw file, so a consumer received a package that did not build. +# +# THE CRITERIA READ THE ARCHIVE AND BUILD FROM IT. `publish` exiting 0 says +# nothing: before the fix it also exited 0 while producing the broken archive. +# +# A. the archived mcpp.toml carries the inherited version, license and +# cxxflags; mcpp.toml.orig is the file as written +# B. a consumer of the unpacked archive builds past an #error guard that +# needs the workspace flag, with this mcpp and with a released one +# C. two runs produce byte-identical archives +# D. a sibling edge with `version` is published as a version edge, and the +# descriptor lists it +# E. a sibling edge without `version` is refused, naming the version to add +# F. `emit xpkg` works in a member that omits `version` +# G. a package outside any workspace is archived exactly as `git archive` +# archives it +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +fail() { echo "FAIL: $*"; exit 1; } + +REPO="$TMP/repo" +mkdir -p "$REPO/util/src" "$REPO/lib/src" "$REPO/edgeless/src" +cd "$REPO" +cat > mcpp.toml <<'EOF' +[workspace] +members = ["util", "lib", "edgeless"] + +[workspace.package] +version = "0.3.0" +license = "MIT" +repo = "https://github.com/example/probe" + +[workspace.build] +cxxflags = ["-DWS_FLAG=1"] +EOF +cat > util/mcpp.toml <<'EOF' +[package] +namespace = "probe" +name = "util" + +[targets.probe_util] +kind = "lib" +EOF +cat > util/src/util.cppm <<'EOF' +module; +#ifndef WS_FLAG +#error "WS_FLAG missing: the published manifest lost [workspace.build]" +#endif +export module probe.util; +export int util_value() { return WS_FLAG + 41; } +EOF +cat > lib/mcpp.toml <<'EOF' +[package] +namespace = "probe" +name = "lib" + +[dependencies] +"probe.util" = { path = "../util", version = "0.3.0" } + +[targets.probe_lib] +kind = "lib" +EOF +cat > lib/src/lib.cppm <<'EOF' +export module probe.lib; +import probe.util; +export int lib_value() { return util_value(); } +EOF +cat > edgeless/mcpp.toml <<'EOF' +[package] +namespace = "probe" +name = "edgeless" + +[dependencies] +"probe.util" = { path = "../util" } + +[targets.probe_edgeless] +kind = "lib" +EOF +cat > edgeless/src/edgeless.cppm <<'EOF' +export module probe.edgeless; +export int edgeless_value() { return 1; } +EOF +git init -q -b main . && git add -A \ + && git -c user.email=e2e@example.invalid -c user.name=e2e commit -qm init + +# ── A. the archived manifest carries what the member inherited ─────────── +( cd util && "$MCPP" publish --dry-run --allow-dirty > "$TMP/pub1.log" 2>&1 ) \ + || { cat "$TMP/pub1.log"; fail "publish of a member that omits version failed"; } +ARCHIVE="$REPO/util/target/dist/util-0.3.0.tar.gz" +[[ -f "$ARCHIVE" ]] || { cat "$TMP/pub1.log"; fail "no archive at $ARCHIVE"; } +tar -tzf "$ARCHIVE" | grep -qx 'util-0.3.0/mcpp.toml.orig' \ + || { tar -tzf "$ARCHIVE"; fail "the archive does not keep the original manifest"; } +tar -xzf "$ARCHIVE" -O util-0.3.0/mcpp.toml > "$TMP/util.toml" +for want in 'version = "0.3.0"' 'license = "MIT"' 'cxxflags = \["-DWS_FLAG=1"\]'; do + grep -q "$want" "$TMP/util.toml" \ + || { cat "$TMP/util.toml"; fail "the archived mcpp.toml lacks: $want"; } +done +tar -xzf "$ARCHIVE" -O util-0.3.0/mcpp.toml.orig | cmp -s - util/mcpp.toml \ + || fail "mcpp.toml.orig is not the file as written" +echo "ok A: the archived manifest carries the inherited version, license and cxxflags" + +# ── C. the archive is a function of the commit ─────────────────────────── +sha1=$(sha256sum "$ARCHIVE" | cut -d' ' -f1) +rm -rf util/target +( cd util && "$MCPP" publish --dry-run --allow-dirty > "$TMP/pub2.log" 2>&1 ) \ + || { cat "$TMP/pub2.log"; fail "second publish failed"; } +sha2=$(sha256sum "$ARCHIVE" | cut -d' ' -f1) +[[ "$sha1" == "$sha2" ]] || fail "two runs produced different archives ($sha1 vs $sha2)" +echo "ok C: two runs produce the same archive ($sha1)" + +# ── B. a consumer builds from the archive ──────────────────────────────── +mkdir -p "$TMP/unpacked" "$TMP/app/src" +tar -xzf "$ARCHIVE" -C "$TMP/unpacked" +cat > "$TMP/app/mcpp.toml" < "$TMP/app/src/main.cpp" <<'EOF' +import std; +import probe.util; +int main() { std::println("value={}", util_value()); return util_value() == 42 ? 0 : 1; } +EOF +( cd "$TMP/app" && "$MCPP" run > "$TMP/new.log" 2>&1 ) \ + || { cat "$TMP/new.log"; fail "a consumer of the published archive does not build"; } +grep -q 'value=42' "$TMP/new.log" || { cat "$TMP/new.log"; fail "wrong value"; } +echo "ok B: a consumer of the published archive builds and runs" + +# The released client. Same guard as 252: a shim that answers --version with +# nothing is not an old client, and saying so is a note, not a verdict. +boot_ver="" +new_ver="$("$MCPP" --version 2>/dev/null || true)" +if [[ -n "${MCPP_BOOT:-}" && -x "${MCPP_BOOT}" ]]; then + boot_ver="$("$MCPP_BOOT" --version 2>/dev/null || true)" +fi +case "$boot_ver" in + mcpp\ [0-9]*) + if [[ "$boot_ver" != "$new_ver" ]]; then + rm -rf "$TMP/app/target" + ( cd "$TMP/app" && "$MCPP_BOOT" run > "$TMP/old.log" 2>&1 ) \ + || { cat "$TMP/old.log"; fail "$boot_ver cannot build the normalised archive"; } + grep -q 'value=42' "$TMP/old.log" || { cat "$TMP/old.log"; fail "old client: wrong value"; } + echo "ok B: the released client ($boot_ver) builds the normalised archive" + else + echo "SKIP B (released client): \$MCPP_BOOT is this binary" + fi ;; + *) echo "SKIP B (released client): \$MCPP_BOOT is unset or not a released mcpp" ;; +esac + +# ── D. a sibling edge with a version ───────────────────────────────────── +( cd lib && "$MCPP" publish --dry-run --allow-dirty > "$TMP/pub3.log" 2>&1 ) \ + || { cat "$TMP/pub3.log"; fail "publish of a member with a versioned sibling edge failed"; } +tar -xzf "$REPO/lib/target/dist/lib-0.3.0.tar.gz" -O lib-0.3.0/mcpp.toml > "$TMP/lib.toml" +grep -q '"probe.util" = { version = "0.3.0" }' "$TMP/lib.toml" \ + || { cat "$TMP/lib.toml"; fail "the sibling edge is not a version edge"; } +if grep -q 'path' "$TMP/lib.toml"; then cat "$TMP/lib.toml"; fail "a path survived"; fi +( cd lib && "$MCPP" emit xpkg > "$TMP/lib.lua" 2>&1 ) || { cat "$TMP/lib.lua"; fail "emit xpkg"; } +grep -q '\["probe.util"\] = "0.3.0"' "$TMP/lib.lua" \ + || { cat "$TMP/lib.lua"; fail "the descriptor does not list the sibling"; } +echo "ok D: the sibling edge is published as a version edge and listed in the descriptor" + +# ── E. a sibling edge without a version is refused ─────────────────────── +if ( cd edgeless && "$MCPP" publish --dry-run --allow-dirty > "$TMP/pub4.log" 2>&1 ); then + cat "$TMP/pub4.log"; fail "a path-only sibling edge was published" +fi +grep -q 'version = "0.3.0"' "$TMP/pub4.log" \ + || { cat "$TMP/pub4.log"; fail "the refusal does not name the version to add"; } +echo "ok E: a path-only sibling edge is refused with the line to write" + +# ── F. emit xpkg in a member that omits version ────────────────────────── +( cd util && "$MCPP" emit xpkg > "$TMP/util.lua" 2>&1 ) \ + || { cat "$TMP/util.lua"; fail "emit xpkg in a member that omits version"; } +grep -q "\['0.3.0'\]" "$TMP/util.lua" || { cat "$TMP/util.lua"; fail "descriptor version"; } +echo "ok F: emit xpkg reads the inherited version" + +# ── G. a package outside any workspace is archived as git archives it ──── +SOLO="$TMP/solo" +mkdir -p "$SOLO/src" +cat > "$SOLO/mcpp.toml" <<'EOF' +[package] +name = "solo" +version = "1.0.0" +repo = "https://github.com/example/solo" + +[targets.solo] +kind = "lib" +EOF +echo 'export module solo; export int solo_v() { return 1; }' > "$SOLO/src/solo.cppm" +( cd "$SOLO" && git init -q -b main . && git add -A \ + && git -c user.email=e2e@example.invalid -c user.name=e2e commit -qm init ) +( cd "$SOLO" && "$MCPP" publish --dry-run --allow-dirty > "$TMP/pub5.log" 2>&1 ) \ + || { cat "$TMP/pub5.log"; fail "publish outside a workspace"; } +git -C "$SOLO" archive --format=tar.gz --prefix=solo-1.0.0/ -o "$TMP/direct.tar.gz" HEAD +cmp -s "$TMP/direct.tar.gz" "$SOLO/target/dist/solo-1.0.0.tar.gz" \ + || fail "a package that needs no normalisation is not archived as git archive HEAD" +if grep -q 'Manifest' "$TMP/pub5.log"; then cat "$TMP/pub5.log"; fail "an unchanged manifest was reported as normalised"; fi +echo "ok G: an unchanged package is archived byte for byte as before" + +echo "PASS: 770_a_published_member_is_self_contained" diff --git a/tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh b/tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh new file mode 100755 index 00000000..20aa11ba --- /dev/null +++ b/tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# requires: gcc +# 771_commands_outside_the_build_read_the_effective_manifest.sh — #690, F5a/F6. +# +# `prepare_build` applies workspace inheritance to the manifest a command +# names. `toolchain list` and `pack` read the project manifest on their own, +# and before #690 they read the raw file: +# +# - `toolchain list` inside a member without `[toolchain]` marked the global +# default, while `mcpp build` in the same directory used the workspace's; +# - `pack` inside a member that leaves `version` to `[workspace.package]` +# failed with "missing required field 'package.version'" before any +# routing question was asked. +# +# Each check has a control at the workspace root or in a member that declares +# the value itself, so a reading that never depended on inheritance cannot +# pass as one that does. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +fail() { echo "FAIL: $*"; exit 1; } + +WS="$TMP/ws" +mkdir -p "$WS/app/src" "$WS/pinned/src" +cd "$WS" +cat > mcpp.toml <<'EOF' +[workspace] +members = ["app", "pinned"] + +[workspace.package] +version = "0.2.0" + +[toolchain] +default = "gcc@16.1.0" +EOF +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > app/src/main.cpp +# The control declares its own version as well, so that it holds for an +# implementation without the fix and isolates the one difference under test. +cat > pinned/mcpp.toml <<'EOF' +[package] +name = "pinned" +version = "0.2.0" + +[toolchain] +default = "gcc@16.1.0" + +[targets.pinned] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > pinned/src/main.cpp + +# ── toolchain list ──────────────────────────────────────────────────────── +LEGEND='effective toolchain from project mcpp.toml' +"$MCPP" toolchain list > "$TMP/root.log" 2>&1 || true +if grep -q 'no toolchains installed' "$TMP/root.log"; then + echo "SKIP toolchain list: no toolchain payload is visible in this home" +else + grep -q "$LEGEND" "$TMP/root.log" \ + || { cat "$TMP/root.log"; fail "control: the workspace root's [toolchain] is not reported"; } + ( cd pinned && "$MCPP" toolchain list > "$TMP/pinned.log" 2>&1 ) || true + grep -q "$LEGEND" "$TMP/pinned.log" \ + || { cat "$TMP/pinned.log"; fail "control: a member's own [toolchain] is not reported"; } + ( cd app && "$MCPP" toolchain list > "$TMP/app.log" 2>&1 ) || true + grep -q "$LEGEND" "$TMP/app.log" \ + || { cat "$TMP/app.log"; fail "a member without [toolchain] does not report the workspace's"; } + echo "ok: toolchain list in a member reports the workspace's [toolchain]" +fi + +# ── pack routing ────────────────────────────────────────────────────────── +( cd app && "$MCPP" pack > "$TMP/pack.log" 2>&1 ) || true +if grep -q "missing required field 'package.version'" "$TMP/pack.log"; then + cat "$TMP/pack.log" + fail "pack in a member that leaves version to the workspace was refused" +fi +echo "ok: pack in a member reads the inherited version" + +echo "PASS: 771_commands_outside_the_build_read_the_effective_manifest" diff --git a/tests/unit/test_toml_serialize.cpp b/tests/unit/test_toml_serialize.cpp new file mode 100644 index 00000000..704861ba --- /dev/null +++ b/tests/unit/test_toml_serialize.cpp @@ -0,0 +1,147 @@ +#include + +import std; +import mcpp.libs.toml; +import mcpp.manifest; + +// `serialize` is what `mcpp publish` writes a normalised manifest with (#690). +// The property it owes is a round trip: the text it produces parses into the +// same tree AND the same set of explicit tables, because the manifest reader +// uses the explicit set to tell a namespace table from a dotted selector. + +namespace t = mcpp::libs::toml; + +namespace { + +bool same(const t::Value& a, const t::Value& b) { + if (a.kind() != b.kind()) return false; + switch (a.kind()) { + case t::Value::Kind::String: return a.as_string() == b.as_string(); + case t::Value::Kind::Int: return a.as_int() == b.as_int(); + case t::Value::Kind::Bool: return a.as_bool() == b.as_bool(); + case t::Value::Kind::Null: return true; + case t::Value::Kind::Array: { + auto const& x = a.as_array(); + auto const& y = b.as_array(); + if (x.size() != y.size()) return false; + for (std::size_t i = 0; i < x.size(); ++i) + if (!same(x[i], y[i])) return false; + return true; + } + case t::Value::Kind::Table: { + auto const& x = a.as_table(); + auto const& y = b.as_table(); + if (x.size() != y.size()) return false; + for (auto const& [k, v] : x) { + auto it = y.find(k); + if (it == y.end() || !same(v, it->second)) return false; + } + return true; + } + } + return false; +} + +void expect_round_trip(std::string_view source, std::string_view label) { + auto first = t::parse(source); + ASSERT_TRUE(first.has_value()) << label << ": " << first.error().message; + auto text = t::serialize(*first); + auto second = t::parse(text); + ASSERT_TRUE(second.has_value()) << label << ": the serialised text does not parse: " + << second.error().message << "\n" << text; + EXPECT_TRUE(same(t::Value{first->root()}, t::Value{second->root()})) + << label << ": the tree changed\n" << text; + EXPECT_EQ(first->explicit_tables(), second->explicit_tables()) + << label << ": the explicit tables changed\n" << text; + + // The manifest reader sees the same dependencies. A namespace table that + // came back as a dotted selector, or the reverse, changes the keys here. + auto m1 = mcpp::manifest::parse_string(source); + auto m2 = mcpp::manifest::parse_string(text); + ASSERT_EQ(m1.has_value(), m2.has_value()) << label; + if (!m1) return; + auto keys = [](auto const& map) { + std::vector out; + for (auto const& [k, v] : map) + out.push_back(std::format("{}|{}|{}|{}", k, v.version, v.path, v.namespace_)); + return out; + }; + EXPECT_EQ(keys(m1->dependencies), keys(m2->dependencies)) << label << "\n" << text; + EXPECT_EQ(keys(m1->devDependencies), keys(m2->devDependencies)) << label; + EXPECT_EQ(keys(m1->buildDependencies), keys(m2->buildDependencies)) << label; + EXPECT_EQ(m1->package.name, m2->package.name) << label; + EXPECT_EQ(m1->package.version, m2->package.version) << label; +} + +} // namespace + +TEST(TomlSerialize, HeadersInlineTablesAndDottedKeys) { + expect_round_trip(R"( +[package] +name = "demo" +version = "1.2.3" +authors = ["a", "b \"quoted\""] + +[build] +cxxflags = ["-DX=1", "-O2"] +include_dirs = ["include"] + +[[build.flags]] +glob = "src/*.cpp" +cxxflags = ["-Wall"] + +[[build.flags]] +glob = "src/x.cpp" + +[dependencies] +mcpplibs.cmdline = "0.0.1" +"probe.util" = { path = "../util", version = "0.3.0" } +plain = "1.0" + +[dependencies.mcpp] +libs = { path = "modules/libs" } + +[target.'cfg(os = "linux")'.dependencies] +extra = { version = "2.0", features = ["a"] } + +[targets.demo] +kind = "bin" +main = "src/main.cpp" +)", "inline fixture"); +} + +TEST(TomlSerialize, ImplicitContainerWithExplicitDescendant) { + // `dependencies` is created by a dotted key at the root and also has an + // explicit descendant: its own entries must stay dotted keys. + expect_round_trip(R"( +dependencies.foo = "1.0" + +[dependencies.acme] +bar = "2.0" + +[package] +name = "p" +version = "0.1.0" +)", "implicit container"); +} + +TEST(TomlSerialize, RepositoryManifestsRoundTrip) { + // Every manifest in the repository's examples, and the project's own. The + // repository is located from this file's own path rather than from the + // working directory, and the count is asserted, so a run that found no + // file cannot pass as one that round-tripped them all. + auto here = std::filesystem::path(std::source_location::current().file_name()); + auto repo = here.parent_path().parent_path().parent_path(); + ASSERT_TRUE(std::filesystem::exists(repo / "mcpp.toml")) << repo.string(); + std::vector files{ repo / "mcpp.toml" }; + for (auto const& e : std::filesystem::recursive_directory_iterator(repo / "examples")) + if (e.is_regular_file() && e.path().filename() == "mcpp.toml") + files.push_back(e.path()); + EXPECT_GE(files.size(), 10u); + for (auto const& f : files) { + std::ifstream is(f, std::ios::binary); + std::stringstream ss; + ss << is.rdbuf(); + expect_round_trip(ss.str(), f.string()); + } +} From 000a29efc1038eb7baf71cb6c0f131cfa455296b Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:48:16 +0800 Subject: [PATCH 08/15] test: renumber the published-form e2e scripts to 772 and 773 --- ...ntained.sh => 772_a_published_member_is_self_contained.sh} | 4 ++-- ...commands_outside_the_build_read_the_effective_manifest.sh} | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) rename tests/e2e/{770_a_published_member_is_self_contained.sh => 772_a_published_member_is_self_contained.sh} (98%) rename tests/e2e/{771_commands_outside_the_build_read_the_effective_manifest.sh => 773_commands_outside_the_build_read_the_effective_manifest.sh} (96%) diff --git a/tests/e2e/770_a_published_member_is_self_contained.sh b/tests/e2e/772_a_published_member_is_self_contained.sh similarity index 98% rename from tests/e2e/770_a_published_member_is_self_contained.sh rename to tests/e2e/772_a_published_member_is_self_contained.sh index fceec873..14dd6aeb 100755 --- a/tests/e2e/770_a_published_member_is_self_contained.sh +++ b/tests/e2e/772_a_published_member_is_self_contained.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # requires: gcc -# 770_a_published_member_is_self_contained.sh — the published form of a +# 772_a_published_member_is_self_contained.sh — the published form of a # workspace member (#690, design 2026-09-25 sections 3.6 and 5.5). # # A member's own mcpp.toml is valid only inside its workspace: it omits the @@ -218,4 +218,4 @@ cmp -s "$TMP/direct.tar.gz" "$SOLO/target/dist/solo-1.0.0.tar.gz" \ if grep -q 'Manifest' "$TMP/pub5.log"; then cat "$TMP/pub5.log"; fail "an unchanged manifest was reported as normalised"; fi echo "ok G: an unchanged package is archived byte for byte as before" -echo "PASS: 770_a_published_member_is_self_contained" +echo "PASS: 772_a_published_member_is_self_contained" diff --git a/tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh b/tests/e2e/773_commands_outside_the_build_read_the_effective_manifest.sh similarity index 96% rename from tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh rename to tests/e2e/773_commands_outside_the_build_read_the_effective_manifest.sh index 20aa11ba..31231857 100755 --- a/tests/e2e/771_commands_outside_the_build_read_the_effective_manifest.sh +++ b/tests/e2e/773_commands_outside_the_build_read_the_effective_manifest.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # requires: gcc -# 771_commands_outside_the_build_read_the_effective_manifest.sh — #690, F5a/F6. +# 773_commands_outside_the_build_read_the_effective_manifest.sh — #690, F5a/F6. # # `prepare_build` applies workspace inheritance to the manifest a command # names. `toolchain list` and `pack` read the project manifest on their own, @@ -87,4 +87,4 @@ if grep -q "missing required field 'package.version'" "$TMP/pack.log"; then fi echo "ok: pack in a member reads the inherited version" -echo "PASS: 771_commands_outside_the_build_read_the_effective_manifest" +echo "PASS: 773_commands_outside_the_build_read_the_effective_manifest" From afd536f4a7a22afb42c72893096b1bfe9b438a41 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:52:54 +0800 Subject: [PATCH 09/15] fix: every reader of a member manifest reads the effective manifest; 2026.9.25.1 (#690 W4) prepare_build loads through load_effective_manifest and no longer inherits a preloaded host-tool manifest a second time. sbom, index list/update, the index refresh of mcpp update, the fast-path identity and test discovery read the effective manifest. --- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/execute.cppm | 7 ++- src/build/prepare.cppm | 79 ++++++++++++++++------------- src/build/test_targets.cppm | 6 ++- src/cli/cmd_sbom.cppm | 9 ++-- src/pm/commands.cppm | 6 ++- src/pm/index_management.cppm | 12 ++++- 8 files changed, 77 insertions(+), 46 deletions(-) diff --git a/mcpp.toml b/mcpp.toml index fd30b216..d34c38e3 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.24.1" +version = "2026.9.25.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 8fe9f5f9..e7dba7a6 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.24.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.25.1"; } // namespace mcpp diff --git a/src/build/execute.cppm b/src/build/execute.cppm index e12b1ade..a9f1edfb 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1328,8 +1328,11 @@ std::optional fast_path_identity(const std::filesystem::path& projectRoot, std::string_view profileOverride = "", std::string_view featuresRequested = "") { - auto m = mcpp::manifest::load(projectRoot / "mcpp.toml"); - if (!m) return std::nullopt; + // The effective manifest: `target` is inheritable from `[workspace.build]` + // (#690, W4). + auto effective = mcpp::project::load_effective_manifest(projectRoot); + if (!effective) return std::nullopt; + const auto* m = &effective->manifest; return FastPathIdentity{ mcpp::build::resolve_profile_name(*m, profileOverride), std::string(mcpp::build::cache_mode_name( diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index b2e41ae8..4aeed236 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -2317,22 +2317,30 @@ prepare_build(bool print_fingerprint, // two: re-deriving could produce a DIFFERENT manifest than the one the // parent resolved against (the L1 cfg merge and feature-activated deps // have already been folded in by then). - auto m = overrides.preloaded_manifest - ? std::expected( - *overrides.preloaded_manifest) - : mcpp::manifest::load(*root / "mcpp.toml"); - // A COMMAND ISSUED INSIDE A MEMBER DIRECTORY loads that member's manifest - // here, before anything knows a workspace is above it — so a member relying - // on `[workspace.package]` for a required field would be refused by the - // parser before inheritance could supply it. Retried, not reordered: the - // workspace lookup walks the tree reading manifests, and paying that on - // every build to serve the error path would be the wrong trade. The - // requirement still holds; it is enforced after inheritance, where "still - // missing" is knowable. - if (!m && !overrides.preloaded_manifest - && !mcpp::project::find_workspace_root(*root).empty()) - m = mcpp::manifest::load(*root / "mcpp.toml", {.insideWorkspace = true}); - if (!m) return std::unexpected(m.error().format()); + // THE EFFECTIVE MANIFEST, FROM THE ONE LOADER EVERY COMMAND USES. + // + // A command issued inside a member directory receives the member's + // manifest after workspace inheritance, exactly as `publish`, `pack`, + // `emit xpkg` and `toolchain list` do (#690, W4). A command at the + // workspace root receives the root manifest as written; the `-p ` + // switch below loads and inherits the member it names. + // + // A PRELOADED manifest (a host-tool sub-build) is already effective: the + // resolver loaded it at the dependency's load site, where a member + // inherits (see `inheritAsMember`). It is not inherited a second time; the + // workspace it belongs to is still recorded below, so that its own sibling + // dependencies inherit as members. + std::optional effective; + std::expected m = + std::unexpected(std::string{}); + if (overrides.preloaded_manifest) { + m = *overrides.preloaded_manifest; + } else { + auto loaded = mcpp::project::load_effective_manifest(*root); + if (!loaded) return std::unexpected(loaded.error()); + m = loaded->manifest; + effective = std::move(*loaded); + } // AND ONLY FOR THE ROOT. A layer name this engine does not know is a // typo in the manifest the author is looking at, and a version gap in a @@ -2424,10 +2432,12 @@ prepare_build(bool print_fingerprint, } runtimeWorkspaceRoot = *root; wsManifest = std::move(*m); // preserve workspace manifest - m = mcpp::manifest::load(memberDir / "mcpp.toml", - {.insideWorkspace = true}); - if (!m) return std::unexpected(std::format( - "workspace member '{}': {}", targetMember, m.error().format())); + auto memberManifest = mcpp::manifest::load(memberDir / "mcpp.toml", + {.insideWorkspace = true}); + if (!memberManifest) return std::unexpected(std::format( + "workspace member '{}': {}", targetMember, + memberManifest.error().format())); + m = std::move(*memberManifest); // ONE call, not a hand-copied list. `*root` is still the WORKSPACE // root here (the `root = memberDir` reassignment below has not @@ -2441,20 +2451,21 @@ prepare_build(bool print_fingerprint, root = memberDir; } } else { - // Not at workspace root — check if we're inside a workspace - auto wsRoot = mcpp::project::find_workspace_root(*root); - if (!wsRoot.empty()) { - auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); - if (wsm && wsm->workspace.present) { - runtimeWorkspaceRoot = wsRoot; - wsManifest = std::move(*wsm); - // The SECOND inheritance site, and it calls the same function - // as the first for that reason. #224: relative `path` and - // `[indices].path` anchor to the workspace root, not to this - // member's own directory. - mcpp::project::inherit_workspace_config(*m, *wsManifest, wsRoot); - if (auto bad = mcpp::project::workspace_inheritance_error(*m, *root)) - return std::unexpected(*bad); + // Not at workspace root: inside a member, the loader above has + // already inherited (#224 anchoring included). Only the workspace is + // recorded here, for the membership test of this member's own `path` + // dependencies. + if (effective && effective->member) { + runtimeWorkspaceRoot = effective->workspaceRoot; + wsManifest = std::move(*effective->workspace); + } else if (overrides.preloaded_manifest) { + auto wsRoot = mcpp::project::find_workspace_root(*root); + if (!wsRoot.empty()) { + if (auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); + wsm && wsm->workspace.present) { + runtimeWorkspaceRoot = wsRoot; + wsManifest = std::move(*wsm); + } } } } diff --git a/src/build/test_targets.cppm b/src/build/test_targets.cppm index 4d3de90e..b0808ed0 100644 --- a/src/build/test_targets.cppm +++ b/src/build/test_targets.cppm @@ -35,8 +35,10 @@ discover_test_targets(const std::filesystem::path& manifestRoot, if (!member) return std::unexpected(member.error()); // workspace member root is the only root accepted by prepare_build. if (!member->empty()) packageRoot = *member; - if (auto manifest = mcpp::manifest::load(packageRoot / "mcpp.toml")) - packageManifest = std::move(*manifest); + // The effective manifest: a member that omits `version` still loads + // instead of falling back to the default discovery (#690, W4). + if (auto manifest = mcpp::project::load_effective_manifest(packageRoot)) + packageManifest = std::move(manifest->manifest); } std::vector globFlags; diff --git a/src/cli/cmd_sbom.cppm b/src/cli/cmd_sbom.cppm index 7e452e1e..531f8bb9 100644 --- a/src/cli/cmd_sbom.cppm +++ b/src/cli/cmd_sbom.cppm @@ -91,11 +91,14 @@ int cmd_sbom(const mcpplibs::cmdline::ParsedArgs& parsed) { return 2; } - auto man = mcpp::manifest::load(*root / "mcpp.toml"); - if (!man) { - mcpp::ui::error(man.error().message); + // The effective manifest: a workspace member's `[package]` fields may come + // from `[workspace.package]` (#690, W4). + auto effective = mcpp::project::load_effective_manifest(*root); + if (!effective) { + mcpp::ui::error(effective.error()); return 2; } + std::optional man = std::move(effective->manifest); const auto lockPath = *root / "mcpp.lock"; std::vector pkgs; diff --git a/src/pm/commands.cppm b/src/pm/commands.cppm index 091233a1..bd06494f 100644 --- a/src/pm/commands.cppm +++ b/src/pm/commands.cppm @@ -569,7 +569,11 @@ inline int cmd_update(const mcpplibs::cmdline::ParsedArgs& parsed) { // the exact behaviour #315 is about. if (auto cfg = mcpp::config::load_or_init( /*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback())) { - auto m = mcpp::manifest::load(*root / "mcpp.toml"); + // The effective manifest: `x.workspace = true` entries are resolved, + // and a member that omits `version` still loads (#690, W4). + std::optional m; + if (auto effective = mcpp::project::load_effective_manifest(*root)) + m = std::move(effective->manifest); bool registryInvolved = false; if (m) { auto indices = mcpp::pm::effective_indices(*root); diff --git a/src/pm/index_management.cppm b/src/pm/index_management.cppm index d4586cfc..48889906 100644 --- a/src/pm/index_management.cppm +++ b/src/pm/index_management.cppm @@ -95,7 +95,11 @@ export int index_list() { // Show project-level custom indices from mcpp.toml [indices]. auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (root) { - auto m = mcpp::manifest::load(*root / "mcpp.toml"); + // The effective manifest: a member inherits the workspace's + // `[indices]` (#690, W4). + std::optional m; + if (auto effective = mcpp::project::load_effective_manifest(*root)) + m = std::move(effective->manifest); if (m && !m->indices.empty()) { std::println(""); std::println("Project indices (mcpp.toml):"); @@ -171,7 +175,11 @@ export int index_update(const std::string& filterName) { // Also update project-level custom indices if present. auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (root) { - auto m = mcpp::manifest::load(*root / "mcpp.toml"); + // The effective manifest: a member inherits the workspace's + // `[indices]` (#690, W4). + std::optional m; + if (auto effective = mcpp::project::load_effective_manifest(*root)) + m = std::move(effective->manifest); if (m && !m->indices.empty()) { // filterName: optional single-index filter (parameter). for (auto& [idxName, spec] : m->indices) { From 81df9afec6821c2cc94bde0933b6bab092bcc86e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 05:56:01 +0800 Subject: [PATCH 10/15] chore: xlings pin 2026.9.16.1 -> 2026.9.20.1 --- .github/actions/bootstrap-mcpp/action.yml | 2 +- .github/actions/setup-macos-llvm/action.yml | 2 +- .github/workflows/bootstrap-macos.yml | 2 +- .github/workflows/ci-fresh-install.yml | 6 +++--- .github/workflows/ci-linux-e2e.yml | 2 +- .github/workflows/cross-build-test.yml | 4 ++-- .github/workflows/release.yml | 14 +++++++------- src/xlings/xlings.cppm | 9 ++++++++- 8 files changed, 24 insertions(+), 17 deletions(-) diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index dafd40cb..d69f3fbf 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.9.16.1' + default: '2026.9.20.1' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index caa13212..8feb5080 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.9.16.1' + default: '2026.9.20.1' image: description: > The runner label the job runs on (macos-15, xcode-27). It is part of the diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index 4d65b1eb..aef5c7e4 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 2f3d5c41..352c77dd 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.16.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -312,7 +312,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.16.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -403,7 +403,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.16.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 20befd35..18c3398a 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -384,7 +384,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.16.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index bed5d332..8a0ff807 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -135,7 +135,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -289,7 +289,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 85d4513e..62463730 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -314,7 +314,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -385,7 +385,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.9.16.1-linux-aarch64.tar.gz" + XLA="xlings-2026.9.20.1-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -394,9 +394,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.9.16.1/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.9.20.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.9.16.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.9.20.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -474,7 +474,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -657,7 +657,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.16.1' + XLINGS_VERSION: '2026.9.20.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index 51241133..ab6ea539 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -96,7 +96,14 @@ namespace pinned { // GitCode fell back to a git clone from GitHub instead of trying GLOBAL; // and an index refresh had no bound on a connection that went silent // after its handshake. - inline constexpr std::string_view kXlingsVersion = "2026.9.16.1"; + // + // Fifth, at 2026.9.20.1 (openxlings/xlings#610). Below it, the global + // workspace could be read out of a project's subos, and the derived shim + // table then removed every global entry a project did not declare; an + // install that failed to download could still print `installed`. mcpp + // drives xlings from inside project and sandbox subos, which is the + // position where the first misread applied. + inline constexpr std::string_view kXlingsVersion = "2026.9.20.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } From c10876ccc447c311c89e8dbc93f66080ce7bb98d Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 06:02:17 +0800 Subject: [PATCH 11/15] fix: a member inside an index archive inherits its workspace; publish reads the one key table (#690) inherit_as_workspace_member is the one function for a sibling path dependency, a git-hosted member and a member inside an index package's archive (Form A pointer), searched no higher than the install root. normalize.cppm writes [workspace.build] back through kWorkspaceBuildKeys instead of a second copy of the key set. e2e 774. --- src/build/prepare.cppm | 95 ++++++++++---- src/publish/normalize.cppm | 124 +++++++----------- ...x_member_inherits_its_archive_workspace.sh | 59 +++++++++ 3 files changed, 173 insertions(+), 105 deletions(-) create mode 100755 tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 4aeed236..26cf0627 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -734,6 +734,54 @@ unfolded_defines_error(const mcpp::manifest::Manifest& m) { d.size(), d.size() == 1 ? "y" : "ies", d.front()); } +// WHAT A MEMBER RECEIVES FROM ITS WORKSPACE WHEN IT IS REACHED AS A DEPENDENCY. +// +// Three parts of the inheritance matter to a dependency: `[workspace.package]` +// (a member may omit `version`), `x.workspace = true` dependency entries +// (without the merge the entry reaches resolution with neither version nor +// path), and `[workspace.build]`. They are applied at the dependency's LOAD +// site, before the conditional merge and the `defines` fold, which is the +// order the root follows; `makePackageRoot` only captures the result (#690). +// `[toolchain]`, `[target.]` and `[indices]` are decided by the root +// for the whole graph and are not applied to a dependency. +// +// One function for every way a member is reached: a sibling `path` +// dependency, a member of a git-hosted workspace, and a member inside an +// index package's archive. The same commit then compiles the same way in its +// own checkout and in every consumer's graph. +std::optional +inherit_as_workspace_member(mcpp::manifest::Manifest& member, + const mcpp::manifest::Manifest& workspace, + const std::filesystem::path& workspaceRoot, + const std::filesystem::path& memberDir) { + mcpp::project::inherit_workspace_package(member, workspace); + mcpp::project::merge_workspace_deps(member, workspace, workspaceRoot); + mcpp::project::inherit_workspace_build(member, workspace, workspaceRoot); + return mcpp::project::workspace_inheritance_error(member, memberDir); +} + +// The workspace whose `members` list `memberDir`, searched upward from its +// parent and never above `bound` (an index package's install root: the +// archive is the only tree the package's author wrote). +std::optional> +workspace_listing(const std::filesystem::path& memberDir, + const std::filesystem::path& bound) { + auto inside = [&](const std::filesystem::path& p) { + auto rel = p.lexically_normal().lexically_relative(bound.lexically_normal()); + return !rel.empty() && *rel.begin() != ".."; + }; + for (auto p = memberDir.parent_path(); inside(p); p = p.parent_path()) { + if (std::filesystem::exists(p / "mcpp.toml")) { + if (auto ws = mcpp::manifest::load(p / "mcpp.toml"); + ws && ws->workspace.present + && mcpp::project::is_workspace_member(*ws, p, memberDir)) + return std::pair{std::move(*ws), p}; + } + if (p == p.parent_path()) break; + } + return std::nullopt; +} + // ── The SECOND conditional pass: predicates that name a target-side layer ──── // // #540/#494. `docs/14` documents a package adapting to the C library it was @@ -2327,7 +2375,7 @@ prepare_build(bool print_fingerprint, // // A PRELOADED manifest (a host-tool sub-build) is already effective: the // resolver loaded it at the dependency's load site, where a member - // inherits (see `inheritAsMember`). It is not inherited a second time; the + // inherits (see `inherit_as_workspace_member`). It is not inherited a second time; the // workspace it belongs to is still recorded below, so that its own sibling // dependencies inherit as members. std::optional effective; @@ -6199,10 +6247,22 @@ prepare_build(bool print_fingerprint, auto loadFrom = [&](const std::filesystem::path& mcppToml) -> std::expected { - auto dm = mcpp::manifest::load(mcppToml); + // A manifest that is a member of a workspace inside the archive + // receives that workspace's inheritance, as it does from a git + // clone of the same commit (#690). + auto repoWorkspace = workspace_listing(mcppToml.parent_path(), verRoot); + auto dm = mcpp::manifest::load( + mcppToml, {.insideWorkspace = repoWorkspace.has_value()}); if (!dm) return std::unexpected(std::format( "dependency '{}' (at '{}'): {}", depName, mcppToml.string(), dm.error().format())); + if (repoWorkspace) { + if (auto bad = inherit_as_workspace_member( + *dm, repoWorkspace->first, repoWorkspace->second, + mcppToml.parent_path())) + return std::unexpected(std::format( + "dependency '{}': {}", depName, *bad)); + } manifest = std::move(*dm); effRoot = mcppToml.parent_path(); return {}; @@ -8279,32 +8339,13 @@ prepare_build(bool print_fingerprint, name, dep_root.string(), dm.error().format())); } dep_manifest = std::move(*dm); - // A MEMBER REACHED AS A DEPENDENCY INHERITS HERE, AT ITS LOAD SITE, - // EXACTLY AS THE ROOT INHERITS AT ITS OWN. - // - // Three parts of what a member receives from its workspace matter - // to a dependency: `[workspace.package]` (a member may omit - // `version`), `x.workspace = true` dependency entries (without the - // merge the entry reaches resolution with no version and no path, - // and is reported as an unreadable index entry), and - // `[workspace.build]`. All three run before the conditional merge - // and the `defines` fold below, which is the order the root - // follows; the snapshot in `makePackageRoot` only captures the - // result (#690). The remaining parts of `inherit_workspace_config` - // (`[toolchain]`, `[target.]`, `[indices]`) are decided by - // the root for the whole graph and are not applied to a dependency. - // - // A member of a git-hosted workspace inherits from ITS repository, - // anchored at the clone, so that the same commit compiles the same - // way in its own checkout and in a consumer's graph. + // A member reached as a dependency inherits here, at its load + // site; see `inherit_as_workspace_member`. A member of a + // git-hosted workspace inherits from ITS repository, anchored at + // the clone. auto inheritAsMember = [&](const mcpp::manifest::Manifest& ws, - const std::filesystem::path& wsRoot) - -> std::optional { - mcpp::project::inherit_workspace_package(*dep_manifest, ws); - mcpp::project::merge_workspace_deps(*dep_manifest, ws, wsRoot); - mcpp::project::inherit_workspace_build(*dep_manifest, ws, wsRoot); - return mcpp::project::workspace_inheritance_error( - *dep_manifest, dep_root); + const std::filesystem::path& wsRoot) { + return inherit_as_workspace_member(*dep_manifest, ws, wsRoot, dep_root); }; if (depIsMember) { if (auto bad = inheritAsMember(*wsManifest, runtimeWorkspaceRoot)) diff --git a/src/publish/normalize.cppm b/src/publish/normalize.cppm index 3cca029f..0a1e8d48 100644 --- a/src/publish/normalize.cppm +++ b/src/publish/normalize.cppm @@ -58,46 +58,12 @@ namespace { namespace t = mcpp::libs::toml; -// `[workspace.build]` keys, each paired with the field of `BuildConfig` that -// carries it. Vectors of flags, vectors of directories and scalars are written -// back by three different rules, so the table records which rule applies. -struct StringVectorKey { - std::string_view key; - std::vector mcpp::manifest::BuildConfig::* field; -}; -struct PathVectorKey { - std::string_view key; - std::vector mcpp::manifest::BuildConfig::* field; -}; -struct ScalarKey { - std::string_view key; - std::string mcpp::manifest::BuildConfig::* field; -}; - +// The `[workspace.build]` keys are read from `mcpp::manifest::kWorkspaceBuildKeys`, +// the one statement of the inheritable subset that the parser also reads. +// Vectors of flags, vectors of directories and scalars are written back by +// three different rules, selected by the type of the row's field. using BC = mcpp::manifest::BuildConfig; -const StringVectorKey kStringVectors[] = { - {"cflags", &BC::cflags}, - {"cxxflags", &BC::cxxflags}, - {"ldflags", &BC::ldflags}, - {"defines", &BC::defines}, - {"dialect_cxxflags", &BC::dialectCxxflags}, -}; -const PathVectorKey kPathVectors[] = { - {"include_dirs", &BC::includeDirs}, - {"include_dirs_after", &BC::includeDirsAfter}, - {"private_include_dirs", &BC::privateIncludeDirs}, -}; -const ScalarKey kScalars[] = { - {"c_standard", &BC::cStandard}, - {"linkage", &BC::linkage}, - {"target", &BC::target}, - {"cxx_runtime", &BC::cxxRuntime}, - {"dependency_linkage", &BC::dependencyLinkage}, - {"macos_deployment_target", &BC::macosDeploymentTarget}, - {"ios_deployment_target", &BC::iosDeploymentTarget}, -}; - t::Value string_array(const std::vector& v) { t::Array a; for (auto const& s : v) a.emplace_back(s); @@ -380,47 +346,49 @@ normalize_for_publish(const std::filesystem::path& packageDir, if (!build) build = table_at(root, "build", "build", explicitTables); return build; }; - for (auto const& k : kStringVectors) { - if ((w.*k.field).empty()) continue; - auto* bt = build_table(); - if (!bt) return std::unexpected(std::format( - "{}: `build` is not a table", manifestPath.string())); - (*bt)[std::string(k.key)] = string_array(b.*k.field); - changed = true; - } - for (auto const& k : kPathVectors) { - const auto n = (w.*k.field).size(); - if (n == 0) continue; - std::vector dirs; - const auto& all = b.*k.field; - for (std::size_t i = 0; i < all.size(); ++i) { - if (i >= n) { dirs.push_back(all[i].string()); continue; } - if (!inside(packageDir, all[i])) - return std::unexpected(std::format( - "{}: [workspace.build] {} entry '{}' resolves to '{}', " - "outside this package's directory, and the published " - "archive contains only that directory. Move the headers " - "into the package, or declare the directory in the " - "package's own [build] for its own build only.", - manifestPath.string(), k.key, (w.*k.field)[i].string(), - all[i].lexically_normal().string())); - dirs.push_back(all[i].lexically_normal() - .lexically_relative(packageDir.lexically_normal()).generic_string()); + for (auto const& row : mcpp::manifest::kWorkspaceBuildKeys) { + const std::string key(row.key); + if (auto const* f = std::get_if BC::*>(&row.field)) { + if ((w.**f).empty()) continue; + auto* bt = build_table(); + if (!bt) return std::unexpected(std::format( + "{}: `build` is not a table", manifestPath.string())); + (*bt)[key] = string_array(b.**f); + changed = true; + } else if (auto const* f = std::get_if< + std::vector BC::*>(&row.field)) { + const auto n = (w.**f).size(); + if (n == 0) continue; + std::vector dirs; + const auto& all = b.**f; + for (std::size_t i = 0; i < all.size(); ++i) { + if (i >= n) { dirs.push_back(all[i].string()); continue; } + if (!inside(packageDir, all[i])) + return std::unexpected(std::format( + "{}: [workspace.build] {} entry '{}' resolves to '{}', " + "outside this package's directory, and the published " + "archive contains only that directory. Move the headers " + "into the package, or declare the directory in the " + "package's own [build] for its own build only.", + manifestPath.string(), key, (w.**f)[i].string(), + all[i].lexically_normal().string())); + dirs.push_back(all[i].lexically_normal() + .lexically_relative(packageDir.lexically_normal()).generic_string()); + } + auto* bt = build_table(); + if (!bt) return std::unexpected(std::format( + "{}: `build` is not a table", manifestPath.string())); + (*bt)[key] = string_array(dirs); + changed = true; + } else if (auto const* f = std::get_if(&row.field)) { + if ((w.**f).empty()) continue; + auto* bt = build_table(); + if (!bt) return std::unexpected(std::format( + "{}: `build` is not a table", manifestPath.string())); + if (bt->contains(key)) continue; + (*bt)[key] = t::Value{b.**f}; + changed = true; } - auto* bt = build_table(); - if (!bt) return std::unexpected(std::format( - "{}: `build` is not a table", manifestPath.string())); - (*bt)[std::string(k.key)] = string_array(dirs); - changed = true; - } - for (auto const& k : kScalars) { - if ((w.*k.field).empty()) continue; - auto* bt = build_table(); - if (!bt) return std::unexpected(std::format( - "{}: `build` is not a table", manifestPath.string())); - if (bt->contains(k.key)) continue; - (*bt)[std::string(k.key)] = t::Value{b.*k.field}; - changed = true; } } } diff --git a/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh b/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh new file mode 100755 index 00000000..36a70f0c --- /dev/null +++ b/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# requires: python3 +# 774 -- a member inside an index package's archive receives its workspace's +# inheritance, as the same commit does from its own checkout and from a git +# clone (#690, design record F11). +# +# The index descriptor points at a member manifest (`mcpp = "*/libs/wlib/mcpp.toml"`) +# inside an archive whose root is a workspace declaring `[workspace.package] +# version` and `[workspace.build] defines`. The member omits `version` and +# guards the define with `#error`. Before #690 the member was loaded as a +# stand-alone manifest and refused for the missing version. +# +# The archive is seeded into the install path, so the test needs no network: +# the resolver accepts an installed tree whose layout matches the descriptor. +set -e + +T=$(mktemp -d) +trap 'rm -rf "$T"' EXIT +export MCPP_HOME="$T/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +X="$MCPP_HOME/registry/data/xpkgs/probe774-x-wlib/1.0.0/wrepo-1.0.0" +mkdir -p "$X/libs/wlib" "$T/idx/pkgs/p" "$T/app/src" +printf '[workspace]\nmembers = ["libs/wlib"]\n\n[workspace.package]\nversion = "1.0.0"\n\n[workspace.build]\ndefines = ["REPO_DEF=4"]\n' > "$X/mcpp.toml" +printf '[package]\nnamespace = "probe774"\nname = "wlib"\n\n[targets.wlib]\nkind = "lib"\n\n[build]\nsources = ["w.cpp"]\n' > "$X/libs/wlib/mcpp.toml" +printf '#ifndef REPO_DEF\n#error "the archive workspace did not reach its member"\n#endif\nint w_v() { return REPO_DEF; }\n' > "$X/libs/wlib/w.cpp" +cat > "$T/idx/pkgs/p/probe774.wlib.lua" <<'EOF' +package = { + spec = "1", + namespace = "probe774", + name = "probe774.wlib", + description = "Form A member inside a workspace archive", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { ["1.0.0"] = { url = "https://example.invalid/wrepo-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + macosx = { ["1.0.0"] = { url = "https://example.invalid/wrepo-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + windows = { ["1.0.0"] = { url = "https://example.invalid/wrepo-1.0.0.tar.gz", sha256 = "0000000000000000000000000000000000000000000000000000000000000000" } }, + }, + mcpp = "*/libs/wlib/mcpp.toml", +} +EOF +printf '[package]\nname = "app"\nversion = "0.1.0"\n\n[indices]\nprobe774 = { path = "../idx" }\n\n[dependencies.probe774]\nwlib = "1.0.0"\n\n[targets.app]\nkind = "bin"\nmain = "src/main.cpp"\n' > "$T/app/mcpp.toml" +printf 'int w_v();\nint main() { return w_v() == 4 ? 0 : 1; }\n' > "$T/app/src/main.cpp" +cd "$T/app" +"$MCPP" run > run.log 2>&1 || fail "the member inside the archive did not build or run" run.log +cdb=compile_commands.json +n=$(python3 - "$cdb" <<'EOF' +import json, sys +for e in json.load(open(sys.argv[1])): + if e["file"].replace("\\", "/").endswith("libs/wlib/w.cpp"): + print(e["arguments"].count("-DREPO_DEF=4")); sys.exit(0) +print("missing") +EOF +) +[ "$n" = 1 ] || fail "-DREPO_DEF=4 occurs $n times in the member's unit" + +echo "PASS: 774_an_index_member_inherits_its_archive_workspace" From 2ae66233713002465ff6b2f69dd3f981ff122ced Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 06:04:00 +0800 Subject: [PATCH 12/15] docs: publishing a workspace member, the index-archive member, CHANGELOG 2026.9.25.1, SPEC-004 criteria 13-14 --- ...026-09-25-issue-690-implementation-plan.md | 8 +- ...workspace-build-inheritance-consistency.md | 2 + CHANGELOG.md | 75 +++++++++++++++++++ docs/07-workspace.md | 6 +- docs/11-publishing-a-library.md | 33 ++++++++ docs/specs/manifest-semantics.md | 18 ++++- docs/zh/07-workspace.md | 4 +- docs/zh/11-publishing-a-library.md | 27 +++++++ 8 files changed, 163 insertions(+), 10 deletions(-) diff --git a/.agents/docs/2026-09-25-issue-690-implementation-plan.md b/.agents/docs/2026-09-25-issue-690-implementation-plan.md index dcc446a2..80b9b2ab 100644 --- a/.agents/docs/2026-09-25-issue-690-implementation-plan.md +++ b/.agents/docs/2026-09-25-issue-690-implementation-plan.md @@ -73,10 +73,10 @@ Each angle names the property it requires and the evidence that shows it. ## 5. Release chain -1. PR in `mcpp-community/mcpp` with the version 2026.9.25.1 in `mcpp.toml` and `modules/versioning/src/version.cppm`. All required checks green. Squash merge. +1. PR in `mcpp-community/mcpp` with the version 2026.9.25.1 in `mcpp.toml` and `modules/versioning/src/version.cppm`, and the internal xlings pin at the latest xlings release (2026.9.20.1). All required checks green. Squash merge. 2. `origin/main` HEAD run green. 3. `release.yml` dispatched. Each archive is uploaded to GitCode with the local tools the moment it appears (`tools/mirror_res.sh` from xlings, GitCode leg only). GET 200 and byte comparison on both mirrors. 4. `openxlings/xim-pkgindex` bump pull request merged. The published index artifact is read to confirm `latest`. -5. Bootstrap pin pull request in `mcpp-community/mcpp` (`.xlings.json`). The open #689 is resolved first. -6. `mcpp-community/mcpp-index`: CI run against the release. No package change is expected (the design's search found no package that relies on the root include broadcast). If a package fails, it is fixed in one pull request. -7. Sandbox verification: `xlings subos use --sandbox --cmd ...` with the CN mirror set for both xlings and mcpp inside the sandbox. It covers the #690 fixture, the git-member fixture, the F7 shadow fixture, a publish dry run of a workspace member, and a build of two representative index packages. +5. `.xlings.json` (the mcpp that builds mcpp) is not moved: review of 2026-09-25 asked for no separate bootstrap pull request unless a build needs it. +6. `mcpp-community/mcpp-index`: one pull request that moves its CI pins to 2026.9.25.1, which runs every member on every platform against the release. Before the release, 16 members were run locally with the candidate binary (all passed), and no member declares `include_dirs`, so the W6 change has no consumer-side reliance to break there. +7. Sandbox verification: `.agents/docs/2026-09-25-issue-690-verify.sh` in `xlings subos use --sandbox --cmd ...`, with the CN mirror set for both xlings and mcpp inside the sandbox, and the same script against 2026.9.24.1 as the control. diff --git a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md index 79cd8a55..9b4b18ac 100644 --- a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md +++ b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md @@ -75,6 +75,8 @@ The decisions below are derived from these rules. Each rule names its source in **F10 (measured during implementation): a member reached as a dependency does not resolve its own `x.workspace = true` entries.** A sibling `lib` whose manifest says `util.workspace = true` builds under `-p lib`. Under `-p app` it fails with `dependency 'util' has SemVer constraint '' but no readable index entry for it`. The dependency load site applied only `inherit_workspace_package`, so the entry reached resolution with neither version nor path. This is the same placement defect as F1, for the third part of what a member inherits. +**F11 (measured during implementation): a member inside an index package's archive does not inherit its archive's workspace.** A descriptor may point at a member manifest (`mcpp = "*/mcpp/cairo/mcpp.toml"`; 22 installed index packages on the measuring machine have a workspace root in their archive). The resolver loaded that manifest as a stand-alone file, so a member that omits `version` was refused and `[workspace.build]` was ignored, while the same commit consumed through `git` inherits (D1). None of the 22 installed archives declares `[workspace.package]`, `[workspace.build]` or `[workspace.dependencies]`, so applying the inheritance changes no existing package. It is applied through the same function as the sibling and git cases, searching no higher than the install root (e2e 774). + **F8 (reasoned): the F1 dependency's cache key records a define its compile does not carry.** `cache_key.cppm:534` reads `pkg.manifest.buildConfig.defines`, which retains the unfolded entries. Root cause: the build half of inheritance runs inside the snapshot, after the fold. diff --git a/CHANGELOG.md b/CHANGELOG.md index 60873548..28d515ad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,81 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.9.25.1] - 2026-09-25 + +### 工作空间成员在每种位置上以相同方式编译(#690) + +`[workspace.build]` 的继承此前在把清单固定进构建图的那一步执行,晚于 `defines` 的展开。 +作为兄弟成员 `path` 依赖被编译的成员因此收不到工作空间的 `defines`(#690 报告的情形), +而被命令选中的根成员收到两份工作空间的 `cflags`、`cxxflags` 与 `ldflags`。现在成员在自己的 +加载处继承,顺序与根包相同:先继承,再合并条件节,最后展开 `defines`;固定进构建图的一步只读取 +结果,遇到未展开的 `defines` 时报告内部错误。 + +同一处还补上了两件同形的缺口: + +- 作为依赖出现的成员解析自己的 `x.workspace = true` 条目。此前该条目以既无版本也无路径的形式 + 进入解析,报出的是「no readable index entry」。 +- 通过 `git` 引用的、托管在 git 上的工作空间的成员,除 `[workspace.package]` 外,也继承所在仓库 + 的 `[workspace.build]` 与 `[workspace.dependencies]`,相对路径以仓库根为锚点。同一个提交在它 + 自己的检出中与在使用方的依赖图中以相同方式编译。 +- 索引描述符以 `mcpp = "<子目录>/mcpp.toml"` 指向归档内的成员时,该成员以同样方式取得归档中的 + 工作空间;在归档内查找工作空间根不越出该版本的安装根。本机已安装的 22 个带工作空间根的索引包 + 都没有声明可继承的表,这一变化不改变任何现有包。 + +`[workspace.build] ios_deployment_target` 此前被解析、被继承,却被已知键检查拒绝;可继承键现在 +只在一张表里陈述,解析、已知键检查与报错文本都取自这张表。 + +### `defines` 按宏名构成集合 + +`defines` 的条目按包接收它们的顺序合并:`[workspace.build]`、包自己的 `[build]`、命中的 +`[target..build]`。同一宏名的后一个条目在原位替换前一个,条目 `!NAME` 移除该宏名, +每个宏名只以一个 `-D` 词到达编译器。成员因此可以覆盖或移除继承来的宏,而不再产生重定义诊断 +(`-Werror` 下是错误)。`defines` 条目同样取代同一个包的 `cflags`、`cxxflags` 中同名的 `-D` 词。 +更早的 mcpp 会把 `!NAME` 作为 `-D!NAME` 交给编译器,使用它的包需要 mcpp 2026.9.25.1 或更新版本。 + +### 使用方的头文件目录不再进入依赖 + +根包的 `include_dirs` 与 `include_dirs_after`(包括 `private_include_dirs`)此前被写进 +`build.ninja` 的文件级编译标志,图中每个编译单元都读到它们:根包目录中与系统头或依赖头同名的 +文件会在依赖内部遮蔽它们。依赖缓存键不含这些目录,于是按一个工程的根包头文件编译出的依赖对象 +会被另一个无关工程取用(实测:`compat.cjson` 的 `cJSON_Compare(1.0, 1.2)` 在未写任何头文件的 +工程中答 1)。现在每个编译单元只收到自己的包的目录,以及它的依赖公开的目录;根包自己的编译单元 +中每个目录只出现一次。 + +**行为变化。** 依赖若曾依靠使用方的头文件目录才能编译,现在报告缺少头文件;mcpp 在编译器报错 +之后指出该头文件所在的使用方目录。依赖需要通过自己的 `include_dirs` 或它的某个依赖找到头文件。 +依赖缓存的 epoch 由 3 升为 4,升级后首次构建重建一次依赖缓存,不需要任何操作。 + +### 成员清单只有一种读法 + +`mcpp publish`、`mcpp pack`、`mcpp emit xpkg`、`mcpp toolchain list`、`mcpp sbom`、`mcpp index list`/`update`、 +`mcpp update` 的索引刷新、快路径的身份判定与测试发现,此前直接读取成员目录中的原始清单:省略 +`version` 的成员被拒绝,未写 `[toolchain]` 的成员在 `toolchain list` 中显示全局默认工具链,而同一 +目录下的 `mcpp build` 用的是工作空间的工具链。现在它们与 `prepare_build` 经同一个函数读取继承后的 +清单。 + +### 工作空间成员的发布形态自包含 + +在成员目录中执行 `mcpp publish` 或 `mcpp emit xpkg` 时,归档只包含成员自己的目录,此前其中的 +`mcpp.toml` 原样照搬:继承的 `version`、`[build]` 标志不在其中,兄弟成员之间的 `path` 依赖原样保留 +而使用方无法解析,描述符的 `deps` 静默略去这些依赖。现在发布的清单是归一化后的:继承来的值被写出, +`x.workspace = true` 条目取得解析后的来源,兄弟依赖以版本依赖发布,原清单以 `mcpp.toml.orig` 保留, +归一化后的文件同时写到 `target/dist/-.mcpp.toml`。归档由 git 对象按提交日期构建, +两次运行逐字节相同。不带 `version` 的兄弟 `path` 依赖、包外非成员的 `path` 依赖、位于成员目录之外的 +继承头文件目录会被拒绝,报错给出应写的内容。不是工作空间成员的包与此前一样原样归档。 +归一化后的清单只使用 2026.9.24.1 已接受的键,实测该版本的客户端可以构建它。 + +### xlings 2026.9.20.1 + +内置的 xlings 由 2026.9.16.1 升为 2026.9.20.1(openxlings/xlings#610):项目 subos 中执行的命令 +不再从项目的 subos 读取全局工作空间并据此删除全局 shim;下载失败的安装不再报告 `installed`。 + +### 测试基础设施 + +e2e 辅助脚本 `_inherit_toolchain.sh` 改为按版本链接开发者的载荷。此前它链接整个包目录,测试 +新装的版本穿过链接写进开发者的 `~/.mcpp`,链接脚本指向测试结束后即被删除的临时目录(#293 的 +第二种形态;实测 `31_transitive_deps.sh` 写出了这样的 `xim-x-glibc/2.44.3`)。 + ## [2026.9.24.1] - 2026-09-24 ### MSVC ABI 上的 toolset:只选一次,可以指定,记录在案 diff --git a/docs/07-workspace.md b/docs/07-workspace.md index d0547fb5..de122f9b 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -408,7 +408,11 @@ member inherits `[workspace.package]` as it does here (mcpp 2026.9.16.1+; see [05 — Dependencies](05-dependencies.md)). It also inherits its repository's `[workspace.build]` and resolves its `x.workspace = true` entries against that repository's `[workspace.dependencies]` (2026.9.25.1+), so the same commit -compiles the same way in its own checkout and in a consumer's graph. +compiles the same way in its own checkout and in a consumer's graph. A member +that an index descriptor points at inside a tag tarball receives the tarball's +workspace in the same way. Publishing a member with `mcpp publish` writes the +inherited values into the published manifest +([11 — Publishing a Library](11-publishing-a-library.md)). ## 7. Relationship to C++ Modules diff --git a/docs/11-publishing-a-library.md b/docs/11-publishing-a-library.md index 1205ac39..f1ebca59 100644 --- a/docs/11-publishing-a-library.md +++ b/docs/11-publishing-a-library.md @@ -142,6 +142,38 @@ mcpp's build sandbox is network-isolated, so `file://` and seeded one is the copy that will still be there when the publish silently failed. +## Publishing a workspace member *(mcpp 2026.9.25.1+)* + +A member's manifest may leave `version`, `[build]` flags and `x.workspace = true` +entries to its workspace ([07](07-workspace.md) §4.1). Both publishing routes +keep that working. + +- **The descriptor points into the repository's tag tarball** + (`mcpp = "*/libs/http/mcpp.toml"`). The consumer reads the member's manifest + from the tarball and applies the workspace root the tarball contains, as a + `git` consumer does. The manifest is published as written. +- **`mcpp publish` or `mcpp emit xpkg` inside the member.** The archive contains + only the member's directory, so its `mcpp.toml` is normalised: the values the + member inherits are written out, `x.workspace = true` entries receive their + resolved source, and a dependency on a sibling member is published as a + version dependency. The manifest as written is kept beside it as + `mcpp.toml.orig`, and the normalised file is also written to + `target/dist/-.mcpp.toml` for review. The archive is built from + git objects with the commit's dates, so two runs produce the same bytes. A + package that is not a workspace member is archived exactly as before. + +A sibling dependency states the version it is published under: + +```toml +[dependencies] +"acme.util" = { path = "../util", version = "0.3.0" } # path for development, version for consumers +``` + +`mcpp publish` refuses a sibling dependency without `version` and names the +member's version and the line to write. It refuses a `path` dependency outside +the package that is not a workspace member, and an inherited include directory +outside the member's directory, because the archive contains neither. + ## Manifest keys that need a version floor Most `[build]` keys degrade cleanly on an older mcpp: it warns that the key is @@ -169,4 +201,5 @@ re-refreshing the index looking for it. - [ ] `publish-artifact.yml` succeeded - [ ] cold resolve (seeded copy deleted) downloads and compiles it - [ ] consumers bumped +- [ ] a workspace member: the normalised `target/dist/-.mcpp.toml` reviewed diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 6cfd92f2..79f409fa 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -344,6 +344,15 @@ feature-deps feature-xlings ← 限定词是门 的编译命令在两个仅 include 设置不同的根包下相同;根包自己的编译单元中每个根包目录 恰好出现一次;依赖因此找不到头文件时,报错之后指出消费者目录 (`tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh`)。 +13. §9 第 1 条索引归档一种情况的判据:描述符指向归档内一个省略 `version` 的成员时,该成员 + 取得归档工作空间的版本与 `defines`(`tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh`)。 + 第 5 条的判据:成员目录内的 `emit xpkg`、`publish --dry-run`、`toolchain list` 读取继承后的 + 清单(`tests/e2e/773_commands_outside_the_build_read_the_effective_manifest.sh`)。 +14. §9 第 7 条的判据:发布归档中的清单写出继承来的 `version`、`license` 与 `cxxflags`,兄弟边 + 以版本边出现且描述符的 `deps` 列出它,原清单以 `mcpp.toml.orig` 保留;归档的使用方在新旧两个 + 客户端上都能构建;两次发布的归档逐字节相同;缺少 `version` 的兄弟 `path` 边被拒绝且报错给出 + 应写的一行;无需修改的包的归档与此前逐字节相同 + (`tests/e2e/772_a_published_member_is_self_contained.sh`)。 ## 8. 编译 flag 列表的元素 @@ -385,9 +394,10 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 ## 9. 工作空间继承与构建需求的作用域 1. 工作空间成员**必须**恰好接收一次 `[workspace.package]`、`[workspace.build]` 与 - `x.workspace = true` 条目的继承,无论它是命令构建的包、另一个成员的 `path` 依赖, - 还是通过 `git` 引用的、托管在 git 上的工作空间的成员。后一种情况按该成员所在仓库的 - 工作空间根继承,相对路径以该仓库的根为锚点。 + `x.workspace = true` 条目的继承,无论它是命令构建的包、另一个成员的 `path` 依赖、 + 通过 `git` 引用的托管在 git 上的工作空间的成员,还是索引包归档内的成员(描述符的 + `mcpp` 字段指向该成员的清单)。后两种情况按该成员所在仓库或归档的工作空间根继承, + 相对路径以该根为锚点;在归档内查找工作空间根时**禁止**越出该版本的安装根。 2. 向量按工作空间、成员、命中的 `[target..build]` 的顺序追加;`defines` 按 §8 的集合语义合并。标量仅在成员未**声明**该键时取工作空间的值。 3. 继承**必须**在 `defines` 展开之前、在清单被固定进构建图之前完成。实现**必须**拒绝 @@ -414,4 +424,4 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 | 1.3 | 2026-09-14 | 条件依赖声明替换同一身份的无条件声明,`targets.` 成为可条件化的 section,不读取的 section 必须报出(mcpp 2026.9.14.2):新增 §3.1.1 与 §7 第 8 条判据。 | | 1.4 | 2026-09-15 | 库目标的默认链接形态 `linkage`(mcpp 2026.9.15.2):§3.1.1 补默认值的语义、优先顺序与拒绝条件;§7 补第 9 条判据。 | | 1.5 | 2026-09-17 | 编译 flag 列表元素的读法(mcpp 2026.9.17.1,#655):新增 §8 与 §7 第 10 条判据。 | -| 1.6 | 2026-09-25 | 工作空间继承与构建需求的作用域(mcpp 2026.9.25.1,#690):§8 补 `defines` 的集合语义;新增 §9 与 §7 第 11、12 条判据。 | +| 1.6 | 2026-09-25 | 工作空间继承与构建需求的作用域(mcpp 2026.9.25.1,#690):§8 补 `defines` 的集合语义;新增 §9 与 §7 第 11 至 14 条判据。 | diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index 876a7d20..63d15cb4 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -391,7 +391,9 @@ myproject/ `[workspace.package]`（mcpp 2026.9.16.1+；见 [05 —— 依赖](05-dependencies.md)）。 它还继承所在仓库的 `[workspace.build]`，并按该仓库的 `[workspace.dependencies]` 解析自己的 `x.workspace = true` 条目（2026.9.25.1+），因此同一个提交在它自己的 -检出中与在使用方的依赖图中以相同方式编译。 +检出中与在使用方的依赖图中以相同方式编译。索引描述符指向 tag tarball 内的成员时， +该成员以同样方式取得 tarball 中的工作空间。用 `mcpp publish` 发布成员时，继承来的值 +被写入发布的清单（[11 —— 发布库](11-publishing-a-library.md)）。 ## 7. 与 C++ 模块的关系 diff --git a/docs/zh/11-publishing-a-library.md b/docs/zh/11-publishing-a-library.md index 8891333d..b75d9f49 100644 --- a/docs/zh/11-publishing-a-library.md +++ b/docs/zh/11-publishing-a-library.md @@ -130,6 +130,32 @@ mcpp 的构建沙箱与网络隔离，`file://` 和 `http://127.0.0.1` 形式的 **在相信「真的能用」之前，先删掉播种的那一份。** 播种出来的 0.0.48 与已发布的 0.0.48 对构建而言毫无区别，而当发布实际上已悄悄失败时，留在那里的正是播种的那一份。 +## 发布工作空间成员 *(mcpp 2026.9.25.1+)* + +成员的清单可以把 `version`、`[build]` 标志与 `x.workspace = true` 条目交给工作空间 +提供（[07](07-workspace.md) §4.1）。两种发布方式都保留这一点。 + +- **描述符指向仓库 tag tarball 中的成员**（`mcpp = "*/libs/http/mcpp.toml"`）。使用方 + 从 tarball 中读取成员的清单，并应用 tarball 所含的工作空间根，与通过 `git` 引用时 + 相同。清单按原样发布。 +- **在成员目录中执行 `mcpp publish` 或 `mcpp emit xpkg`。** 归档只包含成员自己的目录， + 因此其中的 `mcpp.toml` 被归一化：成员继承的值被写出，`x.workspace = true` 条目取得 + 解析后的来源，对兄弟成员的依赖以版本依赖发布。原样的清单以 `mcpp.toml.orig` 保留在 + 旁边，归一化后的文件同时写到 `target/dist/-.mcpp.toml` 供审阅。归档由 + git 对象按提交的日期构建，两次运行产生相同的字节。不是工作空间成员的包与此前一样 + 原样归档。 + +兄弟依赖写出它发布时的版本： + +```toml +[dependencies] +"acme.util" = { path = "../util", version = "0.3.0" } # path 用于开发，version 用于使用方 +``` + +`mcpp publish` 拒绝不带 `version` 的兄弟依赖，并给出该成员的版本和应写的一行。它同样 +拒绝包外、且不是工作空间成员的 `path` 依赖，以及位于成员目录之外的继承头文件目录，因为 +归档中二者都不存在。 + ## 需要版本下限的 manifest 键 多数 `[build]` 键在较旧的 mcpp 上会干净地降级：警告该键不受支持、忽略它，构建要么 @@ -153,4 +179,5 @@ mcpp 的构建沙箱与网络隔离，`file://` 和 `http://127.0.0.1` 形式的 - [ ] `publish-artifact.yml` 已成功 - [ ] 冷解析（已删掉播种拷贝）能下载并编译 - [ ] 消费方已升版本 +- [ ] 工作空间成员：已审阅归一化后的 `target/dist/-.mcpp.toml` From c592395459fae1da5e7625345770b7351c7c2c0a Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 06:07:44 +0800 Subject: [PATCH 13/15] publish: the archive commit ignores commit.gpgSign and the manifest blob ignores .gitattributes --- src/pm/publisher.cppm | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/pm/publisher.cppm b/src/pm/publisher.cppm index f61e7c9a..3cb438e0 100644 --- a/src/pm/publisher.cppm +++ b/src/pm/publisher.cppm @@ -490,7 +490,9 @@ std::string make_release_tarball(const std::filesystem::path& root, os << o.content; if (!os) return std::format("cannot write '{}'", file.string()); } - auto blob = git_value({ "hash-object", "-w", file.string() }); + // `--no-filters`: the scratch file lies inside the repository, and a + // `.gitattributes` rule for its path must not rewrite the manifest. + auto blob = git_value({ "hash-object", "-w", "--no-filters", file.string() }); if (!blob) return blob.error(); auto entry = *prefixInRepo + o.path.generic_string(); if (auto r = git_value({ "update-index", "--add", "--cacheinfo", @@ -504,8 +506,11 @@ std::string make_release_tarball(const std::filesystem::path& root, { "GIT_AUTHOR_DATE", fields[2] }, { "GIT_COMMITTER_NAME", fields[3] }, { "GIT_COMMITTER_EMAIL", fields[4] }, { "GIT_COMMITTER_DATE", fields[5] }, }; - auto commit = git_value({ "commit-tree", *tree, "-p", "HEAD", "-m", - "mcpp publish: normalized manifest" }, commitEnv); + // `--no-gpg-sign`: a signature carries its own timestamp, and a + // `commit.gpgSign = true` configuration would make the object, and so the + // archive, differ between runs, or fail where no key is available. + auto commit = git_value({ "commit-tree", "--no-gpg-sign", *tree, "-p", "HEAD", + "-m", "mcpp publish: normalized manifest" }, commitEnv); if (!commit) return commit.error(); // Run from the package directory: `git archive` then archives that // subtree only, as it does for HEAD in the overload above. From fff80dc935f9102af669ae65a510445c52f34ac3 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 06:13:27 +0800 Subject: [PATCH 14/15] fix: a host-tool sub-build merges its dependency's conditional sections once (#690 F12); self-review record --- .../docs/2026-09-25-issue-690-self-review.md | 50 ++++++++++++ ...workspace-build-inheritance-consistency.md | 2 + .agents/docs/README.md | 4 +- CHANGELOG.md | 7 ++ modules/manifest/src/types.cppm | 8 ++ src/build/prepare.cppm | 18 ++++- ...ol_merges_its_conditional_sections_once.sh | 79 +++++++++++++++++++ 7 files changed, 163 insertions(+), 5 deletions(-) create mode 100644 .agents/docs/2026-09-25-issue-690-self-review.md create mode 100755 tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh diff --git a/.agents/docs/2026-09-25-issue-690-self-review.md b/.agents/docs/2026-09-25-issue-690-self-review.md new file mode 100644 index 00000000..8192b23c --- /dev/null +++ b/.agents/docs/2026-09-25-issue-690-self-review.md @@ -0,0 +1,50 @@ +--- +subject: review +status: active +--- + +# #690: self-review before release, engine and ecosystem + +- Pull request: mcpp-community/mcpp#691 (2026.9.25.1) +- Design: [2026-09-25-issue-690-workspace-build-inheritance-consistency.md](2026-09-25-issue-690-workspace-build-inheritance-consistency.md); plan: [2026-09-25-issue-690-implementation-plan.md](2026-09-25-issue-690-implementation-plan.md) +- Method: the full `src/` and `modules/` diff read against the principles of the design record (P1 to P8), each finding checked by a measurement or a code citation, and the ecosystem consumers enumerated. + +--- + +## 1. Findings of the review, and what was done + +| # | Finding | Evidence | Resolution | +|---|---|---|---| +| R1 | The archive commit made by `mcpp publish` honoured `commit.gpgSign`. On a host that signs commits, the object would carry a signature timestamp, which breaks reproducibility, or it would fail where no key is available. | Code: `git commit-tree` without `--no-gpg-sign`. After the fix, measured with `commit.gpgsign = true` and `gpg.program = /bin/false` on the repository: `publish --dry-run` succeeds twice with the same sha256. | `--no-gpg-sign` added. | +| R2 | The manifest blob was hashed with the repository's filters, and the scratch file lies inside the repository (`target/dist`). | Code: `git hash-object -w` without `--no-filters`. | `--no-filters` added. | +| R3 | The publisher carried a second copy of the inheritable key set, written in parallel with `kWorkspaceBuildKeys` because the two tasks started from the same base. | Code: `kStringVectors`, `kPathVectors`, `kScalars` in `normalize.cppm`. | Replaced by `kWorkspaceBuildKeys`. | +| R4 | A host-tool sub-build receives its dependency's manifest preloaded. After W1 that manifest is already inherited, and the sub-build's member branch would have inherited it a second time. | Code: `prepare_build` member branch before W4. | The preloaded manifest is treated as effective. Only its workspace is recorded, for the membership test of its own dependencies. | +| R5 | Two e2e scripts from the parallel tasks used the number 770, the same as the lead's. | Directory listing. | Renumbered to 772 and 773. | +| R7 | A host-tool sub-build merged its dependency's conditional sections a second time (design record F12). Found as an open item of this review and then measured: `-include once.h` from a matching section reached the tool twice, on 2026.9.24.1 as well. | e2e 775 fails on 2026.9.24.1 and passes on the candidate; the package builds on its own (control). | The sub-build receives `Manifest::beforeConditionalMerge`. | +| R6 | A member inside an index archive (a Form A descriptor pointing at `*//mcpp.toml`) did not inherit its archive's workspace, which is the same position independence gap as F4 for a third route. | Measured, design record F11 and e2e 774. | Applied through `inherit_as_workspace_member`, bounded by the install root. | + +## 2. Principles, checked + +| Principle | Holds because | Residual | +|---|---|---| +| P1 position independence | One function (`inherit_as_workspace_member`) serves the sibling, git and index-archive routes. The root inherits at load through the effective loader. e2e 770 and 774 count the words in each position. | None known. | +| P2 merge, normalise, snapshot | `makePackageRoot` performs no merge and refuses unfolded `defines`. | The layer-conditional second pass folds again by design, and the keyed fold removes superseded words across passes (unit test `SecondPassRemovesAWordTheFirstPassFolded`). | +| P3 one source of truth | One key table, one loader, one inheritance function. | `inherit_workspace_build` in `project.cppm` still lists its fields explicitly. The unit test `EveryTableRowIsParsedAndInherited` fails if the table and that function disagree. | +| P4 scope | No include directory is broadcast; the std module, the scanner and every rule read per-unit includes. | A consumer-supplied configuration header has no channel. None is needed today (section 3). | +| P5 cache soundness | `kCacheEpoch` 4; a dependency's command is shown identical under two roots (e2e 765 (c)). | None known. | +| P6 published form | Normalised manifest, reproducible archive, `.orig` kept, released 2026.9.24.1 client builds it (e2e 772 with `MCPP_BOOT`). | `[indices]` inherited from the workspace is not written into the published manifest; a member whose dependencies resolve through a workspace-declared index publishes a manifest that names no index for them. This matches a non-member package, which also publishes no `[indices]`. | +| P7 loud invariants | Internal error text follows `plan.cppm`'s form. | None. | +| P8 measured blast radius | Section 3. | The full mcpp-index matrix runs after release, on the pin-moving pull request. | + +## 3. Ecosystem review + +- **mcpp-index members.** 166 test members. None declares `include_dirs` (the two matches are comments), so W6 removes nothing a member relied on. The root workspace declares no `[workspace.build]`, so W1 changes no member's flags. 16 members with C sources, `defines` and include directories (`cjson`, `zlib`, `brotli`, `c-ares`, `expat`, `libpng`, `sqlite3`, `pcre2`, `spdlog-compiled`, `fmtlib.fmt`, `yaml-cpp`, `xxhash`, `md4c`, `libffi`, `mimalloc`, `yyjson`) pass `mcpp test -p` with the candidate. +- **Index packages with a workspace in their archive.** 22 are installed on the measuring machine. None declares `[workspace.package]`, `[workspace.build]` or `[workspace.dependencies]`, so F11's inheritance changes none of them. +- **Descriptor comments.** `compat.godot-cpp.lua:163` states that "a consumer-side header shadow never reaches" the package. That statement was false for an uncached compile before this release and is true after it; no descriptor change is needed. +- **Published clients.** The normalised manifest uses only keys 2026.9.24.1 accepts. `!NAME` in `defines` needs 2026.9.25.1, and docs/04 states the floor. +- **Caches.** Epoch 4 orphans every dependency-cache entry once. mcpp-index's CI caches already key on `MCPP_VERSION`, so the pin move costs the same cold run it always does. +- **xlings.** The internal pin moves to 2026.9.20.1 (openxlings/xlings#610). `check_version_pins.sh` passes. The bootstrap `.xlings.json` does not move (review decision 2026-09-25). + +## 4. Open items outside #690 + +- **The e2e harness still shares existing payload versions by link.** An in-place rewrite of an existing payload by a test still reaches the developer's registry (#293, first shape). diff --git a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md index 9b4b18ac..c5bab7d3 100644 --- a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md +++ b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md @@ -77,6 +77,8 @@ The decisions below are derived from these rules. Each rule names its source in **F11 (measured during implementation): a member inside an index package's archive does not inherit its archive's workspace.** A descriptor may point at a member manifest (`mcpp = "*/mcpp/cairo/mcpp.toml"`; 22 installed index packages on the measuring machine have a workspace root in their archive). The resolver loaded that manifest as a stand-alone file, so a member that omits `version` was refused and `[workspace.build]` was ignored, while the same commit consumed through `git` inherits (D1). None of the 22 installed archives declares `[workspace.package]`, `[workspace.build]` or `[workspace.dependencies]`, so applying the inheritance changes no existing package. It is applied through the same function as the sibling and git cases, searching no higher than the install root (e2e 774). +**F12 (measured during review): a dependency built as a host tool merges its conditional sections twice.** The resolver merges a dependency's `[target..build]` sections for the consumer's target, and the host-tool sub-build received that merged manifest and merged it again for the host. A package whose matching section adds `-include once.h` (a header without a guard) builds on its own and fails as a host tool with `redefinition of 'int once_counter'`, on 2026.9.24.1 as well. The comment above the sub-build called the manifest "pristine". The manifest now records its state before the first merge (`Manifest::beforeConditionalMerge`), and the sub-build receives that state (e2e 775). + **F8 (reasoned): the F1 dependency's cache key records a define its compile does not carry.** `cache_key.cppm:534` reads `pkg.manifest.buildConfig.defines`, which retains the unfolded entries. Root cause: the build half of inheritance runs inside the snapshot, after the fold. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 6f184353..187c34b6 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -308 records. +309 records. ## By subject @@ -65,6 +65,7 @@ Records that declare one. Everything else is listed by date below. ### review +- [#690: self-review before release, engine and ecosystem](2026-09-25-issue-690-self-review.md) — active - [本轮生态级自审](2026-09-20-wave-self-review.md) — active - [#674 设计方案评审：`-include unistd.h` 在 Windows + `presents = "posix"` 上的可行性](2026-09-20-issue-674-design-review.md) — active - [`__cxa_thread_atexit` 在 openkal-Windows 上:两层都已定位并修复](2026-09-20-cxa-thread-atexit-finding.md) — landed @@ -100,6 +101,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 - [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — active +- [#690: self-review before release, engine and ecosystem](2026-09-25-issue-690-self-review.md) — active - [#690: implementation plan](2026-09-25-issue-690-implementation-plan.md) — active - [工具链选择与载荷可信度：实施计划](2026-09-24-toolchain-selection-implementation-plan.md) — landed - [MSVC toolset 的选择、#685、#687 与工具链管理规范：总体设计](2026-09-24-toolchain-selection-and-payload-trust-design.md) — active diff --git a/CHANGELOG.md b/CHANGELOG.md index 28d515ad..bd3426eb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,13 @@ 之后指出该头文件所在的使用方目录。依赖需要通过自己的 `include_dirs` 或它的某个依赖找到头文件。 依赖缓存的 epoch 由 3 升为 4,升级后首次构建重建一次依赖缓存,不需要任何操作。 +### 作为 host tool 构建的依赖只合并一次条件节 + +依赖的 `[target..build]` 条件节先按使用方的目标合并,host tool 子构建此前拿到的正是合并后的 +清单,又按宿主合并一次,同时命中两者的条目因此到达工具两次。实测:条件节中的 `-include once.h` +(无 include guard)让单独构建正常的包作为工具时报重定义错误,2026.9.24.1 同样如此。清单现在记录 +第一次合并之前的状态,子构建从该状态按宿主合并。 + ### 成员清单只有一种读法 `mcpp publish`、`mcpp pack`、`mcpp emit xpkg`、`mcpp toolchain list`、`mcpp sbom`、`mcpp index list`/`update`、 diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index af5cf1de..b9f48b69 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -2083,6 +2083,14 @@ struct Manifest { // values are paths. Kept as text so this module's interface names no JSON // type. std::string packageMetadataJson; + // The manifest as it was before its first conditional merge, recorded by + // `merge_conditional_config`. The merge evaluates `[target.]` + // sections for ONE target, and a package's manifest can be needed for a + // second one: a host-tool sub-build compiles a dependency for the host, + // and merging the already merged manifest again applied the consumer's + // entries and then the host's, so a flag in a matching section reached + // the tool twice (#690, F12). Null when no merge has run. + std::shared_ptr beforeConditionalMerge; }; struct ManifestError { diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 26cf0627..e020bb30 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -402,6 +402,9 @@ void replace_dependencies( export void merge_conditional_config(mcpp::manifest::Manifest& m, const cfgpred::Ctx& ctx) { + // Recorded before the first merge; see Manifest::beforeConditionalMerge. + if (!m.beforeConditionalMerge) + m.beforeConditionalMerge = std::make_shared(m); // A DISTRIBUTION package may carry a leg's link line twice: as `ldflags` // (GNU spelling, which is all an older mcpp reads) and as the neutral // `[target..runtime]` pair, which mcpp renders per dialect. Applying @@ -10374,11 +10377,18 @@ prepare_build(bool print_fingerprint, // twice. A `compat` (Form B) package has no mcpp.toml on // disk at all, so without this the sub-build could not read // a manifest for it in the first place. + // + // UNMERGED, because the sub-build targets the HOST: the + // resolver merged this manifest's conditional sections for + // the consumer's target, and the sub-build merges them for + // its own (#690, F12). if (depIdx >= 1 && depIdx - 1 < dep_manifests.size() - && dep_manifests[depIdx - 1]) - sub.preloaded_manifest = - std::make_shared( - *dep_manifests[depIdx - 1]); + && dep_manifests[depIdx - 1]) { + auto const& dep = *dep_manifests[depIdx - 1]; + sub.preloaded_manifest = dep.beforeConditionalMerge + ? dep.beforeConditionalMerge + : std::make_shared(dep); + } sub.inherited_runtime_selection = std::make_shared< const mcpp::xlings::runtime::RuntimeSelection>( runtimeSelection); diff --git a/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh b/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh new file mode 100755 index 00000000..61f364fd --- /dev/null +++ b/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# 775 -- a dependency compiled as a host tool receives each entry of its +# matching `[target..build]` sections once (#690, design record F12). +# +# The resolver merges a dependency's conditional sections for the consumer's +# target. A host-tool sub-build used to receive that merged manifest and merge +# it again for the host, so an entry of a section matching both reached the +# tool twice. The section here adds `-include once.h`, and `once.h` defines a +# variable without an include guard: included twice, the tool does not compile. +# The package builds on its own, which is the control. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME="$TMP/home" +source "$(dirname "$0")/_inherit_toolchain.sh" +fail() { echo "FAIL: $1"; [ -n "${2:-}" ] && cat "$2"; exit 1; } + +mkdir -p "$TMP/toolpkg/src" "$TMP/toolpkg/inc" "$TMP/app/src" +echo 'int once_counter = 1;' > "$TMP/toolpkg/inc/once.h" +cat > "$TMP/toolpkg/mcpp.toml" <<'EOF' +[package] +name = "toolpkg" +version = "0.1.0" + +[targets.gen] +kind = "bin" +main = "src/gen.cpp" + +[build] +include_dirs = ["inc"] + +[target.'cfg(not(os = "none"))'.build] +cxxflags = ["-include once.h"] +EOF +cat > "$TMP/toolpkg/src/gen.cpp" <<'EOF' +#include +int main(int argc, char** argv) { + if (argc < 2) return 2; + std::FILE* f = std::fopen(argv[1], "w"); + if (!f) return 3; + std::fprintf(f, "int generated_answer() { return %d; }\n", 40 + once_counter); + std::fclose(f); + return 0; +} +EOF +cat > "$TMP/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +toolpkg = { path = "../toolpkg", tools = ["gen"] } +EOF +cat > "$TMP/app/src/main.cpp" <<'EOF' +#include +int generated_answer(); +int main() { std::printf("ANSWER=%d\n", generated_answer()); } +EOF +cat > "$TMP/app/build.mcpp" <<'EOF' +import std; +import mcpp; +int main() { + const char* tool = mcpp::dep_bin("toolpkg", "gen"); + if (!tool || !*tool) return 1; + std::string out = std::string(mcpp::out_dir()) + "/gen.cpp"; + std::string cmd = std::string("\"") + tool + "\" \"" + out + "\""; + if (std::system(cmd.c_str()) != 0) return 1; + mcpp::generated(out.c_str()); +} +EOF + +( cd "$TMP/toolpkg" && "$MCPP" build > "$TMP/direct.log" 2>&1 ) \ + || fail "control: the tool package does not build on its own" "$TMP/direct.log" +cd "$TMP/app" +"$MCPP" build > build.log 2>&1 || fail "the host tool received a conditional entry twice" build.log +"$MCPP" run > run.log 2>&1 || fail "the program did not run" run.log +grep -q '^ANSWER=41' run.log || fail "wrong answer" run.log +echo "PASS: 775_a_host_tool_merges_its_conditional_sections_once" From 7bc8d47550d1496238a56aceb284e74c3ea97622 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Fri, 25 Sep 2026 06:45:25 +0800 Subject: [PATCH 15/15] test: host-spelled fixture path in 772, build-directory words excluded from 765 (c), cmd.exe quoting in 775 --- ...onsumer_include_directory_stays_in_the_consumer.sh | 8 +++++++- tests/e2e/772_a_published_member_is_self_contained.sh | 4 +++- ..._host_tool_merges_its_conditional_sections_once.sh | 11 ++++++++++- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh b/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh index 17809e76..f79ad538 100755 --- a/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh +++ b/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh @@ -159,8 +159,14 @@ def words(root): for e in entries(root): f = norm(e["file"]).split("/")[-1] if f not in ("dep.cpp", "dep_c.c"): continue + # A word naming the root's own build directory (clang's + # `-fprebuilt-module-path=`, MSVC's `/ifcSearchDir`) is where that + # project keeps its BMIs, not an input the dependency is compiled + # against, and it differs between two projects by construction. + own = f"/{root}/target/" out[f] = sorted(norm(x) for x in args(e) - if x.startswith(("-I", "-D", "-f", "-idirafter", "/I", "/D"))) + if x.startswith(("-I", "-D", "-f", "-idirafter", "/I", "/D")) + and own not in norm(x)) return out w1, w2 = words("app"), words("app2") if not w1 or w1 != w2: diff --git a/tests/e2e/772_a_published_member_is_self_contained.sh b/tests/e2e/772_a_published_member_is_self_contained.sh index 14dd6aeb..814b0396 100755 --- a/tests/e2e/772_a_published_member_is_self_contained.sh +++ b/tests/e2e/772_a_published_member_is_self_contained.sh @@ -30,6 +30,8 @@ TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT export MCPP_HOME="$TMP/mcpp-home" source "$(dirname "$0")/_inherit_toolchain.sh" +source "$(dirname "$0")/_host_path.sh" +TMP_HOST="$(host_path "$TMP")" fail() { echo "FAIL: $*"; exit 1; } @@ -132,7 +134,7 @@ name = "app" version = "0.1.0" [dependencies] -"probe.util" = { path = "$TMP/unpacked/util-0.3.0" } +"probe.util" = { path = "$TMP_HOST/unpacked/util-0.3.0" } [targets.app] kind = "bin" diff --git a/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh b/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh index 61f364fd..f27408df 100755 --- a/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh +++ b/tests/e2e/775_a_host_tool_merges_its_conditional_sections_once.sh @@ -65,6 +65,11 @@ int main() { if (!tool || !*tool) return 1; std::string out = std::string(mcpp::out_dir()) + "/gen.cpp"; std::string cmd = std::string("\"") + tool + "\" \"" + out + "\""; +#ifdef _WIN32 + // cmd.exe removes the first and the last quote of a command that starts + // with one; the outer pair keeps both inner pairs intact. + cmd = "\"" + cmd + "\""; +#endif if (std::system(cmd.c_str()) != 0) return 1; mcpp::generated(out.c_str()); } @@ -73,7 +78,11 @@ EOF ( cd "$TMP/toolpkg" && "$MCPP" build > "$TMP/direct.log" 2>&1 ) \ || fail "control: the tool package does not build on its own" "$TMP/direct.log" cd "$TMP/app" -"$MCPP" build > build.log 2>&1 || fail "the host tool received a conditional entry twice" build.log +"$MCPP" build > build.log 2>&1 || { + grep -q "redefinition" build.log \ + && fail "the host tool received a conditional entry twice" build.log + fail "the consumer did not build" build.log +} "$MCPP" run > run.log 2>&1 || fail "the program did not run" run.log grep -q '^ANSWER=41' run.log || fail "wrong answer" run.log echo "PASS: 775_a_host_tool_merges_its_conditional_sections_once"