Skip to content

Commit 9d4995e

Browse files
authored
fix(link): 响应文件按行分隔,拆掉 link.exe 的单行 128 KiB 上限 (2026.8.5.3) (#360)
mcpp-index 的 opencv-module / opencv-module-dnn 在 windows 上编译完 795s / 1166s 之后死在链接: fatal error LNK1170: line in command file contains 135135 or more characters 一条链接边和操作系统之间有**两道**上限,而「改用响应文件」只拆掉了第一道: 1. **命令行** —— Windows CreateProcess 32 KiB;POSIX 下 ninja 用 `sh -c "<整条命令>"`,整条命令是**一个** argv 项,撞的是 MAX_ARG_STRLEN 128 KiB(不是谁都会去查的 2 MiB ARG_MAX)。这道 #344 / PR#345 已经拆掉。 2. **响应文件的单行长度** —— link.exe 上限 128 KiB。`rspfile_content = $in` 让 ninja 把所有对象写成**一行**,于是对象一多就 LNK1170。 改成 `$in_newline`:ninja 用换行分隔。之后**没有任何上限再随对象数增长**。 全平台同一条规则形状,不需要按方言分支:GNU 与 LLVM 的响应文件解析把任何空白 (含换行)当分隔符,而 link.exe / lib.exe 要的正是这种写法。 ── 测试 ───────────────────────────────────────────────────────────────────── 新增 e2e 190,**两面都钉**: - 结构面:生成的规则必须是 `$in_newline`,且不得再出现裸 `$in`; - 可观测面:`ninja -d keeprsp` 保留下来的响应文件必须一行一个对象,且最长行远 低于 128 KiB。 只钉前者的话,ninja 哪天改了 `$in_newline` 的展开方式,测试仍然会绿。 结构断言已在 2026.8.5.2 上验证为红(它那里是 `rspfile_content = $in`)。 链接相关 e2e 全绿:28 / 47 / 86 / 07 / 183 / 190。 ── 同一族的第四次 ────────────────────────────────────────────────────────── #274#247#344,再加本条。前三次的教训写的是「一条命令有多长,不该有人放在 心上」;这次补上的是它的孪生兄弟:**一行有多长同样不该**。构建系统不该有一个 「靠崩溃才发现」的工程规模上限。
1 parent 1f711cc commit 9d4995e

5 files changed

Lines changed: 128 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,25 @@
33
> 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。
44
> 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)
55
6+
## [2026.8.5.3] — 2026-08-05
7+
8+
### 修复
9+
10+
- **链接响应文件按行分隔(`$in_newline`)。** 一条链接边和操作系统之间有**两道**上限,而「改用响应文件」只拆掉了第一道:
11+
12+
1. **命令行**:Windows `CreateProcess` 32 KiB;POSIX 下 ninja 用 `sh -c "<整条命令>"`,整条命令是**一个** argv 项,撞的是 `MAX_ARG_STRLEN` 128 KiB。这道在 #344 / PR#345 已经拆掉。
13+
2. **响应文件的单行长度**:`link.exe` 上限 128 KiB。所有对象写在一行,于是
14+
15+
```
16+
fatal error LNK1170: line in command file contains 135135 or more characters
17+
```
18+
19+
mcpp-index 的 `opencv-module` 与 `opencv-module-dnn` 在 windows 上正是死在这里 —— 链接前的 795s / 1166s 编译全部白做。
20+
21+
改成 `rspfile_content = $in_newline` 之后,**没有任何上限再随对象数增长**。全平台同一条规则形状:GNU 与 LLVM 的响应文件解析把任何空白(含换行)当分隔符,而 link.exe / lib.exe 要的正是这种写法。
22+
23+
> 同一族的第四次(#274 / #247 / #344 / 本条)。前三次的教训写的是「命令有多长不该有人放在心上」;这次补上的是它的孪生兄弟 —— **一行有多长同样不该**。e2e 190 两面都钉:既断言生成的规则,也断言 ninja 真正写出来的文件(`-d keeprsp`),因为只断言前者的话,ninja 哪天改了 `$in_newline` 的展开方式测试仍会绿。
24+
625
## [2026.8.5.2] — 2026-08-05
726
827
修复 `host-module = true`(规则包)的两个缺陷。二者都是 2026.8.5.1 引入的,合起来的效果是:**规则包只能写「手工 printf 指令」的玩具规则**,一旦规则要用它本该用的 API 就编不过。第一个真实使用者(`grpc-m` 的 protoc/gRPC codegen 规则)在第一分钟就同时撞上了这两个。

mcpp.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "mcpp"
3-
version = "2026.8.5.2"
3+
version = "2026.8.5.3"
44
description = "Modern C++ build & package management tool"
55
license = "Apache-2.0"
66
authors = ["mcpp-community"]

src/build/ninja_backend.cppm

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -785,7 +785,26 @@ std::string emit_ninja_string(const BuildPlan& plan) {
785785
cmd.replace(pos, 3, "@$out.rsp");
786786
append(std::format(" command = {}\n", cmd));
787787
append(" rspfile = $out.rsp\n");
788-
append(" rspfile_content = $in\n");
788+
// `$in_newline`, not `$in`: ninja separates by newlines instead
789+
// of spaces. Routing the objects through a response file
790+
// removed the COMMAND-LINE ceiling but left a second one
791+
// nobody had reached yet — link.exe caps a response file's
792+
// LINE at 128 KiB:
793+
//
794+
// fatal error LNK1170: line in command file contains
795+
// 135135 or more characters
796+
//
797+
// which is where mcpp-index's opencv-module landed on windows.
798+
// Same failure shape as the one above it: a build system may
799+
// not have a maximum project size it discovers by crashing.
800+
// Newline separation removes the last per-line bound — no
801+
// ceiling scales with the number of objects any more.
802+
//
803+
// Safe everywhere, so there is still one rule shape: GNU and
804+
// LLVM response-file parsing treat any whitespace as a
805+
// separator, newline included, and link.exe/lib.exe want
806+
// exactly this form.
807+
append(" rspfile_content = $in_newline\n");
789808
} else {
790809
append(std::format(" command = {}\n", cmd));
791810
}

src/version.cppm

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,6 @@ import std;
3131

3232
export namespace mcpp {
3333

34-
inline constexpr std::string_view MCPP_VERSION = "2026.8.5.2";
34+
inline constexpr std::string_view MCPP_VERSION = "2026.8.5.3";
3535

3636
} // namespace mcpp
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
#!/usr/bin/env bash
2+
# 190_link_rspfile_newlines.sh — the link response file separates objects by
3+
# NEWLINES, not spaces.
4+
#
5+
# Two ceilings sit between a link edge and the OS, and routing the objects
6+
# through a response file only removes the first:
7+
#
8+
# 1. the COMMAND LINE — Windows CreateProcess 32 KiB, POSIX MAX_ARG_STRLEN
9+
# 128 KiB (ninja spawns `sh -c "<whole command>"`, so the command is one
10+
# argv entry). Removed by using @rspfile at all — mcpp#344 / PR#345.
11+
# 2. the response file's LINE LENGTH — link.exe caps it at 128 KiB:
12+
#
13+
# fatal error LNK1170: line in command file contains 135135
14+
# or more characters
15+
#
16+
# which is where mcpp-index's opencv-module landed on windows, with every
17+
# object written onto a single line.
18+
#
19+
# `rspfile_content = $in_newline` removes the second. After it, no ceiling
20+
# scales with the number of objects.
21+
#
22+
# Asserted structurally (the generated rule) AND observably (the file ninja
23+
# actually writes, kept with -d keeprsp) — the first alone would still pass if
24+
# ninja ever changed what $in_newline expands to.
25+
set -e
26+
27+
TMP=$(mktemp -d)
28+
trap "rm -rf $TMP" EXIT
29+
cd "$TMP"
30+
31+
mkdir -p multi/src
32+
cat > multi/mcpp.toml <<'EOF'
33+
[package]
34+
name = "multi"
35+
version = "0.1.0"
36+
EOF
37+
# Enough objects that "one per line" is unambiguous — a single-object link
38+
# would look identical either way.
39+
i=1
40+
while [ "$i" -le 24 ]; do
41+
printf 'int f%d() { return %d; }\n' "$i" "$i" > "multi/src/f$i.cpp"
42+
i=$((i + 1))
43+
done
44+
printf 'int main() { return 0; }\n' > multi/src/main.cpp
45+
46+
cd multi
47+
"$MCPP" build > b.log 2>&1 || { cat b.log; echo "FAIL: build"; exit 1; }
48+
49+
ninja_file=$(find target -name build.ninja | head -1)
50+
[ -n "$ninja_file" ] || { echo "FAIL: no build.ninja"; exit 1; }
51+
52+
# 1. Structural: no link rule may write its response file on one line.
53+
if grep -qE '^[[:space:]]*rspfile_content = \$in[[:space:]]*$' "$ninja_file"; then
54+
grep -nE '^[[:space:]]*rspfile_content' "$ninja_file"
55+
echo "FAIL: a link rule still writes its response file on ONE line (\$in)"
56+
exit 1
57+
fi
58+
grep -qE '^[[:space:]]*rspfile_content = \$in_newline[[:space:]]*$' "$ninja_file" || {
59+
grep -nE '^[[:space:]]*rspfile_content' "$ninja_file"
60+
echo "FAIL: no link rule uses \$in_newline"; exit 1; }
61+
echo " ok: link rules declare rspfile_content = \$in_newline"
62+
63+
# 2. Observable: ninja keeps the response file under -d keeprsp, and it holds
64+
# one object per line rather than all of them on the first.
65+
bdir=$(dirname "$ninja_file")
66+
bin_rel=$(cd "$bdir" && ls bin/ 2>/dev/null | head -1)
67+
[ -n "$bin_rel" ] || { echo "FAIL: no linked binary to inspect"; exit 1; }
68+
(cd "$bdir" && rm -f "bin/$bin_rel" && ninja -d keeprsp "bin/$bin_rel" > /dev/null 2>&1) \
69+
|| { echo "FAIL: relink under -d keeprsp"; exit 1; }
70+
71+
rsp=$(find "$bdir" -name '*.rsp' | head -1)
72+
[ -n "$rsp" ] || { echo "FAIL: -d keeprsp left no response file"; exit 1; }
73+
74+
# 25 objects -> 24 newlines (the last line carries no trailing newline).
75+
lines=$(wc -l < "$rsp")
76+
[ "$lines" -ge 20 ] || {
77+
echo "response file has $lines newline(s):"; head -c 300 "$rsp"; echo
78+
echo "FAIL: objects are not one-per-line — the LNK1170 shape is back"
79+
exit 1; }
80+
81+
# ...and no single line is anywhere near link.exe's 128 KiB cap.
82+
longest=$(awk '{ if (length($0) > m) m = length($0) } END { print m+0 }' "$rsp")
83+
[ "$longest" -lt 4096 ] || {
84+
echo "FAIL: longest response-file line is $longest chars"; exit 1; }
85+
echo " ok: $((lines + 1)) objects, longest response-file line $longest chars"
86+
87+
echo "OK"

0 commit comments

Comments
 (0)