Skip to content

Commit 9909e71

Browse files
speak-agentspeak-agent
committed
docs(bench): publish the pinned-workload run — raw JSON plus the report
第一次「所有会移动数字的东西都被钉住」的运行:工具版本、编译器、被测源码、 参照 mcpp。此前的运行与它不可比。 五条臂(旧 mcpp / 新 mcpp / 新 mcpp+bmi_schedule / cmake / xmake)在钉住的 mcpp 工作负载上,以及 xlings 两种代码风格的对照。原始 JSON 一并提交,表格由 `bench/tools/report.py` 生成而不是手抄。 报告里显式记了三条不该被抹掉的东西:xmake 那一列来自单独的一次运行(它在五臂 那次里的 cold 是无效的)、分离式 cold 差点被当成回归发出去、以及 `edit-comment` 在两个工程上 199x vs 1.00x 的真正原因。
1 parent 4691f47 commit 9909e71

7 files changed

Lines changed: 962 additions & 0 deletions

File tree

‎bench/results/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ than repeating what the directory already says.
1212
|---|---|
1313
| [`five-way-20260812/`](five-way-20260812/) | six engines × three source forms × six scenarios, on a **generated fixture**. cmake is the baseline. Two compilers, one file each. |
1414
| [`mcpp-self-20260813/`](mcpp-self-20260813/) | the same scenarios on the **real project** — mcpp building itself, 138 module interface units. cmake is the baseline. |
15+
| [`pinned-workloads-20260813/`](pinned-workloads-20260813/) | **the first run in which everything that moves a number is pinned** — tools, compiler, measured sources, reference mcpp. mcpp building itself five ways, and xlings in two code styles. Earlier runs are not comparable to it. |
1516
| [`hyperfine-20260812/`](hyperfine-20260812/) | the earlier one-off mcpp-vs-xmake runs, driven by hyperfine before the harness existed. Superseded by the two above; kept because `NOTES.md` records how those numbers were taken. |
1617

1718
**Read the reports, not the JSON.** The raw files are what makes a claim

‎bench/results/pinned-workloads-20260813/mcpp-linux-gcc-5way.json‎

Lines changed: 391 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,391 @@
1+
{
2+
"protocol_version": 1,
3+
"started_at": "2026-08-13T06:04:30Z",
4+
"host": {
5+
"os": "linux",
6+
"arch": "x86_64",
7+
"cpu_model": "13th Gen Intel(R) Core(TM) i9-13900K",
8+
"logical_cores": 32,
9+
"physical_cores": 24,
10+
"heterogeneous": true,
11+
"ram_bytes": 67147722752,
12+
"toolchain": "/home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin/g++"
13+
},
14+
"cells": [
15+
{
16+
"engine": "mcpp@2026.8.13.1",
17+
"compiler": "gcc",
18+
"profile": "release",
19+
"scenario": "cold",
20+
"fixture": "mcpp-2026.8.11.3",
21+
"variant": "modules",
22+
"status": "ok",
23+
"note": "mcpp 2026.8.13.1",
24+
"runs": 1,
25+
"median_s": 79.538,
26+
"min_s": 79.538,
27+
"max_s": 79.538,
28+
"samples": [79.538]
29+
},
30+
{
31+
"engine": "mcpp@2026.8.13.1",
32+
"compiler": "gcc",
33+
"profile": "release",
34+
"scenario": "noop",
35+
"fixture": "mcpp-2026.8.11.3",
36+
"variant": "modules",
37+
"status": "ok",
38+
"note": "mcpp 2026.8.13.1",
39+
"runs": 1,
40+
"median_s": 0.161,
41+
"min_s": 0.161,
42+
"max_s": 0.161,
43+
"samples": [0.161]
44+
},
45+
{
46+
"engine": "mcpp@2026.8.13.1",
47+
"compiler": "gcc",
48+
"profile": "release",
49+
"scenario": "touch-hub",
50+
"fixture": "mcpp-2026.8.11.3",
51+
"variant": "modules",
52+
"status": "ok",
53+
"note": "mcpp 2026.8.13.1",
54+
"runs": 1,
55+
"median_s": 0.401,
56+
"min_s": 0.401,
57+
"max_s": 0.401,
58+
"samples": [0.401]
59+
},
60+
{
61+
"engine": "mcpp@2026.8.13.1",
62+
"compiler": "gcc",
63+
"profile": "release",
64+
"scenario": "edit-body",
65+
"fixture": "mcpp-2026.8.11.3",
66+
"variant": "modules",
67+
"status": "ok",
68+
"note": "mcpp 2026.8.13.1",
69+
"runs": 1,
70+
"median_s": 76.241,
71+
"min_s": 76.241,
72+
"max_s": 76.241,
73+
"samples": [76.241]
74+
},
75+
{
76+
"engine": "mcpp@2026.8.13.1",
77+
"compiler": "gcc",
78+
"profile": "release",
79+
"scenario": "edit-comment",
80+
"fixture": "mcpp-2026.8.11.3",
81+
"variant": "modules",
82+
"status": "ok",
83+
"note": "mcpp 2026.8.13.1",
84+
"runs": 1,
85+
"median_s": 0.382,
86+
"min_s": 0.382,
87+
"max_s": 0.382,
88+
"samples": [0.382]
89+
},
90+
{
91+
"engine": "mcpp@2026.8.13.1+schedule=on",
92+
"compiler": "gcc",
93+
"profile": "release",
94+
"scenario": "cold",
95+
"fixture": "mcpp-2026.8.11.3",
96+
"variant": "modules",
97+
"status": "ok",
98+
"note": "mcpp 2026.8.13.1",
99+
"runs": 1,
100+
"median_s": 35.426,
101+
"min_s": 35.426,
102+
"max_s": 35.426,
103+
"samples": [35.426]
104+
},
105+
{
106+
"engine": "mcpp@2026.8.13.1+schedule=on",
107+
"compiler": "gcc",
108+
"profile": "release",
109+
"scenario": "noop",
110+
"fixture": "mcpp-2026.8.11.3",
111+
"variant": "modules",
112+
"status": "ok",
113+
"note": "mcpp 2026.8.13.1",
114+
"runs": 1,
115+
"median_s": 0.161,
116+
"min_s": 0.161,
117+
"max_s": 0.161,
118+
"samples": [0.161]
119+
},
120+
{
121+
"engine": "mcpp@2026.8.13.1+schedule=on",
122+
"compiler": "gcc",
123+
"profile": "release",
124+
"scenario": "touch-hub",
125+
"fixture": "mcpp-2026.8.11.3",
126+
"variant": "modules",
127+
"status": "ok",
128+
"note": "mcpp 2026.8.13.1",
129+
"runs": 1,
130+
"median_s": 0.221,
131+
"min_s": 0.221,
132+
"max_s": 0.221,
133+
"samples": [0.221]
134+
},
135+
{
136+
"engine": "mcpp@2026.8.13.1+schedule=on",
137+
"compiler": "gcc",
138+
"profile": "release",
139+
"scenario": "edit-body",
140+
"fixture": "mcpp-2026.8.11.3",
141+
"variant": "modules",
142+
"status": "ok",
143+
"note": "mcpp 2026.8.13.1",
144+
"runs": 1,
145+
"median_s": 30.166,
146+
"min_s": 30.166,
147+
"max_s": 30.166,
148+
"samples": [30.166]
149+
},
150+
{
151+
"engine": "mcpp@2026.8.13.1+schedule=on",
152+
"compiler": "gcc",
153+
"profile": "release",
154+
"scenario": "edit-comment",
155+
"fixture": "mcpp-2026.8.11.3",
156+
"variant": "modules",
157+
"status": "ok",
158+
"note": "mcpp 2026.8.13.1",
159+
"runs": 1,
160+
"median_s": 0.181,
161+
"min_s": 0.181,
162+
"max_s": 0.181,
163+
"samples": [0.181]
164+
},
165+
{
166+
"engine": "mcpp@2026.8.11.3",
167+
"compiler": "gcc",
168+
"profile": "release",
169+
"scenario": "cold",
170+
"fixture": "mcpp-2026.8.11.3",
171+
"variant": "modules",
172+
"status": "ok",
173+
"note": "mcpp 2026.8.11.3",
174+
"runs": 1,
175+
"median_s": 79.459,
176+
"min_s": 79.459,
177+
"max_s": 79.459,
178+
"samples": [79.459]
179+
},
180+
{
181+
"engine": "mcpp@2026.8.11.3",
182+
"compiler": "gcc",
183+
"profile": "release",
184+
"scenario": "noop",
185+
"fixture": "mcpp-2026.8.11.3",
186+
"variant": "modules",
187+
"status": "ok",
188+
"note": "mcpp 2026.8.11.3",
189+
"runs": 1,
190+
"median_s": 0.341,
191+
"min_s": 0.341,
192+
"max_s": 0.341,
193+
"samples": [0.341]
194+
},
195+
{
196+
"engine": "mcpp@2026.8.11.3",
197+
"compiler": "gcc",
198+
"profile": "release",
199+
"scenario": "touch-hub",
200+
"fixture": "mcpp-2026.8.11.3",
201+
"variant": "modules",
202+
"status": "ok",
203+
"note": "mcpp 2026.8.11.3",
204+
"runs": 1,
205+
"median_s": 76.528,
206+
"min_s": 76.528,
207+
"max_s": 76.528,
208+
"samples": [76.528]
209+
},
210+
{
211+
"engine": "mcpp@2026.8.11.3",
212+
"compiler": "gcc",
213+
"profile": "release",
214+
"scenario": "edit-body",
215+
"fixture": "mcpp-2026.8.11.3",
216+
"variant": "modules",
217+
"status": "ok",
218+
"note": "mcpp 2026.8.11.3",
219+
"runs": 1,
220+
"median_s": 77.330,
221+
"min_s": 77.330,
222+
"max_s": 77.330,
223+
"samples": [77.330]
224+
},
225+
{
226+
"engine": "mcpp@2026.8.11.3",
227+
"compiler": "gcc",
228+
"profile": "release",
229+
"scenario": "edit-comment",
230+
"fixture": "mcpp-2026.8.11.3",
231+
"variant": "modules",
232+
"status": "ok",
233+
"note": "mcpp 2026.8.11.3",
234+
"runs": 1,
235+
"median_s": 75.692,
236+
"min_s": 75.692,
237+
"max_s": 75.692,
238+
"samples": [75.692]
239+
},
240+
{
241+
"engine": "cmake",
242+
"compiler": "gcc",
243+
"profile": "release",
244+
"scenario": "cold",
245+
"fixture": "mcpp-2026.8.11.3",
246+
"variant": "modules",
247+
"status": "ok",
248+
"note": "cmake version 4.0.2 + ninja",
249+
"runs": 1,
250+
"median_s": 92.325,
251+
"min_s": 92.325,
252+
"max_s": 92.325,
253+
"samples": [92.325]
254+
},
255+
{
256+
"engine": "cmake",
257+
"compiler": "gcc",
258+
"profile": "release",
259+
"scenario": "noop",
260+
"fixture": "mcpp-2026.8.11.3",
261+
"variant": "modules",
262+
"status": "ok",
263+
"note": "cmake version 4.0.2 + ninja",
264+
"runs": 1,
265+
"median_s": 0.282,
266+
"min_s": 0.282,
267+
"max_s": 0.282,
268+
"samples": [0.282]
269+
},
270+
{
271+
"engine": "cmake",
272+
"compiler": "gcc",
273+
"profile": "release",
274+
"scenario": "touch-hub",
275+
"fixture": "mcpp-2026.8.11.3",
276+
"variant": "modules",
277+
"status": "ok",
278+
"note": "cmake version 4.0.2 + ninja",
279+
"runs": 1,
280+
"median_s": 83.387,
281+
"min_s": 83.387,
282+
"max_s": 83.387,
283+
"samples": [83.387]
284+
},
285+
{
286+
"engine": "cmake",
287+
"compiler": "gcc",
288+
"profile": "release",
289+
"scenario": "edit-body",
290+
"fixture": "mcpp-2026.8.11.3",
291+
"variant": "modules",
292+
"status": "ok",
293+
"note": "cmake version 4.0.2 + ninja",
294+
"runs": 1,
295+
"median_s": 85.639,
296+
"min_s": 85.639,
297+
"max_s": 85.639,
298+
"samples": [85.639]
299+
},
300+
{
301+
"engine": "cmake",
302+
"compiler": "gcc",
303+
"profile": "release",
304+
"scenario": "edit-comment",
305+
"fixture": "mcpp-2026.8.11.3",
306+
"variant": "modules",
307+
"status": "ok",
308+
"note": "cmake version 4.0.2 + ninja",
309+
"runs": 1,
310+
"median_s": 82.960,
311+
"min_s": 82.960,
312+
"max_s": 82.960,
313+
"samples": [82.960]
314+
},
315+
{
316+
"engine": "xmake",
317+
"compiler": "gcc",
318+
"profile": "release",
319+
"scenario": "cold",
320+
"fixture": "mcpp-2026.8.11.3",
321+
"variant": "modules",
322+
"status": "ok",
323+
"note": "xmake v3.0.7+HEAD.77d94ad, A cross-platform build utility based on Lua",
324+
"runs": 1,
325+
"median_s": 0.603,
326+
"min_s": 0.603,
327+
"max_s": 0.603,
328+
"samples": [0.603]
329+
},
330+
{
331+
"engine": "xmake",
332+
"compiler": "gcc",
333+
"profile": "release",
334+
"scenario": "noop",
335+
"fixture": "mcpp-2026.8.11.3",
336+
"variant": "modules",
337+
"status": "ok",
338+
"note": "xmake v3.0.7+HEAD.77d94ad, A cross-platform build utility based on Lua",
339+
"runs": 1,
340+
"median_s": 0.341,
341+
"min_s": 0.341,
342+
"max_s": 0.341,
343+
"samples": [0.341]
344+
},
345+
{
346+
"engine": "xmake",
347+
"compiler": "gcc",
348+
"profile": "release",
349+
"scenario": "touch-hub",
350+
"fixture": "mcpp-2026.8.11.3",
351+
"variant": "modules",
352+
"status": "ok",
353+
"note": "xmake v3.0.7+HEAD.77d94ad, A cross-platform build utility based on Lua",
354+
"runs": 1,
355+
"median_s": 82.790,
356+
"min_s": 82.790,
357+
"max_s": 82.790,
358+
"samples": [82.790]
359+
},
360+
{
361+
"engine": "xmake",
362+
"compiler": "gcc",
363+
"profile": "release",
364+
"scenario": "edit-body",
365+
"fixture": "mcpp-2026.8.11.3",
366+
"variant": "modules",
367+
"status": "ok",
368+
"note": "xmake v3.0.7+HEAD.77d94ad, A cross-platform build utility based on Lua",
369+
"runs": 1,
370+
"median_s": 96.060,
371+
"min_s": 96.060,
372+
"max_s": 96.060,
373+
"samples": [96.060]
374+
},
375+
{
376+
"engine": "xmake",
377+
"compiler": "gcc",
378+
"profile": "release",
379+
"scenario": "edit-comment",
380+
"fixture": "mcpp-2026.8.11.3",
381+
"variant": "modules",
382+
"status": "ok",
383+
"note": "xmake v3.0.7+HEAD.77d94ad, A cross-platform build utility based on Lua",
384+
"runs": 1,
385+
"median_s": 83.207,
386+
"min_s": 83.207,
387+
"max_s": 83.207,
388+
"samples": [83.207]
389+
}
390+
]
391+
}

0 commit comments

Comments
 (0)