test: automated performance benchmarking, and the main-vs-#51 measurement

[RhizoNymph]

Adds automated performance benchmarking across all three layers, then uses it to measure PR #51 against main. Stacks on #51 (feat/perf-integration).

The layers

Criterion microbenchmarks (crates/cc-core/benches/). The subgraph fixture was rebuilt: the old one made N flat File nodes with empty children, which leaves the child->parent map empty, so find_render_ancestor returned on its first probe and the aggregation path never ran. It benchmarked the one input for which the parent map is irrelevant. common::nested_graph now builds Directory > File > CodeBlock > CodeBlock trees (5 directory levels, 3-way fanout) with edges between leaf blocks, rendered with containers COLLAPSED so both endpoints walk real ancestor chains. The flat case is kept as subgraph_flat_no_hierarchy, renamed to say what it is. New groups: parent-map-sensitive subgraph extraction at 2k/10k/50k nodes, subgraph_fully_expanded, neighborhood_bfs, parse_result_serialize (time + printed payload size), and resolve_hub_ambiguity at 500/1000 definer files with a unique-name control.

End-to-end harness (crates/cc-core/examples/perf_harness.rs). Runs scan -> parse -> resolve, then the query battery the UI issues — subgraph extraction at four render-set sizes, 200 neighborhood queries, 50 edge-detail drill-ins, parse-payload build + serialize — and prints one JSON object of timings, graph stats and payload bytes. It compiles UNCHANGED on main: it uses only the cc-core API identical on both sides and never names ParseResult as a type. Workload selection is a seeded SplitMix64 stream over sorted id lists, so nothing depends on HashMap iteration order.

Synthetic repo generator (benchmarks/gen_repo.py). Seeded, deterministic, byte-identical between checkouts. Presets 200/2000/10000 modules. --hub-fraction sets the share of modules defining the same names (get, run, new, __init__), which is what drives symbol resolution into its ambiguous tiers.

Frontend routing bench (packages/app/benchmarks/edgeRouting.bench.ts). Deliberately outside the tests/*.test.ts glob. Three scenarios: full_scan_routing (reimplemented in the bench, so identical on every branch — the control), indexed_routing (needs obstacleIndex.ts, reported as skipped where absent), and shipped_redraw (whatever this branch actually does, budget gate included).

Runner and CI. benchmarks/run_all.sh drives every layer with uniform parameters. .github/workflows/bench.yml (workflow_dispatch, or a bench PR label) produces the same artifacts on demand, with no thresholds — shared runners are too noisy for a gate, and a gate built on that noise would be ignored within a week.

How to run

python3 benchmarks/gen_repo.py --preset medium --out "$TMPDIR/cc-medium" --force
cargo bench -p cc-core                                    # or -- --quick
cargo run --release --example perf_harness -- --repo "$TMPDIR/cc-medium" --label mine
cd packages/app && node benchmarks/edgeRouting.bench.ts
benchmarks/run_all.sh <label> <out-dir> <repos-dir> [real-repo]   # all of it

Full procedure, including the two-worktree branch comparison, in docs/features/benchmarking.md.

Headline: main @ 30ab9b5 vs feat/perf-integration @ 1e8903e

One machine (Ryzen 7 7840U), sequential runs, identical inputs.

Metric	main	#51	Delta
Open a 10k-file repo, full pipeline	10 750 ms	3 068 ms	-71.5%
...of which symbol resolution	8 435 ms	789 ms	-90.6%
200 neighborhood queries (10k-file repo)	24 147 ms	6 170 ms	-74.4%
50 edge-detail drill-ins (10k-file repo)	12 686 ms	7 185 ms	-43.4%
Subgraph battery, 24 extractions (10k-file repo)	14 021 ms	11 331 ms	-19.2%
Parse payload bytes, 10k-file repo	69.61 MB	64.21 MB	-7.8%
neighborhood_bfs/50000 (criterion)	1.021 s	26.1 ms	-97.6%
resolve_hub_ambiguity/ambiguous_resolve/1000 (criterion)	1.222 s	3.8 ms	-99.6%
subgraph_nested_collapsed/directories_only/10000 (criterion)	9.43 ms	4.46 ms	-55.0%
Canvas redraw, 800n/500e	789 ms	136 ms	-82.7%
Canvas redraw, 1500n/1200e	3 643 ms	~0 ms (routing skipped)	-100%

cargo bench inherits [profile.release], which #51 changes (lto = "thin", codegen-units = 1; main has cargo defaults). That asymmetry is real shipping behaviour, so the table keeps it — but RESULTS.md also carries a profile-matched column (#51 rebuilt with main's flags). In it, every benchmark #51 did not touch lands within ±2% while the targeted ones keep their full win, so the 5-8% seen elsewhere is thin LTO rather than a code change.

The frontend full_scan_routing control agrees across branches to within 1.1% at all three sizes, which is the evidence that the two runs are comparable.

Two findings that go the other way

The "~-22% payload" figure did not reproduce. Measured -7.8% / -8.4% / -9.8% on the three synthetic sizes and -9.1% on this repo. Dumping the payload and measuring per field (via the harness's new --dump-payload) shows why: signature is 9.1% of the payload and the FIFTH largest contributor, behind children (14.7%), id (12.9%), the node-map keys (11.9%) and span (11.5%) — so the doc comment calling it "the single largest contributor to the payload" is wrong for this repo. The node id is paid for three times (map key, id field, and again in the parent's children array), together 39.5% of the payload. That is where the next payload win is.

Skipping the node-map clone is a build-side win, not a serialize-side one. Building the ParseResult is -39.0% at the large size, but SERIALIZING the slim borrowed form is +51.0% despite emitting fewer bytes. Net build+serialize is -8.8% (large), -0.6% (medium), -15.4% (this repo) — between neutral and modest, not the step change the build number alone suggests. The memory saving stands regardless.

Everything else in #51 confirmed, most of it by more than claimed. Details and the full tables are in benchmarks/RESULTS.md.