Skip to content

scale.md

Latest commit

 

History

History
76 lines (61 loc) · 2.53 KB

scale.md

File metadata and controls

76 lines (61 loc) · 2.53 KB
Raw

Million-line scale baseline

Symgliph includes a deterministic scale gate rather than relying on a synthetic microbenchmark alone:

scripts/scale-gate.sh

The runner generates exactly 1,000,000 lines of valid Rust across 100 files. Half of the lines define uniquely named functions and almost half call them, so the analyzer and resolver must materialize and link roughly one million graph records. It then:

  1. compiles the corpus and semantic graph;
  2. serializes the readable JSON artifact;
  3. serializes and verifies the checksummed Postcard artifact;
  4. drops the original graph and decodes the binary artifact;
  5. constructs secondary indexes; and
  6. executes 100,000 name queries and 100,000 ID queries.

GNU time measures peak resident memory around the benchmark process. Release compilation happens before the measurement.

Baseline

Measured on 2026-08-01 with Rust 1.95.0 on a 64-CPU Linux host. The current pipeline is single-threaded.

Measurement Result
Source lines 1,000,000
Source bytes 22,497,400
Semantic nodes 500,000
Semantic references 499,900
Build time 10,997 ms
Throughput 90,933 lines/s
Peak RSS 1,402,892 KiB
Pretty JSON 683,825,789 bytes
JSON encode 1,292 ms
Binary artifact 304,809,114 bytes
Binary encode 732 ms
Binary decode 1,169 ms
Secondary-index build 497 ms
Mean name query 62 ns
Mean ID query 153 ns

Artifact roots for this deterministic fixture:

corpus    b2bc97c3264c2d9060a6fda909f4307a20378b3eb069e148b7f333d1be794fdb
semantics 3363246e026b0650ae8a4c104e1fb20928297ab5fede55d4ecc2d6f136b11780

The Postcard representation is about 55% smaller than pretty JSON without an additional compression layer. The largest remaining scaling cost is the owned in-memory graph, not parsing or lookup. A future disk-backed or interned graph can attack that cost without changing the artifact identity model.

Default regression limits

Gate Default
Build time 120,000 ms
Minimum throughput 10,000 lines/s
Peak RSS 2,097,152 KiB
Binary artifact 419,430,400 bytes
Mean indexed query 10,000 ns
Resolution 100% of synthetic references

Every limit is configurable through the SYMGLIPH_SCALE_* and SYMGLIPH_MAX_* environment variables in scripts/scale-gate.sh.

This fixture is intentionally regular and reproducible. It is a regression gate for scale behavior, not a claim that every real million-line repository has the same symbol density or memory profile.