feat: OpenSCAD language support (.scad) - #1630
Conversation
Index parametric CAD projects. `.scad` files are detected, parsed with the OpenSCAD organisation's own tree-sitter grammar, and turned into symbols. Grammar: @openscad/tree-sitter-openscad 0.6.1, vendored as wasm because it is absent from tree-sitter-wasms and the package publishes no prebuilt binary. ABI 15, no external scanner; built from the pinned package with tree-sitter-cli 0.25.10, matching how every other vendored grammar here is produced. Mappings follow the language rather than forcing it into a shape it lacks: - `module` and `function` both become `function` nodes. A module is a named, parameterised, callable definition whose result is geometry — structurally a function. NOT the `module` kind, which already means a file-level module and would collide with that concept in every cross-language query. - classTypes/methodTypes/interfaceTypes/structTypes/enumTypes/typeAliasTypes are empty. OpenSCAD has none of these, and an approximated class is wrong in a way the caller cannot detect. - `assignment` is deliberately NOT the variable type: the grammar reuses it for default parameter values, named call arguments and let/for bindings, so mapping it would mint a variable for every `cube(center = true)`. A real declaration is `var_declaration`, handled in the visitNode hook. - resolveBody: a named `function` has no body field — its body is an unnamed expression child — so without this, calls inside `function total(v) = sum(scale(v));` produce no edges. - extractImport reads the `include_path` child and records the path VERBATIM. Nothing is joined onto a filesystem path or opened here; path handling lives in the resolver. Transform chains nest as transform_chain -> module_call + transform_chain, so the body walker reaches every operator in `translate(…) rotate(…) cube(…)`, and a leading `!`/`#`/`%`/`*` modifier cannot hide the call it decorates. Validated on BOSL2 (67 .scad, 86k lines): 1,855 functions, 17,295 edges, 66 of 67 files parse with no ERROR node. 9 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`include <p>` / `use <p>` name a PATH, not a symbol, so they now route through path resolution and never fall back to the name-matcher — the same treatment PHP includes, COBOL copybooks and Nix path imports already get. Name-matching got this mostly right, and the measurement says so: on a fixture vendoring BOSL2 + MCAD + dotSCAD with eight colliding basenames, 1,627 of 1,633 imports already resolved correctly with zero wrong edges. Two defects survived, and they are the reason for this change: 1. An INVENTED edge. `include <math.scad>` with no sibling and no library-root match still produced an edge to some indexed math.scad, while OpenSCAD itself reports "Can't open include file". A wrong edge is worse than none (colbymchenry#660). 2. A MISSING edge. `include <../polyhedra.scad>` resolved to nothing because two files share that basename — though the written path names exactly one. After: the invented edges are gone, the relative path resolves, and all 1,627 correct edges are byte-identical. Nothing else moved. Search order is the language's own — the including file's directory first, then the project's library roots. That order is normative: a sibling must beat a same-named file in a library, because that is what the renderer does. Library-root discovery deliberately UNDER-reaches. dotSCAD's examples/ and test/ import as though dotSCAD/src were a declared root; a probe accepting any directory that looks like one would resolve them and assert five edges the actual build does not have — real OpenSCAD refuses them too unless OPENSCADPATH says otherwise. Discovery follows what the project declares, not what the tree suggests. Two tests pin both directions. Path validation reuses validatePathWithinRoot at the INDEXING tier (allowSymlinkEscape), not a new check: the lexical `../` guard still rejects a traversal out of the project, while an in-root symlink to a vendored library is followed — refusing it would leave discovery and resolution disagreeing, which is the defect colbymchenry#935 fixed for the indexing read sites. 10 tests, one per behaviour, escape targets are real fixture files rather than system paths so no assertion can pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…notes
Runs the REQUIRED per-language validation methodology and records what it
found, including where it deviates from the letter of it and why.
Results (sonnet/high, n=2 per arm, 18 runs, 0 contaminated, every run served by
the pinned build and index integrity asserted before and after):
repo dur with dur without Read with Read without explore
KeyV2 14-27s 31-47s 0-1 1-3 1-2
NopSCADlib 10-23s 21-46s 0 1-4 1
dotSCAD 14-31s 23-56s 0-1 1-5 1
Read is 0 in 15 of 18 runs, one explore call answers 16 of 18, and the with-arm
is faster in 18 of 18. Control (express, JS, same harness/model/day): Read 0,0
vs 1,2, faster both runs — and its sufficiency is lower than OpenSCAD's.
Coverage was probed before anything paid: all 9 flow questions connect
end-to-end, node counts stable across re-index.
Deviation, recorded rather than papered over: the Large tier (>1500 source
files) DOES NOT EXIST for OpenSCAD. The repositories large enough by file count
are multi-CAD distribution trees — VoronUsers is 6,644 files of which 12 are
.scad (2,413 STL, 170 Fusion .f3d); fosscad-repo is 4,686 of which 8 (772
SolidWorks .sldprt, 128 Inventor .ipt). GitHub labels them language:OpenSCAD
because Linguist counts only recognised text sources and is blind to binary CAD
formats. The largest genuine OpenSCAD codebase is dotSCAD at 695 .scad. The
tier is left declared empty rather than filled with a repository that would
measure STL distribution instead of this language.
The coverage row also carries a methodology warning that cost two full 36-run
campaigns to learn: run-all.sh falls back to `command -v codegraph` when CG_BIN
is unset, and a `readonly CG_BIN` passed as a command prefix silently fails to
export. Both campaigns therefore ran the RELEASED build, which has no OpenSCAD
support — explore answered "No relevant code found" and its file watcher
deleted every .scad symbol from the index under measurement (KeyV2 5,873->23,
NopSCADlib 6,046->276, dotSCAD 5,499->0, each exactly that repo's non-OpenSCAD
node count). Verify the binary from the mcp-codegraph.json the harness writes,
and assert the index node count is unchanged across each run.
Known frontier, stated plainly: children() — OpenSCAD's dynamic dispatch — is
uncovered; module and function are indistinguishable in the graph by design;
import("part.stl")/surface() asset dependencies are not edges.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ba01b85 to
ee81cff
Compare
|
Three small things turned up while working through 1. 2. 3. Separately, the one finding substantial enough for its own issue is filed as #1631 — The fourth finding — |
Indexes parametric CAD projects:
module/functiondefinitions with their parameters, module instantiations and function calls (including every operator in atranslate(…) rotate(…) cube(…)transform chain), top-level assignments with$fn-style special variables keeping their sigil, andinclude/useresolved by path.Grammar
The OpenSCAD organisation's own
@openscad/tree-sitter-openscad0.6.1, vendored as wasm — it is absent fromtree-sitter-wasmsand the package publishes no prebuilt binary. ABI 15, no external scanner, built from the pinned package withtree-sitter-cli0.25.10, matching how every other vendored grammar here is produced.Mappings follow the language
moduleandfunctionboth becomefunctionnodes. A module is a named, parameterised, callable definition whose result is geometry — structurally a function. Not themodulekind, which already means a file-level module and would collide with that concept in every cross-language query. The cost is accepted knowingly: the graph cannot distinguish the two.classTypes/methodTypes/interfaceTypes/structTypes/enumTypes/typeAliasTypesare empty. OpenSCAD has none of these, and an approximated class is wrong in a way the caller cannot detect.assignmentis deliberately not the variable type. The grammar reuses it for default parameter values, named call arguments andlet/forbindings — mapping it would mint a variable for everycube(center = true). A real declaration isvar_declaration.resolveBodyhook: a namedfunctionhas no body field, its body is an unnamed expression child. Without it, calls insidefunction total(v) = sum(scale(v));produce no edges at all.Import resolution follows the language too
include <p>/use <p>name a path, not a symbol, so they route through path resolution and never fall back to the name-matcher — the same treatment PHP includes, COBOL copybooks and Nix path imports already get.Name-matching got this mostly right, and the measurement says so: on a fixture vendoring BOSL2 + MCAD + dotSCAD with eight colliding basenames, 1,627 of 1,633 imports already resolved correctly with zero wrong edges. Two defects survived, and they are the reason for the change:
include <math.scad>with no sibling and no library-root match still produced an edge to some indexedmath.scad, while OpenSCAD itself reportsCan't open include file. A wrong edge is worse than none ([PHP] include/require(_once) not mapped to dependency edges — only namespaceuse; impact/callers miss the file-include graph #660).include <../polyhedra.scad>resolved to nothing because two files share that basename — though the written path names exactly one.After: invented edges gone, the relative path resolves, and all 1,627 correct edges byte-identical. Nothing else moved.
Search order is the language's own — the including file's directory first, then the project's library roots. That order is normative: a sibling must beat a same-named file in a library, because that is what the renderer does.
Library-root discovery deliberately under-reaches. dotSCAD's
examples/andtest/import as thoughdotSCAD/srcwere a declared root. A probe accepting any directory that looks like one would resolve them and assert five edges the actual build does not have — real OpenSCAD refuses them too unlessOPENSCADPATHsays otherwise. Discovery follows what the project declares, not what the tree suggests. Two tests pin both directions.Path validation reuses
validatePathWithinRootat the indexing tier (allowSymlinkEscape), not a new check. The lexical../guard still rejects a traversal out of the project; an in-root symlink to a vendored library is followed, because refusing it would leave discovery and resolution disagreeing — the defect #935 fixed for the indexing read sites.Validation
Per the REQUIRED methodology in
CLAUDE.md.--model sonnet --effort high, n=2 per arm, 18 runs, 0 contaminated, every run served by the pinned build and index integrity asserted before and after..scad)Read is 0 in 15 of 18 runs; one explore call answers 16 of 18; the with-arm is faster in 18 of 18. Sufficiency (
moved on / answered) is 100% in 15 of 18 — the three exceptions are recall misses, never allocation misses.Control — express (JavaScript), same harness, model and day: Read
0, 0with vs1, 2without, faster both runs. Its sufficiency is lower than OpenSCAD's (grep follow-up in both runs).Coverage was probed before anything paid: all 9 flow questions connect end-to-end in
probe-explore, node counts stable across re-index.Deviation from the tiering, declared rather than papered over
The Large tier (>1500 source files) does not exist for OpenSCAD. The repositories large enough by file count are multi-CAD distribution trees:
.scadVoronDesign/VoronUsers.stl, 478.step, 170 Fusion.f3dmaduce/fosscad-repo.sldprt, 128 Inventor.ipt, 82 FreeCAD.fcstdGitHub labels them
language:OpenSCADbecause Linguist counts only recognised text sources and is blind to binary CAD formats. The largest genuine OpenSCAD codebase found is dotSCAD at 695.scad. The tier is left declared empty rather than filled with a repository that would measure STL distribution instead of this language — theadd-langprocedure already says to skip repos tagged for a language they are mostly not written in.A methodology warning, learned expensively
The coverage row carries this, because two full 36-run campaigns were void before the numbers above.
run-all.shresolvesCG_BIN="${CG_BIN:-$(command -v codegraph)}". Areadonly CG_BINpassed as a command prefix silently fails to export — bash rejects the assignment — so both campaigns ran the released build, which has no OpenSCAD support.codegraph_exploreansweredNo relevant code found(correctly, for that build), and its file watcher re-synced and deleted every.scadsymbol from the index under measurement: KeyV2 5,873 → 23, NopSCADlib 6,046 → 276, dotSCAD 5,499 → 0 — each exactly that repository's non-OpenSCAD node count. The JavaScript control survived untouched, which is precisely why OpenSCAD appeared to fail while the control appeared to pass.Worth guarding in the harness: refuse the fallback, or log the resolved binary and version at the top of every run. Verifying the binary from the
mcp-codegraph.jsonthe harness itself writes is what finally settled it.Known frontier
children()is uncovered — OpenSCAD's dynamic dispatch. No flow question exercised it, so its cost is unmeasured rather than dismissed.moduleandfunctionare indistinguishable in the graph, by the mapping decision above. It did not prevent a single question from being answered.import("part.stl")/surface()asset dependencies are not edges. Unused by all three corpus repositories.Tests
19 added — 9 extraction, 10 resolution. Full suite green at 3,071.
Escape-path assertions use real fixture files in a sibling directory rather than system paths, so none can pass vacuously; the symlink test is POSIX-gated.