build(grammars): pin npm installs in grammar-regen scripts

CHANGELOG.md

@@ -80,6 +80,25 @@ for historical reference.
   `dev` extra and `maturin` the `examples` extra so the exports cover
   exactly what each CI job needs.
 
+- The grammar-regeneration scripts now install npm dependencies
+  hash-verified (OpenSSF Scorecard Pinned-Dependencies, code-scanning
+  alerts #759–#761, issue #1012). The four internal grammar crates
+  (`tree-sitter-ccomment`, `tree-sitter-preproc`, `tree-sitter-mozcpp`,
+  `tree-sitter-mozjs`) now commit their `package-lock.json` (previously
+  gitignored — the lockfiles were never actually in git) and
+  `generate-grammars/generate-grammar.sh` installs with
+  `npm ci --include=dev`, which fails loudly on a missing or drifted
+  lockfile. `generate-mozcpp.sh` uses `npm ci` inside the upstream
+  `tree-sitter-cpp` checkout (upstream commits a lockfile at the pinned
+  revision) and replaces the `npm install --no-save tree-sitter-c@0.23.1`
+  override with a registry tarball fetched by exact version and verified
+  against a recorded sha512 before extraction — no npm version
+  resolution at all, and the package's install scripts are never run.
+  `generate-mozjs.sh` gains the `set -euo pipefail` fail-loud guard its
+  mozcpp sibling already had, so an aborted regen can no longer fall
+  through to cleanup and report success. Both regens were verified
+  byte-reproducible from a clean checkout.
+
 - Cleared three RUSTSEC advisories flagged by the `cargo-deny` gate.
   `crossbeam-epoch` (a shipped transitive dependency via `crossbeam`)
   moves `0.9.18` → `0.9.20` for the invalid-pointer-dereference in its

big-code-analysis-book/src/developers/update-grammars.md

@@ -46,7 +46,12 @@ Commit your changes and create a new pull request
 ## Internal grammars
 
 Update the version of `tree-sitter-cli` in the `package.json` file of
-the internal grammar and then install the updated version.
+the internal grammar, then refresh its committed `package-lock.json`
+with `npm install --package-lock-only --ignore-scripts` in the same
+directory and commit both files together. The regen scripts install
+with `npm ci`, which fails loudly when the lockfile is missing or out
+of sync with `package.json` — this keeps every regen hash-verified
+and byte-reproducible (OpenSSF Scorecard Pinned-Dependencies).
 
 The five vendored grammars publish under the `bca-tree-sitter-*`
 namespace (see `RELEASING.md` for the rename rationale), but consumer

generate-grammars/generate-grammar.sh

@@ -17,8 +17,14 @@ set -euo pipefail
 # Enter grammar directory
 pushd "$1" || exit
 
-# Install dependencies
-npm install --include=dev
+# Install dependencies exactly as recorded in the committed
+# package-lock.json (hash-verified by npm). `npm ci` fails loudly when
+# the lockfile is missing or out of sync with package.json — the right
+# behaviour for a reproducibility-sensitive regen path (issue #1012,
+# OpenSSF Scorecard Pinned-Dependencies). After bumping a dependency
+# in package.json, refresh the lockfile with
+# `npm install --package-lock-only --ignore-scripts` and commit it.
+npm ci --include=dev
 
 # Generate grammar
 ./node_modules/.bin/tree-sitter generate

generate-grammars/generate-mozcpp.sh

@@ -15,8 +15,8 @@
 # (cargo compiles against the local glibc, sidestepping the issue).
 
 # Fail loud rather than limp on with a half-regenerated, garbage
-# parser: a failed download / npm install / fetch must abort the
-# run, not silently fall through to `tree-sitter generate`.
+# parser: a failed download / npm ci / fetch / hash check must abort
+# the run, not silently fall through to `tree-sitter generate`.
 set -euo pipefail
 
 # Name of the tree-sitter-cpp crate
@@ -70,18 +70,35 @@ git remote add origin https://github.com/tree-sitter/tree-sitter-cpp.git
 git fetch --depth 1 origin "$TS_CPP_SHA1"
 git checkout FETCH_HEAD
 
-# Install tree-sitter-cpp dependencies
-npm install -y
+# Install tree-sitter-cpp dependencies exactly as recorded in the
+# package-lock.json committed by upstream at the pinned revision
+# (issue #1012, OpenSSF Scorecard Pinned-Dependencies).
+npm ci
 
 # Pin the tree-sitter-c base grammar that tree-sitter-cpp's grammar.js
 # extends (`require('tree-sitter-c/grammar')`). tree-sitter-cpp declares
-# it as a floating `^0.23.1`, so a bare install would silently float to
+# it as a floating `^0.23.1`, so npm resolution would silently float to
 # the latest 0.23.x and change the generated parser — the second
 # non-reproducible axis behind issue #406 (the first being the
 # tree-sitter-cli version, pinned in tree-sitter-mozcpp/package.json).
 # 0.23.1 is the version the committed grammar.json/node-types.json
-# correspond to; pinning it keeps the regen byte-reproducible.
-npm install --no-save tree-sitter-c@0.23.1
+# correspond to. The tarball is fetched by exact version and verified
+# against the sha512 recorded below (matching the npm registry's
+# integrity value for tree-sitter-c@0.23.1) before it replaces
+# whatever upstream's lockfile resolved — no npm resolution at all
+# (issue #1012). Only grammar.js is consumed from the package, so its
+# install scripts are never run.
+TS_C_VERSION="0.23.1"
+TS_C_TARBALL="tree-sitter-c-$TS_C_VERSION.tgz"
+TS_C_SHA512="701344cff2c5d027483c3a848808059c7f0265367f08b81a395cc6e66e60d415c3eb0d9e08080fbdad40d92ce44cae60be73dd12281c41bce97d1c27b6b6eb71"
+wget --header="User-Agent: big-code-analysis grammar regen" \
+	-O "$TS_C_TARBALL" \
+	"https://registry.npmjs.org/tree-sitter-c/-/tree-sitter-c-$TS_C_VERSION.tgz"
+echo "$TS_C_SHA512  $TS_C_TARBALL" | sha512sum --check --quiet -
+rm -rf node_modules/tree-sitter-c
+mkdir -p node_modules/tree-sitter-c
+tar -xzf "$TS_C_TARBALL" --strip-components=1 -C node_modules/tree-sitter-c
+rm -f "$TS_C_TARBALL"
 
 # Exit tree-sitter-cpp directory
 popd || exit

generate-grammars/generate-mozjs.sh

@@ -4,6 +4,12 @@
 #
 # Usage: ./generate-grammars/generate-mozjs.sh
 
+# Fail loud rather than limp on with a half-regenerated, garbage
+# parser: a failed download / fetch — or generate-grammar.sh aborting
+# on `npm ci` lockfile drift — must abort the run, not fall through
+# to the final cleanup and report success (issue #1012).
+set -euo pipefail
+
 # Name of the tree-sitter-javascript crate
 TS_JS_CRATE="tree-sitter-javascript"
 
@@ -38,7 +44,12 @@ rm -rf "$CRATE_OUTPUT" "$CRATE_DIR"
 # Enter the mozjs directory
 pushd tree-sitter-mozjs || exit
 
-# Create tree-sitter-javascript directory
+# Create a fresh tree-sitter-javascript directory. Remove any leftover
+# from an interrupted prior run first: under `set -e` a stale clone
+# would wedge the next run at `git remote add origin` ("remote origin
+# already exists", exit 3). Starting clean keeps the script idempotent
+# and avoids stale partial-clone state.
+rm -rf "$TS_JS_CRATE"
 mkdir -p "$TS_JS_CRATE"
 
 # Enter tree-sitter-javascript directory

tree-sitter-ccomment/.gitignore

@@ -1,4 +1,3 @@
 node_modules
 build
-package-lock.json
 /target/

tree-sitter-mozcpp/.gitignore

@@ -1,4 +1,3 @@
 node_modules
 build
-package-lock.json
 /target/

tree-sitter-mozjs/.gitignore

@@ -1,4 +1,3 @@
 node_modules
 build
-package-lock.json
 /target/

tree-sitter-preproc/.gitignore

@@ -1,4 +1,3 @@
 node_modules
 build
-package-lock.json
 /target/