diff --git a/docs/quality-scorecard.md b/docs/quality-scorecard.md index 2b98ec4b..5ebc577e 100644 --- a/docs/quality-scorecard.md +++ b/docs/quality-scorecard.md @@ -16,7 +16,7 @@ Readiness score: | secure-code-review | 1 vulnerable / 1 benign | 1 expected finding(s) | 1 benign case(s) | not measured; 1 evidence string(s) validated | not measured; 1 benign fixture(s) | valid | not recorded | 5/5 | covered | | owasp-top-10-web | 0 vulnerable / 0 benign | 0 expected finding(s) | 0 benign case(s) | not measured; 0 evidence string(s) validated | not measured; 0 benign fixture(s) | valid | not recorded | 3/5 | metadata-only | | api-security | 1 vulnerable / 1 benign | 1 expected finding(s) | 1 benign case(s) | not measured; 1 evidence string(s) validated | not measured; 1 benign fixture(s) | valid | not recorded | 5/5 | covered | -| dependency-scanning | 1 vulnerable / 1 benign | 1 expected finding(s) | 1 benign case(s) | not measured; 1 evidence string(s) validated | not measured; 1 benign fixture(s) | valid | not recorded | 5/5 | covered | +| dependency-scanning | 3 vulnerable / 2 benign | 3 expected finding(s) | 2 benign case(s) | not measured; 3 evidence string(s) validated | not measured; 2 benign fixture(s) | valid | not recorded | 5/5 | covered | | iam-review | 0 vulnerable / 0 benign | 0 expected finding(s) | 0 benign case(s) | not measured; 0 evidence string(s) validated | not measured; 0 benign fixture(s) | valid | not recorded | 3/5 | metadata-only | | access-review | 0 vulnerable / 0 benign | 0 expected finding(s) | 0 benign case(s) | not measured; 0 evidence string(s) validated | not measured; 0 benign fixture(s) | valid | not recorded | 3/5 | metadata-only | | rbac-design | 0 vulnerable / 0 benign | 0 expected finding(s) | 0 benign case(s) | not measured; 0 evidence string(s) validated | not measured; 0 benign fixture(s) | valid | not recorded | 3/5 | metadata-only | diff --git a/skills/appsec/dependency-scanning/SKILL.md b/skills/appsec/dependency-scanning/SKILL.md index 07a4ad38..f07d6a35 100644 --- a/skills/appsec/dependency-scanning/SKILL.md +++ b/skills/appsec/dependency-scanning/SKILL.md @@ -181,6 +181,62 @@ Typosquatting (also called dependency confusion or combosquatting) is a supply c - Implement dependency confusion protections: claim your internal package names on public registries, or use registry proxy tools like Artifactory or Nexus with routing rules. - Run `socket.dev`, `npm audit signatures`, or `sigstore` verification to validate package provenance. +## Authoritative Dependency Evidence Gates + +Dependency findings must distinguish declared intent from installed reality. A +manifest-only signal is not enough to score exposure when a lockfile, CI install +mode, SBOM, or build artifact proves a different resolved package tree. + +### Manifest, Lockfile, and Build Alignment + +Before scoring a vulnerable dependency, identify the authoritative artifact: + +1. **Manifest intent**: declared ranges in `package.json`, `pyproject.toml`, + `go.mod`, `Cargo.toml`, or equivalent files. +2. **Resolved install**: pinned versions in `package-lock.json`, + `pnpm-lock.yaml`, `yarn.lock`, `poetry.lock`, `go.sum`, `Cargo.lock`, + `packages.lock.json`, or equivalent lockfiles. +3. **Build evidence**: SBOMs, container layers, package manager logs, release + artifacts, or CI install commands such as `npm ci`, + `pnpm install --frozen-lockfile`, `yarn --immutable`, + `pip install --require-hashes`, `cargo build --locked`, or + `dotnet restore --locked-mode`. + +Apply these gates: + +- If the manifest range looks vulnerable but the lockfile and build artifact pin + a patched version, report the manifest range as maintenance debt instead of an + installed vulnerable dependency. +- If the manifest and lockfile disagree, flag **lockfile drift** and state + whether CI installs from the lockfile or resolves fresh versions. +- If a dependency bot PR changes the manifest without updating the lockfile, + require a lockfile refresh or build evidence before treating the fix as + complete. +- In monorepos, map each manifest to the lockfile and package manager used by + that workspace. Do not let one workspace's clean lockfile clear another + workspace's dependency tree. +- When evidence is missing, classify the result as needing evidence or human + review rather than overstating confirmed exposure. + +### Private Registry and Namespace Proof + +For scoped or internal package names, verify registry identity before declaring +dependency confusion risk: + +- Check `.npmrc`, `.yarnrc.yml`, `.pypirc`, `pip.conf`, `nuget.config`, + `settings.xml`, package source mappings, or CI registry configuration for + explicit source-to-namespace bindings. +- For npm and pnpm, verify scoped registry mappings such as + `@company:registry=https://registry.company.example/` and confirm the lockfile + `resolved` URL points to the intended private registry. +- For NuGet, require `` entries that bind internal package + prefixes to the private feed. +- For Maven, verify repository IDs and groupId ownership for internal + coordinates before comparing them with public artifacts. +- If private namespace proof is absent and a public package can satisfy the same + name, flag dependency confusion exposure. If proof is present, record the + mapping and avoid a false positive. + ## Assessment Output Template Before applying or proposing dependency changes, classify each remediation path using [Security Fixer Policy](../../../docs/fixer-policy.md). Include the policy review gate, reviewer evidence, and rollback guidance in the remediation plan. @@ -214,6 +270,8 @@ When performing a dependency scan, produce findings in the following structure: - [ ] Packages with install scripts - [ ] Unmaintained packages (no release in 2+ years) - [ ] Dependency confusion risk (internal name collisions) +- [ ] Manifest/lockfile/build artifact drift +- [ ] Missing private registry namespace proof ### Recommendations @@ -224,12 +282,13 @@ When performing a dependency scan, produce findings in the following structure: 1. **Identify manifests**: Use Glob to locate all package manifest and lockfiles in the project. 2. **Inventory dependencies**: Read manifest files to enumerate direct dependencies and their declared version ranges. -3. **Analyze lockfiles**: Read lockfiles to map the full transitive dependency tree with pinned versions. +3. **Analyze lockfiles**: Read lockfiles to map the full transitive dependency tree with pinned versions, then compare them with manifests and CI install mode for drift. 4. **Vulnerability scan**: Cross-reference packages and versions against known CVE databases. Apply the EPSS+CVSS+KEV triage model. 5. **License audit**: Extract license declarations from lockfiles or registry metadata. Flag copyleft and unlicensed packages. 6. **Typosquatting check**: Review dependency names for patterns described in the detection section. -7. **Supply chain assessment**: Evaluate SLSA posture -- lockfile presence, pinned versions, provenance availability. -8. **Report**: Produce the assessment using the output template above, with prioritized remediation recommendations. +7. **Private registry proof**: Verify namespace-to-registry mappings for internal or scoped package names before scoring dependency confusion risk. +8. **Supply chain assessment**: Evaluate SLSA posture -- lockfile presence, pinned versions, provenance availability. +9. **Report**: Produce the assessment using the output template above, with prioritized remediation recommendations. ## Limitations diff --git a/tests/fixtures/dependency-scanning/lockfile-drift-vulnerable/evidence.md b/tests/fixtures/dependency-scanning/lockfile-drift-vulnerable/evidence.md new file mode 100644 index 00000000..a0872e60 --- /dev/null +++ b/tests/fixtures/dependency-scanning/lockfile-drift-vulnerable/evidence.md @@ -0,0 +1,8 @@ +# Dependency Evidence + +package.json declares lodash ^4.17.21. + +package-lock.json still resolves lodash 4.17.20 while package.json declares lodash ^4.17.21. + +CI uses `npm ci`, so the stale lockfile is the installed dependency source of +truth until the lockfile is refreshed. diff --git a/tests/fixtures/dependency-scanning/lockfile-drift-vulnerable/manifest.yaml b/tests/fixtures/dependency-scanning/lockfile-drift-vulnerable/manifest.yaml new file mode 100644 index 00000000..252f77dd --- /dev/null +++ b/tests/fixtures/dependency-scanning/lockfile-drift-vulnerable/manifest.yaml @@ -0,0 +1,9 @@ +skill: dependency-scanning +case_id: lockfile-drift-vulnerable +kind: vulnerable +target: evidence.md +expected_findings: + - id: lockfile-drift + severity: medium + framework: SLSA-v1.0 + evidence_contains: 'package-lock.json still resolves lodash 4.17.20 while package.json declares lodash ^4.17.21' diff --git a/tests/fixtures/dependency-scanning/manifest-range-lockfile-patched-benign/evidence.md b/tests/fixtures/dependency-scanning/manifest-range-lockfile-patched-benign/evidence.md new file mode 100644 index 00000000..66a56118 --- /dev/null +++ b/tests/fixtures/dependency-scanning/manifest-range-lockfile-patched-benign/evidence.md @@ -0,0 +1,9 @@ +# Dependency Evidence + +package.json declares lodash ^4.17.0. + +package-lock.json resolves lodash 4.17.21, the build uses `npm ci`, and the +SBOM generated from the release artifact also lists lodash 4.17.21. + +Treat the broad manifest range as maintenance debt, not as confirmed installed +exposure. diff --git a/tests/fixtures/dependency-scanning/manifest-range-lockfile-patched-benign/manifest.yaml b/tests/fixtures/dependency-scanning/manifest-range-lockfile-patched-benign/manifest.yaml new file mode 100644 index 00000000..1f39933a --- /dev/null +++ b/tests/fixtures/dependency-scanning/manifest-range-lockfile-patched-benign/manifest.yaml @@ -0,0 +1,5 @@ +skill: dependency-scanning +case_id: manifest-range-lockfile-patched-benign +kind: benign +target: evidence.md +expected_findings: [] diff --git a/tests/fixtures/dependency-scanning/private-registry-fallback-vulnerable/evidence.md b/tests/fixtures/dependency-scanning/private-registry-fallback-vulnerable/evidence.md new file mode 100644 index 00000000..5ff8e8c1 --- /dev/null +++ b/tests/fixtures/dependency-scanning/private-registry-fallback-vulnerable/evidence.md @@ -0,0 +1,8 @@ +# Dependency Evidence + +The project imports `@company/utils` as an internal helper package. + +No scoped registry mapping exists for @company/*, and the lockfile resolved @company/utils from the public npm registry. + +This can allow a public package with the internal name to satisfy the install +unless registry routing or namespace ownership evidence is added. diff --git a/tests/fixtures/dependency-scanning/private-registry-fallback-vulnerable/manifest.yaml b/tests/fixtures/dependency-scanning/private-registry-fallback-vulnerable/manifest.yaml new file mode 100644 index 00000000..2a875b91 --- /dev/null +++ b/tests/fixtures/dependency-scanning/private-registry-fallback-vulnerable/manifest.yaml @@ -0,0 +1,9 @@ +skill: dependency-scanning +case_id: private-registry-fallback-vulnerable +kind: vulnerable +target: evidence.md +expected_findings: + - id: private-registry-namespace-proof-missing + severity: high + framework: SLSA-v1.0 + evidence_contains: 'No scoped registry mapping exists for @company/*, and the lockfile resolved @company/utils from the public npm registry'