Bound recursive project scans

[andrew]

Recursive glob patterns such as **/Snakefile could each walk the entire source tree. Scanning a home directory repeated those walks for every matching knowledge-base entry and could run well beyond 30 seconds.

Build one bounded file index per scan and reuse it for recursive matching, content checks, style inference, and nested Cargo lookup. Default scans cover eight levels and 10,000 entries; --scan-depth, --scan-limit, and --line-count-timeout can override those bounds. Knowledge-base validation rejects unsupported or excessively broad path signals, and reports expose truncation in scan stats.

[Copilot]

Pull request overview

Bounds recursive project scans with reusable indexes, configurable limits, knowledge-base validation, and truncation reporting.

Changes:

• Adds scan depth, entry-count, and line-count controls.
• Reuses indexes across detection and lookup operations.
• Updates CLI wiring, reports, documentation, and tests.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Summary	Findings
report/report.go	Reports scan truncation.	No final findings.
report/report_test.go	Tests report output.	No final findings.
report/markdown.go	Displays truncation in Markdown.	No final findings.
README.md	Documents scan bounds and overrides.	No final findings.
kb/kb.go	Validates tool path patterns.	Critical (1 vote): / is trimmed before absolute-path validation, allowing project-root matches.
kb/kb_test.go	Tests path validation.	No final findings.
detect/detect.go	Implements bounded indexing and scan controls.	Critical (4 votes): Special files such as FIFOs can be read and block outside the timeout. Moderate (2 votes): WalkDir still fully reads and sorts large directories before enforcing the scan limit.
detect/detect_test.go	Tests scan bounds and defaults.	No final findings.
cmd/brief/threat.go	Adds scan options to detection commands.	No final findings.
cmd/brief/main.go	Adds CLI scan options.	No final findings.
cmd/brief/main_test.go	Tests bounded CLI scanning.	No final findings.
cmd/brief/enrich.go	Adds enrichment scan options.	No final findings.
brief.go	Adds scan statistics.	No final findings.

Suppressed comments (9)

README.md:73

• brief diff also runs detect.New directly, but its flag set does not accept or forward --scan-depth, --scan-limit, or --line-count-timeout. As a result, these documented overrides are unavailable for a command that performs the same local detection, and passing one is rejected. Add and propagate the flags in cmdDiff, or scope this documentation to the commands that support them.

Local scans inspect up to eight directory levels and 10,000 filesystem entries by default. Use `--scan-depth N` or `--scan-limit N` to change those bounds. External line counters have a two-second limit, configurable with `--line-count-timeout D`, and reports mark truncated scans. Set any of these three values to `0` to remove that bound. Use `--skip dir1,dir2` to add directory exclusions, or `--tracked` to consider only files tracked by Git. When Rust source exists without a root `Cargo.toml`, brief uses the shallowest `Cargo.toml` within the scan depth as an additional manifest root. Cargo workspace member manifests are also checked for tool configuration.

cmd/brief/main.go:82

• brief diff also constructs an Engine directly (cmd/brief/diff.go:86), so it now inherits the 8-level/10,000-entry/2-second defaults but has no corresponding flags to override them. The README describes diff as running the same detection, so users cannot use the documented scan-bound and line-timeout overrides for this path. Add these flags to diff and pass them through, or document a separate policy.

	scanDepth := fs.Int("scan-depth", detect.DefaultScanDepth, "Max directory depth for recursive detection (0 = unlimited)")
	scanLimit := fs.Int("scan-limit", detect.DefaultScanLimit, "Max filesystem entries to scan (0 = unlimited)")
	lineCountTimeout := fs.Duration("line-count-timeout", detect.DefaultLineCountTimeout, "Max time for line counting (0 = unlimited)")

detect/detect.go:650

• ScanLimit is documented as 0 = unlimited, but this guard treats any negative value as unlimited too. Thus --scan-limit -1 silently bypasses the new bound and can restore the unbounded WalkDir this change is intended to prevent; reject negative scan settings at the CLI/API boundary and apply the same rule to depth and timeout.

		if e.ScanLimit > 0 && visited > e.ScanLimit {
			e.scanTruncated = true
			return errDone

detect/detect.go:2164

• The documented escape hatch is a zero timeout, but <= 0 also makes every negative duration run the external counter without a deadline. A typo such as --line-count-timeout=-1s can therefore reintroduce an unbounded line-count process; reject negative timeout values at the CLI/API boundary instead of treating them as unlimited.

	if e.LineCountTimeout <= 0 {

detect/detect.go:974

• This bounds discovery of the fallback Cargo root, but manifestPaths() still calls addCargoWorkspaceManifests, whose expandWorkspacePatternFrom uses unrestricted filepath.Glob. A root workspace with many members can therefore enumerate and read manifests outside ScanLimit/ScanDepth, bypassing the new scan bound and reporting out-of-scope members. Apply the same indexed/bounded filtering when expanding workspace members.

	e.loadProjectFiles()

detect/detect.go:639

• The return value of WalkDir is ignored, and callback errors are also swallowed below. If an entry or directory is unreadable, the index silently omits its files while Stats.ScanTruncated remains false, making incomplete detection look complete. Capture walk failures and mark the scan incomplete or propagate the error so the report reflects this case.

	_ = filepath.WalkDir(e.Root, func(filePath string, d os.DirEntry, err error) error {

detect/detect.go:660

• In TrackedOnly mode, the depth check runs before isTracked. A tracked ancestor can therefore contain an untracked directory below the depth limit, which sets scanDepthTruncated even though that subtree would immediately be pruned and cannot affect detection. This falsely reports truncation and suppresses line counting; check tracking before recording depth truncation.

			if e.ScanDepth > 0 && pathDepth(rel) > e.ScanDepth {
				e.scanDepthTruncated = true
				return filepath.SkipDir
			}
			if !e.isTracked(rel) {

kb/kb.go:507

• validateContentPath accepts a trailing-slash pattern when its basename has no glob, for example **/conf.py/. contains treats this as a file pattern, but matchPathPattern compares the trailing empty segment against file paths, so it can never match. Reject directory patterns for file_contains and exclude_file_contains.

	if !HasGlobPattern(pattern) {
		return nil
	}
	if HasGlobPattern(path.Base(pattern)) {
		return fmt.Errorf("%s: %s pattern %q must name a specific file", source, field, pattern)

kb/kb.go:485

• hasKey only reads exact candidate paths; it never expands globs or uses projectFiles. This validation nevertheless accepts detect.key_exists patterns such as **/package.json, so such a knowledge-base entry loads but can never match. Reject glob patterns for key_exists or implement bounded glob expansion before parsing.

	for pattern := range tool.Detect.KeyExists {
		if err := checkAll("detect.key_exists", []string{pattern}); err != nil {
			return err
		}
	}

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.