Skip to content

fix: convert lint results per chunk to avoid OOM on large repos #5018

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lolgab merged 4 commits into from
Aug 28, 2026
Merged

fix: convert lint results per chunk to avoid OOM on large repos #5018

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
100d832
fix: convert lint results per chunk to avoid OOM on large repos
lolgab Aug 28, 2026
66525fa
fix: expand glob patterns before chunking so default full-repo scans …
lolgab Aug 28, 2026
8a99d6e
test: cover expandFiles and chunkFilesByTotalSize
lolgab Aug 28, 2026
2fca54e
revert: drop glob-expansion, keep only per-chunk-conversion fix
lolgab Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 12 additions & 17 deletions src/engineImpl.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,40 +30,35 @@ export const engineImpl: Engine = async function (
const eslint = new ESLint(options)

// Check if there are any glob patterns in the files array
const lintResults = files.some((file: string) => /\*|\?|\[/.test(file))
? await eslint.lintFiles(files)
const results = files.some((file: string) => /\*|\?|\[/.test(file))
? convertResults(await eslint.lintFiles(files))
: await lintFilesInChunks(eslint, files)

await debugAndCountLintIssues(eslint, lintResults)
debug(`engine: ${results.length} issues/errors found`)

debug("engine: finished")
return convertResults(lintResults).map((r) => r.relativeTo(srcDirPath))
return results.map((r) => r.relativeTo(srcDirPath))

@codacy-production codacy-production Bot Aug 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Suggestion: Calling .map() on the full results array at the end doubles the memory required for the results list right before the function returns. To minimize memory pressure, consider mapping to relative paths earlier within the chunking loop so the final array only contains the finished objects.

}

async function lintFilesInChunks (eslint: ESLint, files: string[]): Promise<ESLint.LintResult[]> {
async function lintFilesInChunks (eslint: ESLint, files: string[]): Promise<ToolResult[]> {
const maxTotalSizePerChunk = 5 * 1024 * 1024; // size in bytes (5MB)
const chunksOfFiles = await chunkFilesByTotalSize(files, maxTotalSizePerChunk)

return lintFilesChunkByChunk(eslint, chunksOfFiles)
}

async function lintFilesChunkByChunk (eslint: ESLint, chunksOfFiles: string[][]): Promise<ESLint.LintResult[]> {
async function lintFilesChunkByChunk (eslint: ESLint, chunksOfFiles: string[][]): Promise<ToolResult[]> {
debug("engine: linting chunks started")
const lintResults = []
const results: ToolResult[] = []
let filesLinted = 0
for (const chunkOfFiles of chunksOfFiles) {
const chunkResults = await eslint.lintFiles(chunkOfFiles)
lintResults.push.apply(lintResults, chunkResults)
filesLinted += chunkResults.length
results.push(...convertResults(chunkResults))

@codacy-production codacy-production Bot Aug 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Suggestion: Using the spread operator (...) with results.push can trigger a RangeError: Maximum call stack size exceeded in Node.js if a chunk contains a very large number of linting issues (typically > 65k). A loop is safer for merging large result sets:

Suggested change
results.push(...convertResults(chunkResults))
convertResults(chunkResults).forEach((r) => results.push(r))

}
debug("engine: linting chunks finished")
debug(`engine: linting chunks finished, ${filesLinted} files linted`)

return lintResults
}

async function debugAndCountLintIssues (eslint: ESLint, lintResults: ESLint.LintResult[]): Promise<void> {
if (!DEBUG) return

const nIssues = lintResults.reduce((acc, r) => acc + r.messages.length, 0)
debug(`engine: ${lintResults.length} files linted and ${nIssues} issues found`)
return results
}

async function chunkFilesByTotalSize(files: string[], maxChunkSize: number): Promise<string[][]> {
Expand Down