Skip to content

Disabling semantic search doesn't release the embedding model's memory #2

Description

@s3rverro0m

Disabling semantic search doesn't actually free the embedding model. unloadEmbeddingModel() in src/lib/server/embeddings/model.ts just nulls out pipelinePromise/loadedOnnxId:

export function unloadEmbeddingModel(): void {
	pipelinePromise = null;
	loadedOnnxId = null;
}

That drops the JS reference but never calls .dispose() on the transformers.js pipeline, so the onnxruntime session (model weights + its worker thread pool) stays resident in the process. If semantic search was ever turned on once, RAM never comes back down after disabling it — only a full process restart clears it.

Noticed this because my instance sits right at its container memory limit and gets OOM-killed every ~1h, even with semantic search (and everything else that touches embeddings) toggled off in settings. Confirmed via /proc/1/status the process RSS climbs to the limit, and thread count sits around 80 for a plain SvelteKit app, which lines up with an ORT thread pool never getting torn down.

Fix is small, keep a reference to the extractor returned by pipeline() and call extractor.dispose() in unloadEmbeddingModel() before dropping the refs. Happy to open a PR if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions