Skip to content

Coalesce concurrent Ollama preloads#783

Merged
FuJacob merged 1 commit into
mainfrom
fix/ollama-preload-single-flight
Jul 11, 2026
Merged

Coalesce concurrent Ollama preloads#783
FuJacob merged 1 commit into
mainfrom
fix/ollama-preload-single-flight

Conversation

@FuJacob

@FuJacob FuJacob commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Summary

Address the duplicate-preload review feedback from #781 by making Ollama model warmup single-flight per model. Rapid focus changes now await the existing load instead of queuing duplicate /api/generate calls, while completed flights are cleared so later warmups can retry after an Ollama restart.

Validation

  • swiftlint lint --quiet — exited 0.
  • xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' build-for-testing -derivedDataPath build/DerivedDataTEST BUILD SUCCEEDED with the new concurrency tests compiled and linked.
  • xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' build -derivedDataPath build/DerivedDataBUILD SUCCEEDED.
  • Local app-hosted test execution remains subject to the repository's documented host/test-bundle Team ID mismatch.

Linked issues

Refs #781.

Risk / rollout notes

Single-flight state is isolated to the OpenAI-compatible engine's lifetime and keyed by model name. Different models may still preload independently; same-model work is removed after completion so failures and Ollama restarts remain recoverable.

Greptile Summary

This PR adds single-flight Ollama preloading for model warmup. The main changes are:

  • Engine-owned preload controller keyed by model name.
  • Coalesced same-model warmups with retry after completion.
  • New tests for shared same-model work and independent different-model work.

Confidence Score: 4/5

The production path looks mergeable after wiring the new tests into the test bundle.

  • The single-flight controller is localized and clears completed work for later retries.
  • The new tests appear to be dead code unless the Xcode project includes them in CotabbyTests.
  • No security concerns were found in the changed warmup path.

CotabbyTests/OllamaPreloadWorkControllerTests.swift target membership

Important Files Changed

Filename Overview
Cotabby/Services/Runtime/OpenAICompatibleSuggestionEngine.swift Adds an engine-lifetime preload controller and routes Ollama warmups through per-model single-flight work.
CotabbyTests/OllamaPreloadWorkControllerTests.swift Adds focused controller tests, but the file appears to be outside the Xcode test target.

Fix All in Codex Fix All in Claude Code

Reviews (1): Last reviewed commit: "Coalesce concurrent Ollama preloads" | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

@FuJacob FuJacob merged commit 0c56a96 into main Jul 11, 2026
4 of 5 checks passed
@FuJacob FuJacob deleted the fix/ollama-preload-single-flight branch July 11, 2026 00:38
@@ -0,0 +1,77 @@
import XCTest

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Test File Outside Target

This file is added on disk but is not listed in the Xcode project test sources, so the new coalescing tests will not compile or run under the CotabbyTests target. The single-flight behavior can regress while the build still reports success because this coverage is not part of the test bundle.

Fix in Codex Fix in Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant