Coalesce concurrent Ollama preloads#783
Merged
Merged
Conversation
| @@ -0,0 +1,77 @@ | |||
| import XCTest | |||
Contributor
There was a problem hiding this comment.
This file is added on disk but is not listed in the Xcode project test sources, so the new coalescing tests will not compile or run under the CotabbyTests target. The single-flight behavior can regress while the build still reports success because this coverage is not part of the test bundle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Address the duplicate-preload review feedback from #781 by making Ollama model warmup single-flight per model. Rapid focus changes now await the existing load instead of queuing duplicate
/api/generatecalls, while completed flights are cleared so later warmups can retry after an Ollama restart.Validation
swiftlint lint --quiet— exited 0.xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' build-for-testing -derivedDataPath build/DerivedData— TEST BUILD SUCCEEDED with the new concurrency tests compiled and linked.xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' build -derivedDataPath build/DerivedData— BUILD SUCCEEDED.Linked issues
Refs #781.
Risk / rollout notes
Single-flight state is isolated to the OpenAI-compatible engine's lifetime and keyed by model name. Different models may still preload independently; same-model work is removed after completion so failures and Ollama restarts remain recoverable.
Greptile Summary
This PR adds single-flight Ollama preloading for model warmup. The main changes are:
Confidence Score: 4/5
The production path looks mergeable after wiring the new tests into the test bundle.
CotabbyTests.CotabbyTests/OllamaPreloadWorkControllerTests.swift target membership
Important Files Changed
Reviews (1): Last reviewed commit: "Coalesce concurrent Ollama preloads" | Re-trigger Greptile