Repository navigation
Conversation
resolve_backend("auto") resolved to cuML whenever a GPU was present, which made the size heuristics inside reduce_dim and run_kmeans unreachable from the apps and sent small datasets through cuML start-up for no gain.
Replace both with one shared rule, CUML_AUTO_RANGE:
cuML for PCA from 500 samples and KMeans from 1000, UMAP only from 10000 (subprocess start-up has to be amortized), and t-SNE only below 3000 (the collapse-safe exact solver is O(N^2)).
The sidebars now pass the dataset size and method; explicit backend choices are honored unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
EmbeddingService.generate_embeddings already supported recursive listing, but the sidebar never passed it, so a folder organized b, for example, species subfolders reported 0 images. Add an Include subfolders checkbox (default on) and pass it through.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four independent fixes found while reading the app log for a small image folder, one commit each.
a1df5a7 Size-aware
auto:resolve_backend("auto")resolved to cuML whenever a GPU was present, making the size heuristics insidereduce_dim/run_kmeansunreachable from the apps.Both now use one rule,
CUML_AUTO_RANGE, with per-method bounds chosen from measured crossovers of the app's own paths: cuML for PCA from 500 samples and KMeans from 1000, UMAP only from 10000 (subprocess start-up has to be amortized), t-SNE only below 3000 (the collapse-safe exact solver is O(N^2)).Explicit
cuml/sklearnchoices are honored unchanged.Note, the measured numbers behind the bounds are machine-specific , but it should be applicable to general use case.
cf71f61 Progress context reports failure
StreamlitProgressContextshowed its success message whenever the block exited without raising, so "Embedding complete!" appeared above the "no images" error; the save/repartition sections had the exit message overwrite their error.New
fail(message); all four failure sites use it.87db000 Include subfolders
checkbox in the Embed section (the service already supported recursive listing; the UI never asked for it).
bdb71e1 No model load for 0 images
generate_embeddingsreturns beforeload_model_unifiedwhen the listing is empty.see screenshot above for the verified fix.