Repository navigation
Analysis Export with Provenance #48
Description
Motivation
Neither app can export the analysis table today: embed_explore only saves/repartitions images, precalculated has no download at all. Compute results (2D projections, KMeans runs) live only in the user's Streamlit session.
Exporting projection coordinates as a snapshot is also the only reliable way to reproduce a plot: cuML t-SNE (default solver) is not deterministic even with a seed (see #46 / README reproducibility section), so a "re-run with seed" recipe cannot reproduce a GPU t-SNE view but a coordinate table always can.
Scope
One export action, shared by both apps (shared/components), producing two files:
1. Table (CSV) flat, one row per record
| group | columns | notes |
|---|---|---|
| key | uuid (precalculated) / file_path (embed_explore) |
join key back to source data |
| projections | <tag>_x, <tag>_y per projection run this session |
tag = method + seed, e.g. tsne_s614, umap_s614, pca |
| clustering | every KMeans (k=…) column |
as in df_plot today |
| metadata | original metadata columns | toggle "include original metadata", default on |
Embedding vectors are never exported.
2. Provenance sidecar (JSON)
- app, emb-explorer version, library versions (sklearn / umap-learn / cuml), export timestamp
- source dataset (parquet name or image folder) + record count + data hash
- per projection tag: method, backend, seed, key params (perplexity / n_neighbors), run timestamp
- per KMeans column: k, backend, seed
Session data model
- Retain every projection run as an
(x, y)pair under its tag (tsne_s614_x/_y,umap_s614_x/_y,pca_x/_y); the chart shows the current one. Unseeded runs get a run counter (tsne_r2) since each is a legitimately different view. - Retained projection pairs are carried across re-projection the same way
KMeans (k=…)columns are today (embed_explorecurrently rebuildsdf_plotper projection and copies only KMeans columns).
Activity
egrace479 commentedon Oct 12, 2026
As we discussed, let's also include the representative images for the clusters (actual, not fallbacks, since we're not worried about rendering here).