Context
The schema stores extractor/model versions, but there is no operator-facing workflow for extracting a new semantic version across previously retained artifacts. Reprocessing a large tenant should not require a single unbounded request.
Proposal
Add a corpusctl backfill command and worker workflow that selects artifacts by tenant, format, architecture, and model version; supports dry-run, bounded batches, resume tokens, and cancellation; and reports progress.
Acceptance criteria
- Backfills never compare incompatible formats or architectures.
- A restarted worker resumes without duplicating feature rows or edges.
- Operators can preview counts and estimated work before enqueueing.
- Progress and terminal failures are queryable by tenant and model version.
References
Context
The schema stores extractor/model versions, but there is no operator-facing workflow for extracting a new semantic version across previously retained artifacts. Reprocessing a large tenant should not require a single unbounded request.
Proposal
Add a
corpusctlbackfill command and worker workflow that selects artifacts by tenant, format, architecture, and model version; supports dry-run, bounded batches, resume tokens, and cancellation; and reports progress.Acceptance criteria
References