Skip to content

Add resumable model-version semantic backfills #26

Description

@haasonsaas

Context

The schema stores extractor/model versions, but there is no operator-facing workflow for extracting a new semantic version across previously retained artifacts. Reprocessing a large tenant should not require a single unbounded request.

Proposal

Add a corpusctl backfill command and worker workflow that selects artifacts by tenant, format, architecture, and model version; supports dry-run, bounded batches, resume tokens, and cancellation; and reports progress.

Acceptance criteria

  • Backfills never compare incompatible formats or architectures.
  • A restarted worker resumes without duplicating feature rows or edges.
  • Operators can preview counts and estimated work before enqueueing.
  • Progress and terminal failures are queryable by tenant and model version.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions