Skip to content

Record configured source provenance and download configured sources in download-sequence #1209

Closed

Description

@smoparth

Summary

Record the selected source: provider in build order and make download-sequence use the configured source resolver for every source entry.

Context

Parent epic: #1254. Download's saved-input contract was clarified in #1384 and implemented in #1406.

get_source_type() uses legacy hooks and settings. It can write source_url_type: "sdist" for a configured Git resolver. download-sequence currently downloads source only for entries labeled sdist, so a truthful legacy classification would cause it to skip configured Git and other non-sdist sources.

The three existing SourceType values (sdist, override, prebuilt) cannot describe every configured provider or the artifact returned by a download. The artifact kind is known when a download happens, but may be unavailable when bootstrap reuses a cached wheel. source_url_type is therefore a legacy compatibility field, not an artifact audit record or the dispatch policy for configured sources.

Desired behavior

Build order records the effective configured source provider for provenance. For non-prebuilt entries using source:, download-sequence calls the existing sources.download_source() path with the saved version and URL. This dispatches to the configured resolver's normal download(), including archive, Git, and hook profiles. Prebuilt wheel entries keep their existing --include-wheels behavior. Packages without source: keep the legacy source_url_type selection.

Known Git limitation

Git resolvers clone into work_dir, which is a build workspace rather than the reusable archive cache. If download-sequence and a later build use the same work directory, the default cleanup setting removes the earlier checkout and clones again. With cleanup disabled, the later build can fail with FileExistsError because the checkout exists. Repeated sequence runs have the same limitation. This issue does not add a Git cache or checkout reuse; track that design separately.

Acceptance Criteria

  • Build-order entries for packages using source: include the effective provider name, including a variant-level provider. Legacy entries remain readable.
  • Configured Git sources are not written as source_url_type: "sdist". Document the compatibility meaning of source_url_type and avoid presenting it as the actual downloaded artifact kind.
  • For every configured non-prebuilt source entry, download-sequence calls the configured resolver through sources.download_source() using the saved version and URL, regardless of source_url_type.
  • Archive downloads can be reused from their download cache. Git checkout and hook downloads run their normal resolver behavior.
  • Existing prebuilt wheel handling and legacy build-order behavior continue to work.
  • Focused tests cover configured archive, Git, hook, prebuilt, and legacy entries. Documentation states the Git limitation.

Definition of Done

  • Code and focused tests satisfy the acceptance criteria.
  • Build-order and resolver architecture documentation describe the fields, dispatch, and Git limitation.
  • Relevant file-scoped lint, type checks, and tests pass; CI passes on the PR.
  • PR is reviewed and merged into new-resolver-config.

Agent Notes

Start with src/fromager/sources.py, src/fromager/bootstrapper/_build.py, src/fromager/bootstrapper/_bootstrapper.py, src/fromager/commands/download_sequence.py, and src/fromager/packagesettings/_resolver.py. Keep the version-and-URL download boundary from #1406. DownloadedSource.kind describes a download result; it is not needed to decide whether to dispatch a configured source. Do not add a separate prefetch API or Git checkout reuse in this issue.

Activity

  1. self-assigned this
    on Oct 27, 2026
  2. changed the title [-]Update get_source_type() to report correct source type for new source resolver config[/-] [+]Record configured source provenance and make download-sequence prefetch only supported sources[/+] on Oct 28, 2026
  3. changed the title [-]Record configured source provenance and make download-sequence prefetch only supported sources[/-] [+]Record configured source provenance and download configured sources in download-sequence[/+] on Oct 28, 2026
  4. added a commit that references this issue on Oct 29, 2026
    87cbfb9
  5. LalatenduMohanty commented on Oct 30, 2026

    @LalatenduMohanty
    Member

    Fixed by #1410, merged into new-resolver-config. Closing by hand, because Closes: doesn't work for non-default branches.

    Build orders now record source_provider, and download-sequence uses the configured resolver for those entries. A missing or mismatched provider fails before any download.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions