fix(retrieval-sdg): support clean provider initialization

[shan-nvidia]

Summary

• Fall back to Data Designer's built-in model providers when the user-level model_providers.yaml has not been initialized yet.
• Add a regression test for custom-provider resolution in a clean home directory.
• Make validate-release select the workspace ddp package explicitly so the guarded release target works from a fresh worktree.

Context

The first data-designer-retrieval-sdg/v0.2.0 publication attempt failed in the isolated package-test step before any distribution or package-index asset was uploaded.

What failed

Data Designer has two representations of its default providers:

1. Built-in provider definitions shipped in the installed Python package.
2. A user-level ~/.data-designer/model_providers.yaml file containing those defaults plus any user edits.

The DataDesigner constructor initializes the user-level file from the built-in definitions when the file does not exist. However, the retrieval SDG plugin resolves model providers before it constructs DataDesigner.

The normal and custom-provider paths therefore behaved differently:

• No custom provider: build_model_providers() returns None without reading the user-level file. The plugin then constructs DataDesigner(model_providers=None); its constructor creates model_providers.yaml and safely loads the defaults.
• Custom provider supplied: the plugin must combine that provider with the defaults before passing the complete list into DataDesigner. It called get_default_providers() during this earlier step. That function only reads model_providers.yaml; it does not initialize the file. On a new machine or clean CI runner, the file did not exist yet, so provider resolution raised FileNotFoundError before the constructor that would have created it could run.

In sequence, the failing path was:

build_model_providers(custom provider)
  -> get_default_providers()
  -> read ~/.data-designer/model_providers.yaml
  -> FileNotFoundError
  -> DataDesigner(...) is never reached

The corrected path uses Data Designer's built-in provider definitions when the user-level file is absent. These are the same definitions that the DataDesigner constructor would write to the file. Once DataDesigner is constructed, its normal initialization still creates the user-level file. Existing user files continue to take precedence, so this fallback changes only first-run behavior and does not overwrite user configuration.

Why existing tests did not catch it

The bug was latent in v0.1.0, but that release had no CLI or pipeline tests exercising custom-provider resolution. Local development also usually had model_providers.yaml from earlier Data Designer runs.

Regular repository CI masked the problem through test ordering: another plugin's tests constructed DataDesigner first, which initialized the file under the runner's shared home directory. Retrieval SDG's later tests then found the file even though they ran in a separate virtual environment. Virtual-environment isolation does not isolate files under $HOME.

The publication workflow tested retrieval SDG by itself on a fresh runner. Nothing had constructed DataDesigner beforehand, so its clean home directory exposed the ordering dependency. The new regression test explicitly simulates a missing user-level provider file and verifies fallback to the built-in definitions.

The failed GitHub release and tag were removed after confirming that:

• the release had zero uploaded assets;
• ddp-package-assets still contains retrieval SDG 0.1.0 only; and
• packages.json was not updated.

The package version remains 0.2.0. After this PR merges, the release tag will be recreated at the corrected main commit.

Validation

• Full retrieval SDG suite: 137 passed
• Python 3.10 clean-home rerun of the five publication failures plus the new regression: 6 passed
• Ruff lint and formatting checks
• Fresh-worktree make validate-release PLUGIN=data-designer-retrieval-sdg
• Data Designer 0.9.1 selected in the isolated environment

Failed publication run: https://github.com/NVIDIA-NeMo/DataDesignerPlugins/actions/runs/31535353100