Skip to content

Finish the cutover to source-generated schemas and publish missing version folders #1673

Description

@Gijsreyn

Summary of the new feature / enhancement

As a configuration author or resource developer using the published DSC schemas,
I want the schemas to publish for every released version and stay in sync with the engine automatically,
So that editor validation and IntelliSense help me instead of contradicting DSC.

Currently the published schemas under schemas/ are built from hand-maintained YAML sources (schemas/src via schemas/build.ps1) that were last generated for v3.1.0, while the engine has moved on to 3.4.

The most visible symptom is that recognized $schema URIs don't resolve. The engine accepts every version through v3.2.3 as a valid $schema value, but no schema folder was ever published after v3.1.0, so a document pinned to a URI like this validates in DSC and 404s in the editor:

$schema: https://aka.ms/dsc/schemas/v3.2/bundled/config/document.json

The YAML sources have also drifted from the engine — fields, enum values, and whole subsystems added since v3.1.0 aren't represented, and in several cases the published schemas reject manifests this repository ships. Those content gaps will be filed as individual issues; this issue tracks the underlying problem that makes them inevitable.

That underlying problem is that the repository has two schema systems mid-migration. The YAML pipeline is what publishes, but every release requires manually editing schemas.config.yaml, re-running build.ps1 per version folder, and hand-updating the $schema URI enums in the sources — steps nothing enforces, which is how seven releases shipped without schemas. Meanwhile the source-based generator from #538/#1406 (DscRepoSchema + cargo xtask schema export) derives schemas directly from the engine's types — making drift structurally impossible — but is nearly complete rather than complete: it has never published anything, and no CI exercises either system.

Proposed technical implementation details (optional)

Finish the cutover to generated schemas in phases, each independently reviewable:

  1. Exporter correctness ((GH-538) Enhance schema export functionality and add error handling for duplicate paths #1672): fix the two silent path collisions (AdaptedDscResourceManifest, DeleteWhatIfResult), export every type that derives DscRepoSchema, fail on duplicate output paths, add --schema-version/--release targeting, and route dsc schema through the same machinery so the CLI and exporter agree.
  2. Bundling and namespace parity: expand should_bundle to match the 24 bundled schemas the YAML pipeline publishes today, and decide the disposition of each YAML-only schema (resource/stdout/*, resource/properties/*, metadata/Microsoft.DSC/*), with $ref alias stubs in the floating version folders for moved paths.
  3. Authoring parity: port the title/description/VS Code keywords from the YAML sources onto the Rust types using the existing schema_i18n! pattern, with a comparison report gating the cutover so generated schemas don't regress the editor experience.
  4. CI: commit schemas/vNext, regenerate it in CI and fail on drift, and validate the repository's own example configurations and manifests against the generated schemas.
  5. Backfill: publish the missing v3.1.1v3.2.3 folders from each release tag's own sources so every recognized URI resolves.
  6. Cutover: the next stable release publishes its folders via cargo xtask schema export --release, after which schemas/src and schemas/build.ps1 can be retired.

The CI checks (phase 4) fit naturally as a Pester test in dsc/tests, so they run in the existing ./build.ps1 -Test -PesterTestGroup dsc job as well as in a dedicated schemas workflow that invokes cargo xtask schema export first. Something like:

BeforeDiscovery {
    # Every (folder_path, base_name) pair declared with #[dsc_repo_schema(...)]
    $declarations = Get-ChildItem lib/dsc-lib/src -Recurse -Filter '*.rs' | ForEach-Object {
        [regex]::Matches(
            (Get-Content $_.FullName -Raw),
            '(?s)base_name\s*=\s*"(?<base>[^"]+)"\s*,\s*folder_path\s*=\s*"(?<folder>[^"]+)"'
        ) | ForEach-Object {
            @{ folder = $_.Groups['folder'].Value; base = $_.Groups['base'].Value }
        }
    }
}

Describe 'exported schemas' {
    It 'exports <folder>/<base>.json' -TestCases $declarations {
        Join-Path 'schemas/vNext' $folder "$base.json" | Should -Exist
    }

    It 'has no uncommitted drift' {
        git diff --exit-code -- schemas/vNext
        $LASTEXITCODE | Should -Be 0 -Because 'run `cargo xtask schema export` and commit the result'
    }
}

Describe 'instance validation' {
    It 'validates <name> against the generated manifest schema' -TestCases $manifests {
        Test-Json -Path $path -SchemaFile 'schemas/vNext/bundled/resource/manifest.json' | Should -BeTrue
    }
}

The first block also catches the "derives DscRepoSchema but was never exported" class of bug statically, and the instance validation is the regression proof for the drift issues: manifests in this repository that fail against the published v3.1.0 schemas today must pass against the generated ones.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions