Skip to content

Test-TestEnvironment: verify the seeded estate against the seed data - #41

Merged
fadwen merged 2 commits into
mainfrom
feat/test-environment
Sep 14, 2026
Merged

fadwen merged 2 commits into
mainfrom
feat/test-environment

Conversation

@fadwen

@fadwen fadwen commented Sep 14, 2026

Copy link
Copy Markdown
Owner

What

One exported command, Test-TestEnvironment, dispatching to Test-<Provider>Environment for all six providers. It compares what the connected directory holds with the seed files:

  • every seeded user, group and other named object present, found the way teardown finds them (Get-<Provider>SeededObject, Select-ADTestOwnedObject), and nothing the module owns that the data does not describe;
  • every name equal to the data by codepoint ([string]::Equals(..., Ordinal), never -eq, which calls a decomposed and a precomposed name equal);
  • every membership the data lists, or its rules define, in place. Memberships are judged on what is missing only, because dynamic groups, AD group rules and FreeIPA automember rules add members the data never lists;
  • licence-gated and many-to-one object types counted without a verdict.

One result object for every provider (Provider, Target, VerifiedOn, Checks, Failed, Passed), each check with Name, Kind, Expected, Found, Missing, Unexpected, Passed and the full lists. -SkipMembership skips the expensive reads; -Quiet returns the object alone.

Verify/Invoke-LiveCycle.ps1 runs seed, verify, teardown, re-verify through the exported commands and exits non-zero on any finding. Nothing under Verify/ ships; the rehearsal confirms it.

Live results

Provider After seed After teardown
AD all 6 checks passed (311 users, 25 service accounts, 688 computers, 90 groups, 5,687 memberships) nothing found
Authentik all 5 checks passed (330 users, 98 groups, 9 apps, 3,886 memberships) nothing found
FreeIPA all 8 checks passed (355 + 1 staged + 1 preserved users, 102 groups, 413 hosts, 3,917 + 817 memberships) nothing found
PingOne all 8 checks passed (330 users, 4 populations, 11 groups, 6 apps, 2 resources, 464 memberships) nothing found
Entra honest: users, groups, apps, memberships all present and every name matching; 694 devices refused by the tenant (Authorization_RequestDenied on device create, pre-existing), 1 of 4 guests 3 custom role definitions kept by design (no P2 to read eligibilities), otherwise clean after Graph's read lag

The first FreeIPA run caught a real verifier bug (logins prefixed where the seed never prefixes them: 355 missing, 355 unexpected against a correct realm). Fixed, and the per-provider suites now build their fixtures from the seed files by the seed's own rules so that class of drift fails in CI.

Gates

  • Unit suite: 2,586 passed on Windows PowerShell 5.1, 2,585 passed shuffled on PowerShell 7 (1 skipped, Windows-only ACL test), 0 failed.
  • ScriptAnalyzer clean across the repository. MAML rebuilt. Publish-Module.ps1 -WhatIf passes.

Also

  • Docs page, index entry, about topic, README, provider READMEs, Tests/README and CLAUDE.md invariant.
  • New suites: the Core check/verification helpers, the dispatcher (with a contract that every provider folder implements Test-<Provider>Environment), and one per provider verifier.

…d data

One exported command, dispatching to Test-<Provider>Environment for every provider,
compares what the connected directory holds with the seed files: every seeded user,
group and other named object present and found the way teardown finds them, nothing
the module owns that the data does not describe, every name equal by codepoint, and
every membership the data lists or its rules define in place. Memberships are judged
on what is missing only; licence-gated and many-to-one types are counted without a
verdict. One result object for every provider, built by two Core helpers, with the
full lists of missing and unexpected names.

The AD verifier evaluates the membership rules over the seeded directory the way the
seed does. The Entra verifier expects the schema application the extension step
creates. The FreeIPA verifier compares logins unprefixed, which the first live run
against the realm caught.

Verify/Invoke-LiveCycle.ps1 runs seed, verify, teardown and re-verify through the
exported commands and is not shipped. Docs page, MAML, README, about topic, provider
READMEs and CLAUDE.md updated; suites for the helpers, the dispatcher and each
provider verifier.
@fadwen
fadwen merged commit 186c72d into main Sep 14, 2026
4 checks passed
@fadwen
fadwen deleted the feat/test-environment branch September 14, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant