Repository navigation
test(e2e): run the testkit conformance suite in the host lanes - #5078
Draft
matthewgrossman wants to merge 2 commits into
Draft
matthewgrossman wants to merge 2 commits into
matthewgrossman wants to merge 2 commits into
Conversation
The Docker, Podman, Kubernetes and VM host lanes, including external-driver mode and e2e:rust:push, now run the testkit `sandbox` package with nextest instead of the old CLI conformance package. The tests find the lane's gateway the same way the CLI does, through the OPENSHELL_GATEWAY_ENDPOINT and OPENSHELL_GATEWAY values the gateway wrappers export, and `--no-tests fail` makes a lane that selects no tests fail. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Nothing runs tests/suites/conformance/cli now that the host lanes run the testkit suite. Remove it from the conformance workspace and its lockfile, drop the e2e:testkit exclusion for it, and point the TESTING.md manual run at the `sandbox` package. e2e:rust:push also loses its dependency on e2e:conformance:build, which no longer exists, and the unused OPENSHELL_CONFORMANCE_BIN it set. crates/openshell-conformance stays for the features and drivers suites. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Member
Author
|
/ok to test e0d2385 |
|
🌿 Preview your docs: https://nvidia-preview-pr-5078.docs.buildwithfern.com/openshell |
|
Label |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Run the testkit conformance tests (
tests/suites/conformance/sandbox) in the host E2E lanes, and delete the old CLI conformance suite they replace. This is a parity change: every lane keeps its current gateway and driver setup.Based on #4774. The base branch is #4774's branch until it merges, then this PR retargets
main.Related Issue
Part of #3954
Changes
e2e/support/conformance.shrunscargo nextest run --package sandboxintests/suites/conformancewith--no-tests fail, so a lane that selects zero tests fails. All host lanes (Docker, Docker external-driver, Podman, Kubernetes and its variants, VM, VM external-driver, ande2e:rust:push) go through this script, so none of the lane wrappers changed.tests/suites/conformance/cli, its workspace member, and its lockfile entries.crates/openshell-conformancestays for the suites that still use it.tasks/test.toml:e2e:testkitno longer excludes the deleted suite.e2e:rust:pushdrops a dependency one2e:conformance:build, a task that no longer exists.TESTING.md: updated the suite and lane descriptions and the manual-run command.Gateway selection per lane:
OPENSHELL_GATEWAYfor an mTLS gateway registered in a temporary config folder.OPENSHELL_GATEWAY. The gateway is registered without TLS, or with mTLS for the TLS and OpenShift variants. Operator workspace mode still skips conformance.OPENSHELL_GATEWAY_ENDPOINT. testkit matches it to the active registered gateway and uses that gateway's mTLS certificates.Testing
cargo fmt,cargo clippy -D warnings(sandbox package), lockfile alignment,markdownlint-cli2 TESTING.md,mise run license:check, shellcheck (no new warnings)Local runs, 15/15 conformance tests passing in each:
VM gateway selection was checked by recreating the VM lane's environment on Docker (only
OPENSHELL_GATEWAY_ENDPOINTset). Running with a filter that matches no tests exits non-zero.Not verified locally: the real VM and VM external-driver lanes, Kubernetes external-driver, and the Kubernetes TLS and OpenShift variants. The Docker E2E CI image does not include
cargo-nextest.miseinstalls the pinned version in that image (checked locally), at the cost of one download per job.Checklist