Repository navigation
test: add behavioral tests for all automation scripts (#25) - #142
Merged
Merged
Conversation
Three scripts ended a block with '[[ cond ]] && cmd', so a false condition became the exit status: check-types exited 1 when the noAnyTypes advisory found 1-10 hits, load-test exited 1 on every successful run without --json, and security-scan aborted under set -e on any issue without a file path. Also keep an empty JSON issue list from tripping set -u on bash 3.2 and strip wc padding from the console.log count. Found by the new script tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Cover setup-project (dry-run structure per platform, multi-tenant, refusal paths, full generation with a fake pnpm), run-tests, check-types, security-scan detectors and audit policy, seed-database, load-test, and the OpenAPI and client generators. Tests shell out to the real scripts against throwaway projects with fake pnpm/npx/k6 shims, so nothing touches the network. Document how to run them in CONTRIBUTING.md. Closes #25 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #25.
Adds behavioral tests for every automation script that did not yet have them, on top of the Vitest harness introduced in #141. Each test shells out to the real script against a throwaway project directory with fake
pnpm,npx, ork6shims onPATH, so nothing touches the network and each file asserts the repo stays clean.Deviation from the issue text: the issue proposed BATS under
tests/scripts/. Since #141 already established a Vitest harness underscripts/__tests__/(the same convention Aurelius uses, and it runs in the existingscript-testsCI job), these tests use that instead of adding a second framework.Bugs the tests found and this PR fixes
Three scripts ended a block with
[[ cond ]] && cmd, so a false condition became the exit status:check-types.shexited 1 whenever thenoAnyTypesadvisory found 1 to 10 hits.load-test.shexited 1 on every successful run without--json.security-scan.shaborted underset -eon any issue without a file path (rate limiting, CORS, logging, lockfile, dependency), skipping the summary and ignoring--no-fail.Also fixed:
security-scan.sh --jsonwith zero issues crashed on macOS bash 3.2 (empty array underset -u), and the console.log count carriedwcpadding.Noted, not changed
generate-openapi-docs.shhardwires its Postman output to the framework root'spostman/rather than the API project, so the collection merge is only tested via its skip path.generate-client.shresolves relative--spec/--outputagainst the framework root.setup-project.shdefines its own colors and ignoresNO_COLOR.Test plan
pnpm test: 16 files, 381 tests passpnpm verifypassesscript-testsjob green🤖 Generated with Claude Code