ci: 3-way test-account parallelism + consumer-smoke gets -n 4 - #97
Merged
Merged
Conversation
TEST_EMAIL_3/PASSWORD_3, matching the existing User 1/User 2 pattern.
Two independent changes, landing together since both touch the same
worker/credential system.
## consumer-smoke: -n 4
Was fully serial (~5-6 min of the suite's runtime). Nothing in
tests/consumer/ needs a real login (no test_credentials/auth_token
usage anywhere in it), so unlike api-smoke/provider-smoke this was
never bounded by account count -- just never given -n at all. Verified
locally: -n 4, 55 passed in ~2 min, down from serial.
## 3rd test account (test_1@civicdatalab.in / TEST_EMAIL_3)
Account already created and verified live (real Keycloak token, HTTP
200) before this PR -- wired in, not provisioned here.
- workflow_call secrets schema: TEST_EMAIL_3/PASSWORD_3, optional,
matching TEST_EMAIL_2's existing declaration.
- api-smoke: added TEST_EMAIL_3/PASSWORD_3, and separately TEST_EMAIL_2/
PASSWORD_2 -- found while doing this that api-smoke has been running
-n 2 without ever forwarding TEST_EMAIL_2 at all. gw1 has been
silently falling back to TEST_EMAIL_1 this whole time (test_credentials
does that for any worker slot with no matching credentials, rather
than failing). Fixed alongside adding _3, not filed separately --
same category of gap, cheap to fix while already in this exact code.
- provider-smoke: added TEST_EMAIL_3/PASSWORD_3 alongside the existing
_1/_2 forwarding.
- Both bumped -n 2 -> -n 3.
## org_add_permission docstring corrected, not just extended
It was already stale before this PR: claimed TEST_EMAIL_2 was 'a lone
auditor on CivicDataLab' with canAdd=false everywhere. User changed
TEST_EMAIL_2's role to admin on a different org ('my test agency')
partway through this work. Verified live via the exact GraphQL query
the fixture itself uses, for all three accounts:
- TEST_EMAIL_1: admin/canAdd on 11 orgs (CivicDataLab and 10 others)
- TEST_EMAIL_2: admin/canAdd on 'my test agency' only; still
auditor/canAdd=false on CivicDataLab
- TEST_EMAIL_3: admin/canAdd on 'my test agency', same as TEST_EMAIL_2
-- already write-capable from creation
No test code changes needed -- org_add_permission already resolves
this dynamically from the live permissions query, nothing hardcodes an
org name. Rewrote the docstring to state the verified facts and warn
that it will go stale again the next time a role changes.
## What this does NOT change
The org-create flows this unblocks (test_prv_006 through 011) are
functional-marked, not smoke -- confirmed via collection, only
test_prv_001_login_smoke carries the smoke marker in tests/provider/
smoke/. So this doesn't add real-data-creation load to the default
push/PR path; those flows already only ran on workflow_dispatch/
workflow_call, unchanged by this PR.
## Verified
- actionlint: zero findings.
- consumer-smoke -n 4: 55 passed, ~2 min (real run against dev).
- api-smoke -n 3, all three TEST_EMAIL_* forwarded: 26 passed, 7 skipped
(all pre-existing/expected for a local run), no new failures.
- provider-smoke test_prv_001_login_smoke -n 3: 1 passed (only file at
this marker, so only gw0 got scheduled work -- but all three accounts
independently confirmed able to obtain a real Keycloak token before
this PR, which is the actual risk surface for the credential change).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes, landing together since both touch the same worker/credential system.
consumer-smoke:-n 4Was fully serial — the ~5-6 min chunk of runtime. Nothing in
tests/consumer/needs a real login (notest_credentials/auth_tokenusage anywhere in it), so unlikeapi-smoke/provider-smokethis was never bounded by account count, just never given-nat all.Verified locally:
-n 4, 55 passed in ~2 min, down from serial.3rd test account (
test_1@civicdatalab.in→TEST_EMAIL_3)Account created and verified live (real Keycloak token, HTTP 200) before this PR — wired in here, not provisioned here.
workflow_callsecrets schema:TEST_EMAIL_3/PASSWORD_3, optional, matchingTEST_EMAIL_2's existing declaration.api-smoke: addedTEST_EMAIL_3/PASSWORD_3, and separatelyTEST_EMAIL_2/PASSWORD_2— found while doing this that api-smoke has been running-n 2without ever forwardingTEST_EMAIL_2at all.gw1has been silently falling back toTEST_EMAIL_1this whole time (test_credentialsdoes that for any worker slot with no matching credentials, rather than failing). Fixed alongside adding_3, same category of gap.provider-smoke: addedTEST_EMAIL_3/PASSWORD_3alongside the existing_1/_2forwarding.-n 2→-n 3.org_add_permissiondocstring corrected, not just extendedAlready stale before this PR — claimed
TEST_EMAIL_2was "a lone auditor on CivicDataLab" withcanAdd=falseeverywhere. Its role changed to admin on a different org ("my test agency") partway through this work. Verified live via the exact GraphQL query the fixture itself uses, for all three accounts:canAddon 11 orgs (CivicDataLab and 10 others)canAddon "my test agency" only; still auditor/canAdd=falseon CivicDataLabcanAddon "my test agency", same asTEST_EMAIL_2— already write-capable from creationNo test code changes needed —
org_add_permissionalready resolves this dynamically from the live permissions query, nothing hardcodes an org name. Rewrote the docstring to state the verified facts and flag that it goes stale the moment a role changes.What this does NOT change
The org-create flows this unblocks (
test_prv_006through011) arefunctional-marked, notsmoke— confirmed via collection, onlytest_prv_001_login_smokecarries thesmokemarker intests/provider/smoke/. So this adds no real-data-creation load to the default push/PR path; those flows already only ran onworkflow_dispatch/workflow_call, unchanged by this PR.Verified
actionlint: zero findings.consumer-smoke -n 4: 55 passed, ~2 min (real run against dev).api-smoke -n 3, all threeTEST_EMAIL_*forwarded: 26 passed, 7 skipped (all pre-existing/expected locally), no new failures.provider-smoke test_prv_001_login_smoke -n 3: 1 passed (only file at this marker, so onlygw0got scheduled work — but all three accounts independently confirmed able to obtain a real Keycloak token before this PR, the actual risk surface for the credential change).