Skip to content

ci: 3-way test-account parallelism + consumer-smoke gets -n 4 - #97

Merged
saqibmanan merged 2 commits into
CIfrom
ci-consumer-smoke-parallel
Sep 17, 2026
Merged

saqibmanan merged 2 commits into
CIfrom
ci-consumer-smoke-parallel

Conversation

@saqibmanan

Copy link
Copy Markdown
Contributor

Two changes, landing together since both touch the same worker/credential system.

consumer-smoke: -n 4

Was fully serial — the ~5-6 min chunk of runtime. Nothing in tests/consumer/ needs a real login (no test_credentials/auth_token usage anywhere in it), so unlike api-smoke/provider-smoke this was never bounded by account count, just never given -n at all.

Verified locally: -n 4, 55 passed in ~2 min, down from serial.

3rd test account (test_1@civicdatalab.in → TEST_EMAIL_3)

Account created and verified live (real Keycloak token, HTTP 200) before this PR — wired in here, not provisioned here.

  • workflow_call secrets schema: TEST_EMAIL_3/PASSWORD_3, optional, matching TEST_EMAIL_2's existing declaration.
  • api-smoke: added TEST_EMAIL_3/PASSWORD_3, and separately TEST_EMAIL_2/PASSWORD_2 — found while doing this that api-smoke has been running -n 2 without ever forwarding TEST_EMAIL_2 at all. gw1 has been silently falling back to TEST_EMAIL_1 this whole time (test_credentials does that for any worker slot with no matching credentials, rather than failing). Fixed alongside adding _3, same category of gap.
  • provider-smoke: added TEST_EMAIL_3/PASSWORD_3 alongside the existing _1/_2 forwarding.
  • Both bumped -n 2 → -n 3.

org_add_permission docstring corrected, not just extended

Already stale before this PR — claimed TEST_EMAIL_2 was "a lone auditor on CivicDataLab" with canAdd=false everywhere. Its role changed to admin on a different org ("my test agency") partway through this work. Verified live via the exact GraphQL query the fixture itself uses, for all three accounts:

  • TEST_EMAIL_1: admin/canAdd on 11 orgs (CivicDataLab and 10 others)
  • TEST_EMAIL_2: admin/canAdd on "my test agency" only; still auditor/canAdd=false on CivicDataLab
  • TEST_EMAIL_3: admin/canAdd on "my test agency", same as TEST_EMAIL_2 — already write-capable from creation

No test code changes needed — org_add_permission already resolves this dynamically from the live permissions query, nothing hardcodes an org name. Rewrote the docstring to state the verified facts and flag that it goes stale the moment a role changes.

What this does NOT change

The org-create flows this unblocks (test_prv_006 through 011) are functional-marked, not smoke — confirmed via collection, only test_prv_001_login_smoke carries the smoke marker in tests/provider/smoke/. So this adds no real-data-creation load to the default push/PR path; those flows already only ran on workflow_dispatch/workflow_call, unchanged by this PR.

Verified

  • actionlint: zero findings.
  • consumer-smoke -n 4: 55 passed, ~2 min (real run against dev).
  • api-smoke -n 3, all three TEST_EMAIL_* forwarded: 26 passed, 7 skipped (all pre-existing/expected locally), no new failures.
  • provider-smoke test_prv_001_login_smoke -n 3: 1 passed (only file at this marker, so only gw0 got scheduled work — but all three accounts independently confirmed able to obtain a real Keycloak token before this PR, the actual risk surface for the credential change).

TEST_EMAIL_3/PASSWORD_3, matching the existing User 1/User 2 pattern.
Two independent changes, landing together since both touch the same
worker/credential system.

## consumer-smoke: -n 4

Was fully serial (~5-6 min of the suite's runtime). Nothing in
tests/consumer/ needs a real login (no test_credentials/auth_token
usage anywhere in it), so unlike api-smoke/provider-smoke this was
never bounded by account count -- just never given -n at all. Verified
locally: -n 4, 55 passed in ~2 min, down from serial.

## 3rd test account (test_1@civicdatalab.in / TEST_EMAIL_3)

Account already created and verified live (real Keycloak token, HTTP
200) before this PR -- wired in, not provisioned here.

- workflow_call secrets schema: TEST_EMAIL_3/PASSWORD_3, optional,
  matching TEST_EMAIL_2's existing declaration.
- api-smoke: added TEST_EMAIL_3/PASSWORD_3, and separately TEST_EMAIL_2/
  PASSWORD_2 -- found while doing this that api-smoke has been running
  -n 2 without ever forwarding TEST_EMAIL_2 at all. gw1 has been
  silently falling back to TEST_EMAIL_1 this whole time (test_credentials
  does that for any worker slot with no matching credentials, rather
  than failing). Fixed alongside adding _3, not filed separately --
  same category of gap, cheap to fix while already in this exact code.
- provider-smoke: added TEST_EMAIL_3/PASSWORD_3 alongside the existing
  _1/_2 forwarding.
- Both bumped -n 2 -> -n 3.

## org_add_permission docstring corrected, not just extended

It was already stale before this PR: claimed TEST_EMAIL_2 was 'a lone
auditor on CivicDataLab' with canAdd=false everywhere. User changed
TEST_EMAIL_2's role to admin on a different org ('my test agency')
partway through this work. Verified live via the exact GraphQL query
the fixture itself uses, for all three accounts:
- TEST_EMAIL_1: admin/canAdd on 11 orgs (CivicDataLab and 10 others)
- TEST_EMAIL_2: admin/canAdd on 'my test agency' only; still
  auditor/canAdd=false on CivicDataLab
- TEST_EMAIL_3: admin/canAdd on 'my test agency', same as TEST_EMAIL_2
  -- already write-capable from creation

No test code changes needed -- org_add_permission already resolves
this dynamically from the live permissions query, nothing hardcodes an
org name. Rewrote the docstring to state the verified facts and warn
that it will go stale again the next time a role changes.

## What this does NOT change

The org-create flows this unblocks (test_prv_006 through 011) are
functional-marked, not smoke -- confirmed via collection, only
test_prv_001_login_smoke carries the smoke marker in tests/provider/
smoke/. So this doesn't add real-data-creation load to the default
push/PR path; those flows already only ran on workflow_dispatch/
workflow_call, unchanged by this PR.

## Verified

- actionlint: zero findings.
- consumer-smoke -n 4: 55 passed, ~2 min (real run against dev).
- api-smoke -n 3, all three TEST_EMAIL_* forwarded: 26 passed, 7 skipped
  (all pre-existing/expected for a local run), no new failures.
- provider-smoke test_prv_001_login_smoke -n 3: 1 passed (only file at
  this marker, so only gw0 got scheduled work -- but all three accounts
  independently confirmed able to obtain a real Keycloak token before
  this PR, which is the actual risk surface for the credential change).
@saqibmanan
saqibmanan marked this pull request as ready for review September 17, 2026 18:05
@saqibmanan
saqibmanan merged commit a5c277f into CI Sep 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant