Ship the 2026-08-23 API contract: preference variation images + writable audience exclusions (0.9.0)

[benzend]

Ships against the 2026-08-23 Public API release. Two gaps, both surfaced while building a four-cell monadic study from the API.

Preference options are variations, and can carry an image

A preference question's options are PreferenceVariation records — one full-size image each — not Choice records. GET /tests/:id has always read them back under variations, and as of this release that is the key writes take too.

The CLI could only ever send option names, so every variation came back asset_id: null and the test could never clear its "Add an image" launch blocker.

• New --variations <json|@file> on tests add-question and tests edit-question. Takes plain strings, objects {name (or text), asset_id?, site_link?}, or a mix.
• --choices still works on a preference question as a legacy alias, so existing scripts keep running. Passing both is rejected locally, and errors are named for whichever key the caller actually used (variations[0] vs choices[0]).
• variations on any other question type is rejected with a pointer to choices, rather than silently dropped.
• The parallel asset_ids array — which the API used to accept with a 200 and ignore — is now rejected by name, pointing at per-variation asset_id.
• An option mirrored back out of a GET gets one error listing its read-only keys, not one error per key. (Mirroring a GET into a write is the exact anti-pattern this whole change came from, so it's worth a legible message rather than a wall.)

Read-back gap this exposed

preview and walkthrough both read options off variations[0].choices, which is empty on a preference question — so a preference question previewed with no options at all. Both now list the options and flag any with an empty image slot, since that's a launch blocker the participant view gives no hint of.

exclude_test_ids is writable

Previously readable but not writable, so a run of tests that must not share participants had to be wired up by hand in the editor.

• --exclude-tests <ids...> on tests create and tests update; --clear-exclude-tests on update.
• On update it deliberately does not require --audience-type — unlike --audiences, the API applies exclusions to whatever quota the test already has.

The three API gates are split by what the CLI can actually see:

Gate	Treatment	Why
Panel audiences only	Error — when the same command sets --audience-type	Locally checkable; caught by --dry-run
Duplicate ids, self-exclusion	Error	Locally checkable
beta_group flag	Warning	Account-level; invisible to the CLI
Cap of 5 (30 for internal_group)	Warning	Erroring would falsely block an account that legitimately gets 30

The warning treatment matches how the Enterprise branching gate is already surfaced. --clear-exclude-tests bypasses everything, mirroring the API's own exemption so a test is always undoable.

Reviewer notes

Breaking (JSON output shape) — called out in the changelog:

• tests preview --output json emits variations instead of choices for preference questions. The old choices key was always [] on these, and the new shape is what a write takes, so preview output now feeds straight back into create/add-question. Every other type is unchanged.
• tests walkthrough --output json gains preference_options on every question screen (null on non-preference types).

Verification — 332 tests pass (up from 292); typecheck and build clean. New coverage for preference variation validation, the preferenceOptions read-back, walkthrough preference rendering, the exclusion validator/warnings, and guide-JSON contract assertions. Each new path was also exercised against the built binary: schema output, dual-key rejection, GET-mirrored payloads, wrong-type variations, exclusions on panel vs. open audiences, self-exclusion, and flag conflicts.

One gap worth naming: the printSectionQuestions text renderer for preference has no unit test. Its logic (preferenceOptions) and its walkthrough twin are both tested, and every console printer in this file is untested by existing convention — but that path was verified by inspection only, not execution, since it needs a live API response.

Also folded in: tests question-types --type <type> now prints the type's notes in text mode. JSON consumers always saw them; text mode dropped them, which hid rules that live nowhere else — including the variations/choices aliasing this PR introduces.

🤖 Generated with Claude Code