Skip to content

fix: [SDK-5443] name the structured-question tool and its Codex fallback in skill text - #60

Merged
sherwinski merged 3 commits into
sherwin/sdk-5457from
sherwin/sdk-5443
Oct 12, 2026
Merged

sherwinski merged 3 commits into
sherwin/sdk-5457from
sherwin/sdk-5443

Conversation

@sherwinski

Copy link
Copy Markdown
Collaborator

One Line Summary

Name AskUserQuestion / request_user_input inline in each skill, add the call rules and the Codex Default-mode fallback to safety contract §14, and document the Codex behavior in the README.

Linear: SDK-5443

Motivation

Dogfood feedback: in Codex, the plugin's questions show as free text instead of selectable choices, and sometimes the choices appear and then disappear.

The skills asked for "the harness's native structured-question tool" and pointed to §14 for the name. Codex loads SKILL.md and reads a reference only when the model follows the link. In the Codex CLI, Default mode lists only request_user_input_async; the model called it (9 times in the Oct 6 dogfood session), the CLI rendered the call as text bullets, and the follow-up prose named no reply word. Users typed yes to a send/keep question.

The Codex app renders the same async call as a question card, so the card already worked there. A selectable prompt in the CLI needs the sync request_user_input, which Codex lists only in Plan mode, and Plan mode does not edit files. No skill text can change that.

Scope

  • references/safety-contract.md §14: call the tool by name; 2 to 3 options and 3 questions per call; recommended option first with "(Recommended)"; no own "Other"; in Codex Default mode use request_user_input_async when the sync tool is absent, then end the turn with a plain-text line that names the reply words; never continue past {"accepted":true}.
  • skills/setup/SKILL.md, skills/credentials/SKILL.md, skills/verify/SKILL.md: the first ask names both tools inline. Setup's platform pick and verify's test-send consent are structured questions.
  • README.md, "Install in Codex" → "After install": the app shows a card, the CLI shows text with a reply word, Plan mode does not edit files.

Not in scope: the .onesignal/ dirty-tree gate found during testing, tracked in SDK-5470. No version bump: 1.1.1 is already past the last tag (1.1.0).

Testing

Local checks: package_directory_bundle.py --check passed; the portability grep printed nothing; py_compile and json.tool passed on the 4 manifests.

Setup skill against a Kotlin Android fixture (SkillDemo3), plugin installed from the local clone through codex plugin add onesignal@onesignal, cache diffed against the tree before each run:

Session Host Text Result
01a112de-25c4-73a2-8c76-909a9edd2d54 CLI 0.160.1 (Oct 6 dogfood) before request_user_input_async at every gate; bullets then prose with no reply word
01a12320-814a-7dc3-93c2-93c78a92cd6f CLI 0.162.0 intermediate async call; bullets; no reply word
01a12321-479d-7261-bba9-f4494430d155 Codex Desktop intermediate async call; question card
01a1235f-dede-7962-8998-b108c9a919f0 CLI 0.162.0 this PR plain text with "Reply send or keep local"; send continued the run through detection and version pin
01a12360-4eb1-7113-870f-609d84f996e2 Codex Desktop this PR async call, 3 options, Recommended label; card plus "Reply proceed, stash, or abort"
01a12368-0736-7602-889a-ada939403150 CLI 0.162.1, Plan mode this PR sync request_user_input; selectable widget; model declined to edit files in Plan mode

Not run: the behavior evals and Claude Code. The change touches ask wording only, not the ask points, the gates, or the checkpoint calls.

Affected code

  • Skills (skills/)
  • References (references/)
  • README
  • Scripts
  • Manifests / version

Checklist

  • Bundle gate passes
  • No secrets, no internal ticket IDs in skills, scripts, or references
  • README updated for the behavior it describes
  • Version checked (no bump needed)

@kalley kalley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The §14 rules read clearly, and the session table makes the app/CLI split easy to follow. One thing inline I'd fix before merge.

Comment thread skills/verify/SKILL.md Outdated

@abdulraqeeb33 abdulraqeeb33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Kalley on the one-option message question; the other choice lists all fit the 2 or 3 rule. Heads-up: this conflicts with #58 in the README Codex section once #58 merges. Keep #58's bullet when resolving, since it removes codex mcp login.

…ack in skill text

The skills asked for "the harness's native structured-question tool" and
pointed to safety contract §14 for the name. Codex loads SKILL.md and reads
a reference only when the model follows the link, so the tool name was one
hop away. In the Codex CLI the questions showed as free text with no reply
instruction.

Name AskUserQuestion and request_user_input inline at the first ask in each
skill. In §14, add the call rules both hosts accept (2 to 3 options, 3
questions per call, recommended option first, no own "Other"), and the
Codex Default-mode path: use request_user_input_async when the sync tool is
absent, then end the turn with a plain-text line that names the reply words.
The Codex app shows the async call as a question card; the CLI shows text.
Make the setup platform pick and the verify test-send consent structured
questions. Add a README note for Codex users.
…d verify asks as structured

The consent question keeps the listed order, with send first, and has no
recommended option. The verify message ask is a second question in the
same tool call, because a yes/no structured question cannot take a
free-text body. The setup dirty-tree and wired-App-ID asks now say
structured question, the same as the platform pick.
@sherwinski
sherwinski changed the base branch from main to sherwin/sdk-5457 October 12, 2026 18:20
A 1-option question breaks the §14 rule of 2 or 3 options, and
AskUserQuestion rejects it, which fails the whole batched call with the
send consent. Add "I'll type my own message" as the second option.
@sherwinski
sherwinski added this pull request to stack #64 October 12, 2026 18:36
@sherwinski
sherwinski merged commit cedf53b into main Oct 12, 2026
5 checks passed
@sherwinski
sherwinski deleted the sherwin/sdk-5443 branch October 12, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants