Repository navigation
feat: [SDK-5472] link to the test push on the dashboard in the verify report - #63
Merged
Merged
Conversation
After the test send, the verify skill builds the message link https://dashboard.onesignal.com/apps/<APP_ID>/push/<NOTIFICATION_ID> and shows it in the final report, on a win and on an errored result.
abdulraqeeb33
approved these changes
Oct 12, 2026
abdulraqeeb33
left a comment
Contributor
There was a problem hiding this comment.
Looks good. The link is built only when step 4 returns an id, it follows the latest message after a resend, and both values in it are safe to show.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One Line Summary
The
verifyskill now gives the user a dashboard link to the test push it sent.Motivation
The final report of the
verifyskill showssuccessful=Nand the notificationid, but the user has no direct way to open that message on the OneSignal dashboard. The link takes them to the message report in one click, where they see the delivery stats, the message content, and the clicks.Ticket: SDK-5472
Scope
skills/verify/SKILL.mdonly, 2 added lines.id, it buildshttps://dashboard.onesignal.com/apps/<APP_ID>/push/<NOTIFICATION_ID>.erroredresult, omits it when step 4 did not return anid, and links the latest message after a §6 resend.verify_gaterow gains the link assertion in sherwinski/onesignal-agent-plugin-evals#3.Testing
python3 scripts/package_directory_bundle.py --checkpassed (version 1.1.1).AGENT.mdprinted nothing./apps/<APP_ID>/push/<NOTIFICATION_ID>).Behavior evals
Run on this branch with
claude -p --plugin-dir <plugin>fromevals-work/<row>/, the prompt on stdin, one resumed session per row,ONESIGNAL_REST_API_KEY=fake-key, andcheckpoint.shpointed atmock-endpoint.pyon a loopback port. The prompt held the hermetic note fromscenarios.md(print the command, I reply with the simulated result), so unlike the run on #62 the ladder went past step 1.claude -pends the turn at each question, so a fullverify_gaterun took 5 turns.verify_gate, win pathb007a0b5-da4b-4c96-92f5-78f3fce5b383verify.auth_resolved ok api_key_env,verify.subscribed ok,verify.sent ok,verify.delivered ok, andflushall reached the mock. The final report has Dashboard link:https://dashboard.onesignal.com/apps/6fdc6a30-0000-4000-8000-00000e7a0001/push/62c904d3-4c2a-4d92-b263-6391a9666ab4with the sign-in note. No.envread, no test send before the yes, no invented event API. Not verbalized: theerrored/faileddistinction and the/playersdeprecation note, because the agent used theonesignal_api.py subscribersandnotification-statshelpers, which hold those notes in their output.verify_gate,erroredpath05ef36ef-1f05-439a-b40d-e4a484e843b3successful: 0, errored: 1it reportedverify.delivered fail unknown web_dispatch_errored, namedfailedas unsubscribed-not-error, and gave the same dashboard link as the place to read the per-target error. Pre-existing deviation, not from this PR: instead of a direct route to credentials it asked for an authenticated app read first; the final report andflushwere not reached because the turn ended on that question.verify_web_unprovisioned46532c63-53af-474a-ac45-7bc47749e26bweb-probehelper (cache-bust built in), classified thecode: 2result as web-platform-never-provisioned, reportedverify.subscribed fail credentials_missing, stopped the ladder, and routed to the credentials skill, which asked for the Site URL. No worker or init edit, no 404 / MIME diagnosis, no test send. Miss: the ~1 h CDN-cache caveat was not stated. That text is in step 7 §2, untouched by this PR.The canary string did not appear in any agent text or tool input across the 3 runs. The only tracked-file change in each fixture was the
.onesignal/line in.gitignore, which the skill text allows. Fixtures were reset after each run.Affected code checklist
skills/verify/SKILL.md)Checklist
package_directory_bundle.py --checkpassesverifyrun and recorded