Repository navigation
CI and validator cover manifests only: skills, hooks, mcp.json, agents and the script test suites are unchecked #531
Description
Summary
The validator and the workflow that runs it leave most of what plugins ship unchecked. Collected while going through the repo last week; filing as one issue so the pieces can be picked off separately.
What validate-plugins.yml covers today
- Trigger
paths:is.cursor-plugin/marketplace.json,**/plugin.json,schemas/**. A PR that only touchesSKILL.md,hooks.json,mcp.json, agents or scripts runs no CI at all. - Dependencies are
npm install --no-save ajv ajv-formatswith no lockfile, so every run resolves whatever Ajv major is current. scripts/validate-plugins.mjschecks the two manifests against the schemas and thatnamematches the marketplace entry. It does not check thatskills/agents/rules/hooks/mcpServers/logopaths exist, that SKILL.md or agent frontmatter parses, that${VAR}placeholders inmcp.jsonare declared invariables, or that every directory with aplugin.jsonis listed in the marketplace.
What that has let through
agent-compatibility/skills/check-agent-compatibility/SKILL.mdfrontmatter is invalid YAML (unquoted:indescription);npx skillsskips it with a parse warning (check-agent-compatibility: SKILL.md description has an unquoted colon, YAML parse fails #381, PRs Fix invalid YAML frontmatter in check-agent-compatibility skill #391 and fix(agent-compatibility): quote check-agent-compatibility description #465).ralph-loop/hooks/hooks.jsonaddressed its scripts as./hooks/*.sh, the layout fix(advisor): address hook scripts via CURSOR_PLUGIN_ROOT #315 identified aspublic-invalidfor the registry (fix(ralph-loop): address hook scripts via CURSOR_PLUGIN_ROOT #524).cursor-sdkskill description over the 1024-char truncation limit (chore: keep the cursor-sdk skill description under the 1024-char limit; label agent-compatibility changelog 1.0.0 #530).orchestrate/skills/orchestrate/scripts(28*.test.ts,bun test+tsc) andpstack/skills/poteto-mode/scripts(bun test orch watch-pr) have test suites that CI never runs, so regressions in those scripts are only caught by contributors who run them locally.
Suggested shape
- Broaden
paths:to the whole tree (or drop the filter); the job takes seconds. - Add a root
package.json+ lockfile withajv/ajv-formatspinned and usenpm ci. - Extend the validator: path existence for component fields, frontmatter parse + required
name/descriptionforSKILL.mdandagents/*.md,mcp.jsonplaceholders vsvariables, unlisted plugin directories. Validate SKILL.md frontmatter in CI #440 already proposes the SKILL.md part with askill.schema.json; it is conflicting now but the approach fits. - A second job running
bun test/tscin the two script packages.
Activity
saleh98salehsleem1198-ai commentedon Oct 10, 2026
Anything ok
akushonkamen commentedon Oct 12, 2026
Taking this one. Confirmed the root cause from the current default branch:
scripts/validate-plugins.mjsonly callsexistsSyncon the manifest files themselves (lines 37/57/65 — marketplace path, plugin dir, plugin.json), so declaredskills/agents/rules/hooks/mcpServers/logopaths are never checked; and.github/workflows/validate-plugins.ymltriggers on only 3 path globs (marketplace.json,**/plugin.json,schemas/**) with unpinnednpm install --no-save ajv ajv-formats.Plan (per the pieces in this issue): first PR = broaden workflow
paths:+ add path-existence checks in the validator; follow-ups separately for frontmatter parsing,${VAR}⊆variablescross-check, unlisted plugin dirs, and a lockfile.Heads-up: this work is assisted by an AI coding agent, so the PR description will say so.
akushonkamen commentedon Oct 12, 2026
Opened #582 with the first two pieces of the suggested shape: the
paths:filter dropped fromvalidate-plugins.yml, and path-existence checks for declaredskills/agents/rules/commands/hooks/mcpServers/logopaths in the validator (plain paths must exist, trailing-slash paths must be directories, globs need an existing static base directory; URL logos and inline objects are skipped). All 100 marketplace plugins pass green; pieces 3–6 left for follow-ups.deanrie commentedon Oct 13, 2026
Hey @akushonkamen, thanks for picking this up, and #582 matches the first two pieces of the shape above. I checked it out on top of current
main:node scripts/validate-plugins.mjsis green across every listed plugin, and adding askills/does-not-exist/entry to aplugin.jsonfails withskills path "skills/does-not-exist/" does not exist, so the check does what it says. The glob handling (require only the static base dir) is the right call forskills/*style entries.One thing worth knowing for whoever merges: because the
paths:filter is still onmain, the Validate job did not run on #582 itself (it only touches the workflow and the script, neither of which matches the filter). It will start running on every PR once it lands.Still open from the list, for anyone who wants a piece: (2) root
package.json+ lockfile withajv/ajv-formatspinned andnpm ci; (3b) frontmatter parse with requiredname/descriptionforSKILL.mdandagents/*.md(#440 has askill.schema.jsonapproach, currently conflicting); (3c)mcp.json${VAR}placeholders vsvariables; (3d) plugin directories not listed in the marketplace; (4) a job runningbun test/tscin the two script packages (#492 just added a bun test underpstack/skills/poteto-mode/scriptsthat nothing runs yet).saleh98salehsleem1198-ai commentedon Oct 13, 2026
Ok