Skip to content

feat: add optional Jev skill routing - #135

Draft
MelodyVAR wants to merge 1 commit into
mainfrom
feat/jev-skill-routing
Draft

MelodyVAR wants to merge 1 commit into
mainfrom
feat/jev-skill-routing

Conversation

@MelodyVAR

Copy link
Copy Markdown
Collaborator

Step Code can miss a relevant skill or read an unnecessary one when choosing from a large catalog. Add optional Jev suggestions before text requests, enabled with STEP_JEV_SKILL_ROUTING=1 and a separate TypeSafe API key.

  • Rank eligible skill descriptions, then verify up to three bounded excerpts with typed TypeSafe requests. Preserve the complete catalog and explicit skill commands. Use the existing prompt on uncertainty, errors, or the shared 1.5-second deadline.
  • Include before_agent_start in the existing cancellable session lifecycle and expose its existing ctx.signal. Cancelling or disposing the session stops pending routing and prevents later requests or coding-model startup. Runtime shutdown settles cancellation before invalidating extension contexts.
  • Document setup, content sent to TypeSafe, separate billing, and the limits of upstream efficiency evidence. Add English and Chinese README links.

Validation on Node 22.19.0:

  • npm run check and pnpm run build:offline passed.
  • All 219 Jev-related cases passed, including real AgentSession cancellation/disposal regressions with blocked transports, late responses, and lifecycle ordering.
  • The isolated CLI suite passed: 84 files / 625 tests.
  • ./test.sh passed the script tests and other workspace packages; coding-agent reported 275 passed files / 3,671 passed tests, plus the single baseline failure below. The recursive command stops at that failure, so CLI was run separately against the final build.

./test.sh is not fully green in this environment: test/startup-session-name.test.ts:117 hits its existing 10-second child-process timeout (expected exit 1, received null after SIGKILL). The same test fails on the clean base commit e411b1ab497b6ef773a49b86d5237d3c2a4b643c. A longer observation confirms that the startup child exits with code 1 after 27.03 seconds and writes the expected session name. This change leaves the existing timeout and assertions intact.

TypeSafe transport is mocked in automated tests; live Jev accuracy, production latency, Step Code token savings, and usage growth have not been measured.

Use typed TypeSafe requests to rank installed skills and verify a bounded shortlist before text tasks. Keep routing opt-in, preserve the catalog, and fail open within one shared deadline.

Include prompt preparation in session cancellation, stop routing during disposal, and document external data, billing, and the limits of upstream efficiency evidence.

@ZenAlexa ZenAlexa left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the three Jev routing suites: 215 tests passed. The CLI bootstrap test passed. pnpm run check passed all repository checks on Node 22.23.2.

The cancellation path is wired from the per-run controller through before_agent_start and runtime shutdown. The tests cover late HTTP responses, deferred JSON bodies, excerpt reads, direct session disposal, runtime disposal ordering, and a fresh prompt after cancellation. The request body keeps the external payload limited to the documented request and skill metadata or excerpts.

I have no actionable finding on this head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants