Skip to content

Commit 47878f1

Browse files
huangyiireneclaude
andauthored
docs(skills): objectstack-ai factual sweep — 9 false behavioral facts corrected (#13867)
Flight 10 of the published-skills factual sweep. Every corrected claim was settled against the implementing code (packages/spec ai + ui schemas, packages/runtime action-execution, packages/mcp skill-prompts, the platform tool/object-name registries and the liveness ledgers), never against another document, with executed parse probes where the claim is behaviour-bearing. Net: -15 tokens, -10 lines across the package; both ratcheted files stay under their existing ceilings (no ceiling touched). Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 88c30af commit 47878f1

2 files changed

Lines changed: 38 additions & 48 deletions

File tree

skills/objectstack-ai/SKILL.md

Lines changed: 36 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ patterns.
5656
Agent → Skill → Tool
5757
│ │ │
5858
│ │ └─ Atomic operation (query, action, flow, API call)
59-
│ └─ Capability bundle with instructions & trigger phrases
59+
│ └─ Capability bundle with instructions & trigger conditions
6060
└─ Autonomous actor with role, instructions, and guardrails
6161
```
6262

@@ -112,24 +112,23 @@ its surface-compatible skills' tools** — there is no global fall-through
112112

113113
The built-in skills and their affinities:
114114

115-
| Skill | `surface` | Owns | Edition |
116-
|---|---|---|---|
117-
| `schema_reader` | `both` | `list_objects`, `describe_object`, `query_data` | OSS |
118-
| `data_explorer` | `ask` | `query_records`, `get_record`, `aggregate_data`, `visualize_data` | OSS |
119-
| `actions_executor` | `ask` | `action_*` (the business actions an object exposes) | OSS |
120-
| `metadata_authoring` + `solution_design` | `build` | metadata draft / verify / publish + blueprint propose / apply | **cloud only** |
115+
| Skill | `surface` | Owns |
116+
|---|---|---|
117+
| `schema_reader` | `both` | `list_objects`, `describe_object`, `query_data` |
118+
| `data_explorer` | `ask` | `query_records`, `get_record`, `aggregate_data`, `visualize_data` |
119+
| `actions_executor` | `ask` | `action_*` (the business actions an object exposes) |
120+
| `metadata_authoring` + `solution_design` | `build` | metadata draft / verify / publish + blueprint propose / apply |
121121

122122
To grant data exploration to your own (platform-internal) agent, add
123123
`data_explorer` / `schema_reader` to its `skills[]`; deactivating a skill
124124
(`active: false`) revokes that capability for every agent that references it.
125125

126-
> **`surface:'build'` skills are inert on OSS — by design, not a bug.** The open
127-
> single-env framework ships only the `ask` agent; `metadata_authoring` /
128-
> `solution_design` (and any third-party `surface:'build'` skill) are supplied by
129-
> the cloud AI Studio plugin and simply do not resolve in OSS. A `build`-intent
130-
> turn on OSS degrades gracefully ("authoring lives in the cloud Build assistant")
131-
> instead of dead-ending — this is intentional tiering. Do not assume authoring
132-
> tools resolve in the open framework.
126+
> **All four are cloud-runtime skills — none resolves on OSS.** The open
127+
> framework ships *no* in-product agent (there is no `service-ai` package), so
128+
> `surface` / `tools` / `triggerConditions` are authored for cloud and inert.
129+
> What a skill DOES serve on the open edition is its `instructions`, projected
130+
> onto the MCP `prompts` primitive by `@objectstack/mcp`: a skill with blank
131+
> `instructions`, or `active: false`, is not listed as a prompt at all.
133132
134133
> **`visualize_data`:** the only built-in tool that draws a chart —
135134
> it aggregates an object and emits an inline `data-chart` part. Auto-registered
@@ -165,15 +164,15 @@ To grant data exploration to your own (platform-internal) agent, add
165164

166165
| Property | Purpose |
167166
|:---------|:--------|
168-
| `skills` | Array of skill names — **primary capability model** |
169-
| `tools` | Direct tool references — legacy fallback |
167+
| `skills` | Array of skill names — **the** capability model (ADR-0064) |
168+
| ~~`tools`~~ | REMOVED in protocol 17 — the inline slot resolved names against the *full* registry with no surface check. A parse error now; move each reference into a skill |
170169
| `surface` | `'ask' \| 'build'` — the product surface this agent is (default `'ask'`) |
171170
| `model` | LLM model configuration — `provider`, `model`, `temperature`, `maxTokens`, `topP` |
172171
| ~~`knowledge`~~ | REMOVED in protocol 17 — declaring sources/indexes on an agent never scoped retrieval (`search_knowledge` takes `sourceIds` from the LLM's tool-call arguments). Restrict at the knowledge-service/source level; describe intended grounding in `instructions` |
173-
| `guardrails` | `maxTokensPerInvocation`, `maxExecutionTimeSec`, `blockedTopics` |
174-
| `structuredOutput` | Output format (JSON schema, regex, etc.) |
172+
| `guardrails` | `maxTokensPerInvocation`, `maxExecutionTimeSec`, `blockedTopics` — declared only |
173+
| `structuredOutput` | Output format (JSON schema, regex, etc.) — declared only |
175174
| `planning` | Autonomous reasoning — `maxIterations` (default 10) |
176-
| `memory` | `longTerm` persistence + `reflectionInterval` |
175+
| `memory` | `longTerm` persistence + `reflectionInterval` — declared only |
177176
| `permissions` | Permission-set capabilities required to use the agent |
178177
| `active` | Enable/disable the agent |
179178

@@ -206,10 +205,10 @@ export default defineAgent({
206205
model: 'gpt-4o',
207206
temperature: 0.3,
208207
},
209-
guardrails: {
210-
blockedTopics: ['internal_pricing', 'employee_data'], // forbidden topics / action names
211-
maxTokensPerInvocation: 8000, // token budget per invocation
212-
maxExecutionTimeSec: 60, // wall-clock cap per invocation
208+
guardrails: { // declared only — nothing reads these yet
209+
blockedTopics: ['internal_pricing', 'employee_data'],
210+
maxTokensPerInvocation: 8000,
211+
maxExecutionTimeSec: 60,
213212
},
214213
});
215214
```
@@ -236,11 +235,10 @@ trigger conditions.
236235
| `surface` | `'ask' \| 'build' \| 'both'` — agent surface affinity (default `'ask'`; see above) |
237236
| `description` | What the skill does — helps the agent decide when to use it |
238237
| `instructions` | LLM prompt guidance specific to this skill's context |
239-
| `triggerPhrases` | Natural language phrases that activate the skill |
240238
| `triggerConditions` | Programmatic activation rules |
241239
| `active` | Is the skill enabled (default: `true`) |
242240

243-
> A skill has **no `permissions` key**it was removed in 16.x. Skill invocation
241+
> A skill has **no `permissions` key** — removed in 17.0.0. Skill invocation
244242
> was never gated by it (the registry reads only `active` / `triggerConditions` /
245243
> `tools`), and a security-shaped field that enforces nothing is worse than no
246244
> field at all. Gate access at the **agent** instead — `access` / `permissions` on
@@ -326,8 +324,8 @@ export default defineTool({
326324
To gate what a tool can do, gate the underlying action
327325
(`action.requiredPermissions`, ADR-0066) or the objects it touches; to withdraw
328326
a tool, remove it from the skills/agents that reference it. Categorization, if
329-
you need it, belongs on the action side (`action.ai.category`a live,
330-
enforced surface).
327+
you need it, belongs on the action side (`action.ai.category` — live, but
328+
listing-only: carried onto the tool, never sent to the model).
331329

332330
> **Tool metadata is a read-only projection — not an execution entry point.**
333331
> `ToolSchema` has no `handler` / `implementation` field, and no framework
@@ -336,21 +334,14 @@ enforced surface).
336334
> tool metadata is a one-way projection for Studio / discovery. Do not expect a
337335
> hand-authored tool to run in the open edition.
338336
339-
### Inline Agent `tools[]` (legacy)
340-
341-
Entries in an agent's inline `tools[]` array are a **different, legacy shape**
342-
(`AIToolSchema`): `{ type: 'action' | 'flow' | 'query' | 'vector_search',
343-
name, description? }` — references to existing actions / flows / queries, not
344-
tool definitions. Prefer skills + first-class tool names.
345-
346-
### Auto-Exposed Actions
337+
### Actions as AI Tools — opt-in
347338

348339
> **Cloud / EE runtime.** `registerActionsAsTools()`, `AIServicePlugin`, and
349340
> the HITL approval queue below ship in `@objectstack/service-ai` — the closed
350341
> cloud / Enterprise runtime, not an open package. On the open edition, expose
351342
> actions to your own AI via `@objectstack/mcp` instead.
352343
353-
You usually **don't author tool definitions by hand** for action invocation. Every `Action` you attach to an object via `defineObject({ actions: [...] })` is auto-exposed as a tool named `action_<actionName>` by `registerActionsAsTools()` (invoked from `AIServicePlugin`).
344+
You usually **don't author tool definitions by hand** for action invocation. An `Action` you attach to an object via `defineObject({ actions: [...] })` becomes a tool named `action_<actionName>` **only when it opts in**`ai.exposed: true` (default `false`) plus an `ai.description` of ≥ 40 chars, refused by the parse without it (ADR-0011). `registerActionsAsTools()` (from `AIServicePlugin`) walks the opted-in ones; exposure is never automatic, and there is no opt-out key.
354345

355346
Three action types dispatch headlessly:
356347

@@ -360,10 +351,9 @@ Three action types dispatch headlessly:
360351
| `api` | HTTP call to `action.target` (`fetch`-based by default) | `AIServicePlugin({ apiActionBaseUrl, apiActionHeaders })` or custom `apiClient` |
361352
| `flow` | `IAutomationService.execute(target, { triggerData })` | `automation` service registered with the kernel |
362353

363-
**Skipped automatically:**
354+
**Skipped even when opted in:**
364355
- UI-only types (`url`, `modal`, `form`).
365-
- Dangerous variants (`confirmText` set, `mode: 'delete'`, `variant: 'danger'`) — **unless** the plugin is started with `enableActionApproval: true`, in which case they route through the HITL approval queue (see below).
366-
- Owner opt-outs (`aiExposed: false`).
356+
- Dangerous variants — the declared `mode: 'delete'` / `variant: 'danger'` only (`confirmText` is dialog copy, *not* a destructive signal; `ai.requiresConfirmation` overrides either way) — **unless** the plugin is started with `enableActionApproval: true`, in which case they route through the HITL approval queue (see below).
367357

368358
**`type:'api'` body assembly** (last wins): user params → `recordIdParam` (using `recordIdField`, default `'id'`) → `bodyExtra`. `bodyShape: { wrap: 'data' }` nests user params under `data` while keeping `recordIdParam` flat.
369359

@@ -537,10 +527,10 @@ On validation failure the runtime retries by default
537527
more. Be specific about what the agent should and should not do.
538528
2. **Too many tools per skill.** Keep skills focused (3–8 tools). If a skill
539529
has 15+ tools, split it.
540-
3. **Missing guardrails and approval gates.** Define `blockedTopics` (plus the
541-
token / time budgets) in agent `guardrails`; for destructive operations put
542-
a human in the loop with a gate that is **actually enforced**
543-
`enableActionApproval: true` (HITL queue, cloud) for auto-exposed actions,
530+
3. **Mistaking `guardrails` for a gate.** `guardrails` / `memory` /
531+
`structuredOutput` are declared only — no runtime reads them, and real
532+
limits come from the quota service. For a gate that is **enforced**, use
533+
`enableActionApproval: true` (HITL queue, cloud) for AI-exposed actions,
544534
`ai.requiresConfirmation` on the **action**, or `approval: 'always'` on an
545535
MCP tool binding. AI metadata edits are already gated: they land as drafts a
546536
human must publish (ADR-0033).
@@ -550,8 +540,8 @@ On validation failure the runtime retries by default
550540
There is no `requireApprovalFor` field.
551541
4. **Ignoring tool descriptions.** The LLM uses tool `description` to decide
552542
when to call it. Poor descriptions = wrong tool selection.
553-
5. **Not testing trigger phrases.** Ambiguous trigger phrases cause skill
554-
conflicts. Test with edge-case inputs.
543+
5. **Expecting a phrase to activate a skill.** Nothing matches phrases; write
544+
routing as `triggerConditions` and intent in `description`/`instructions`.
555545
6. **Indexing everything.** A knowledge source without a `where` filter and
556546
curated `contentFields` fills the index with drafts and boilerplate that
557547
pollute retrieval. Source hygiene is the metadata's job; relevance tuning
@@ -565,7 +555,7 @@ Reference layout for a scaffolded app:
565555

566556
| Layer | File | Pattern |
567557
|:--|:--|:--|
568-
| Reusable skill | `src/skills/lead-qualification.skill.ts` | `defineSkill`trigger phrases + trigger conditions + bounded toolset; pick a `surface` |
558+
| Reusable skill | `src/skills/lead-qualification.skill.ts` | `defineSkill``instructions` + trigger conditions + bounded toolset; pick a `surface` |
569559
| Tool metadata | `src/tools/query-leads.tool.ts` | `defineTool` — JSON-Schema `parameters`; a discovery projection, not an executor (see caveat above) |
570560
| Knowledge source | `src/knowledge/sales-kb.ts` | `KnowledgeSourceSchema` metadata, registered at runtime via `IKnowledgeService.registerSource()` |
571561
| Central registration | `defineStack({ skills: [...], tools: [...] })` | `agents` / `tools` / `skills` are the only AI stack collections — knowledge sources have none; agents are platform-supplied |

skills/objectstack-ai/evals/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ When implemented, evals will follow this structure:
1313
```
1414
evals/
1515
├── skills/
16-
│ ├── test-trigger-phrases.md
16+
│ ├── test-trigger-conditions.md
1717
│ └── test-surface-affinity.md
1818
├── tools/
1919
│ ├── test-json-schema-parameters.md
20-
│ └── test-requires-confirmation.md
20+
│ └── test-strict-unknown-keys.md
2121
├── knowledge/
2222
│ └── test-knowledge-source-filters.md
2323
└── ...

0 commit comments

Comments
 (0)