Skip to content

feat(rules): detect missing EBTE tool-call claims and mediation (schema 27) - #276

Open
ericksondelacruz wants to merge 2 commits into
mainfrom
feat/analyzer-ebte-claims
Open

ericksondelacruz wants to merge 2 commits into
mainfrom
feat/analyzer-ebte-claims

Conversation

@ericksondelacruz

Copy link
Copy Markdown
Collaborator

Summary

Detects EBTE (Explanation-Bound Tool Execution) tool-call claims on Python tools and whether anything mediates them.

  • New claim_use discovery fact on Python decorated tools that declare a claim parameter. Its value is unused, passthrough, checked or wrapped. An OpenAI tool input guardrail, or an extra decorator, counts as a possible mediator (wrapped).
  • New tool_claim_use predicate (adds absent for tools with no claim parameter). The loader validates its values.
  • SupportedSchemaVersion goes from 26 to 27.
  • Fixture mirror of nine new rules, three per SDK (OpenAI Agents, MCP, LangChain; Python):
    • OAI-034 / MCP-033 / LC-028: side-effecting tool with no claim (info)
    • OAI-035 / MCP-034 / LC-029: claim declared but never read (medium)
    • OAI-036 / MCP-035 / LC-030: claim only printed, logged, serialized or returned (medium)

Notes for review

  • Info severity on the no-claim rules is deliberate. Almost no code uses a claim argument yet, so at low these rules would fail every --strict run on a repo with side-effecting tools. At info the finding and its fix show in every report, never fail --strict, and weigh 0.05 in the score. The unused and passthrough rules stay medium, since they only fire on code that has adopted claims.
  • The rules land in agent-reliability-rules and trustabl-rulebook on branches with the same name (feat/analyzer-ebte-claims). Their CI pairs with this branch, so merge this PR first.
  • A follow-up branch (feat/ebte-claims-coverage, schemas 28–31) extends claims to the other SDKs and stacks on this one.

Test plan

  • go build ./..., go vet ./...
  • go test ./...: unit tests for claim_use classification, predicate tests, policy cases, and scanner tests over the fixture rules
  • Rules repo CI passes against this branch

…ma 27)

New claim_use discovery fact on Python decorated tools that declare a
claim parameter (unused / passthrough / checked / wrapped; an OpenAI tool
input guardrail counts as a possible mediator) and a tool_claim_use
predicate, with a loader check on its values. SupportedSchemaVersion
26 -> 27.

Fixture mirror of nine new rules, a ladder per SDK (OpenAI Agents, MCP,
LangChain; Python): OAI-034/MCP-033/LC-028 side-effecting tool with no
claim, OAI-035/MCP-034/LC-029 claim never read, OAI-036/MCP-035/LC-030
claim only printed, logged, serialized or returned.
…ions

OAI-034 / MCP-033 / LC-028 move from low to info. Nobody uses a claim
argument yet, so at low they would fail every --strict run on a repo
with side-effecting tools. Info shows the finding and its fix in every
report, never fails --strict, and weighs 0.05 in the score instead of
0.15. The unused and passthrough rungs stay medium: they fire only on
code that adopted claims.

The other open questions are settled without code changes. The
passthrough reading and the exact `claim` name stay. The topic file
stays. Claude Agent SDK is the next SDK. Signing and the badge are
left to the rules repo's workflows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants