Schema tools defined at runtime: persist the declaration, build the class on demand

[TonsOfFun]

Follow-up to #440 (the config array restates what the framework already knows) and #438 (the registration seam). Question raised by the repo owner: can schema tools be created dynamically at runtime, with their state persisted in the DB — or read from the app's app/agent_tools directory?

Short answer: yes, and it works today with no gem changes. Verified against a real host app. Writing it up with the evidence, including the one thing that breaks.

It already works

A SchemaTools subclass built at runtime from a plain hash behaves identically to a file-defined one:

spec = { "model" => "Milestone",
         "filterable" => %w[project_id completed out_of_scope],
         "returns"    => %w[id name phase completed out_of_scope] }

klass = Class.new(ActiveAgent::SchemaTools) do
  model spec["model"].constantize
  filterable(*spec["filterable"].map(&:to_sym))
  returns(*spec["returns"].map(&:to_sym))
end

klass.tool_names
# => ["find_milestones", "count_milestones", "get_milestone"]
klass.call("find_milestones", actor: owner, out_of_scope: true)[:count]   # => 2
klass.call("find_milestones", actor: owner, description: "x")[:error]
# => "`description` is not a filterable attribute. Allowed filters: ..."

Two properties make this work, both already true:

• Tool names derive from the model, not the class name (find_milestones, not find_milestone_tools), so an anonymous class is fine.
• define_method over the declared allowlist means the roster is built at declaration time, not tied to a constant.

The scope block can be derived too

This was the part that looked like it needed hand-written Ruby. It does not — the policy is also a naming convention:

policy = "#{model_name}Policy".safe_constantize
scope { |actor| policy::Scope.new(actor, model).resolve } if policy&.const_defined?(:Scope)

Verified the boundary still holds with a fully derived scope:

actor	result
owner (team member)	6 tickets
partner	only their own client — client_ids == [13]
nil	0

So Reservation → ReservationPolicy::Scope → bounded reads, with no Ruby written by the host at all. That is the shape #440 was asking for.

The thing that breaks: anonymous subclasses leak

SchemaTools.inherited tracks every subclass, and anonymous classes are never collected:

descendants before=0  after=2  (leaked 2)
named subclasses: {"(anonymous)" => 2}

Two classes built inside two reload cycles, both retained. A DB-driven registry that rebuilds on every change would grow descendants without bound, and any discovery that reads descendants would see stale duplicates for the same model.

This is the actual design constraint, and it has to be solved before a Class.new approach ships. Options:

1. Do not track anonymous subclasses. inherited skips a class with no name, and the registry holds its own list. Simplest; makes descendants mean "file-defined" only.
2. A registry keyed by model, replacing rather than appending, so rebuilding Reservation's tools evicts the previous class.
3. Do not build classes at all — an instance-based SchemaTools::Definition holding model/filterable/returns/scope, with tool_definitions and call as instance methods. Class-level DSL stays for file-defined tools and becomes a thin wrapper. Most invasive, but no leak and no metaprogramming.

(3) is the clean one if this becomes a real feature rather than a convenience.

Where the declaration would live

Two sources, and they are not exclusive:

• app/agent_tools/*.rb — file-defined, reviewed in a PR, versioned in git. Right for a boundary someone thought about.
• A DB table (active_agent_schema_tools: model, filterable, returns, enabled) — editable in the dashboard, no deploy. Right for iterating during evaluation work.

agents.tools is already jsonb, so per-agent selection needs no migration; this would only add the definitions themselves.

The security argument does not change

Whatever the source, the allowlist is still a judgement, not a schema fact. Measured on a 27-model host app, auto-deriving from every model would expose User, ApiToken and CliDeviceGrant. A DB-editable definition makes that boundary easier to change — including by someone who has not thought about it — so it wants the same guardrails: a refusal list for auth/token/session-shaped tables, and the definition treated as a security-relevant edit rather than a preference.

Reference: #440 has the fuller argument and four options for deriving the roster.

[TonsOfFun]

The "build the class on demand" half landed in #447: ActiveAgent::SchemaTools.define(model, filterable:, returns:, scope: | policy:) builds the same bounded roster a file would, named <Model>Tools, and registers it under its model — a redefinition replaces the previous class, so a registry rebuilt on every change holds one class per model. The dashboard discovers registry entries beside the files and a runtime definition supersedes a file for the same model.

The leak this issue measured is closed at the point it did harm: discovery no longer reads runtime classes out of descendants (option 1 is not possible — Class#subclasses cannot be opted out of), so a superseded class is never offered beside its replacement. Option 3 (an instance-based definition with no classes) remains the cleaner long-term shape.

Still open here: where a declaration is persisted (a table, a dashboard form) and the guardrails a DB-editable boundary needs. The registry is the seam for that.