From e7b5e0695f954690087bd4e5188559c641804258 Mon Sep 17 00:00:00 2001 From: Simon Couch Date: Thu, 27 Aug 2026 15:03:02 -0700 Subject: [PATCH 1/5] document private agent tools --- pkg-r/R/commons.R | 31 ++++++++++++++--- pkg-r/R/data-source.R | 34 +++++++++---------- pkg-r/R/measures.R | 11 +++--- .../commons/references/data-dictionaries.md | 8 ++--- pkg-r/man/commons.Rd | 34 ++++++++++++++++--- pkg-r/man/data_source.Rd | 34 +++++++++---------- pkg-r/man/measure.Rd | 5 +-- pkg-r/man/semantic_layer.Rd | 6 ++-- pkg-r/vignettes/commons.Rmd | 2 +- pkg-r/vignettes/governance.Rmd | 18 +++++----- 10 files changed, 116 insertions(+), 67 deletions(-) diff --git a/pkg-r/R/commons.R b/pkg-r/R/commons.R index 06bc415e..432a5fba 100644 --- a/pkg-r/R/commons.R +++ b/pkg-r/R/commons.R @@ -1,7 +1,7 @@ #' Create a commons agent #' -#' `commons()` creates an [ellmer::Chat] subclass with tools for a semantic -#' layer, context search, table inspection, and SQL queries. +#' `commons()` creates an [ellmer::Chat] subclass with tools for trusted +#' calculations, context search, table inspection, and code execution. #' #' The provider and model come from `client`; commons sets its own system prompt #' and tools. Use `agent$chat()` to ask questions, [commons_theme()] and @@ -13,8 +13,8 @@ #' ignored, with a warning; use `instructions` to add to commons' prompt. #' @param data_sources A [data_source()], or a named list of them. Measures #' can take a source's connection as an argument named after the source; see -#' [semantic_layer()]. When there are several sources, the `run_sql` and -#' `describe_table` tools take a source's name as a `source` argument. +#' [semantic_layer()]. When there are several sources, the SQL query and +#' table inspection tools take a source's name as a `source` argument. #' @param semantic_layer An optional [semantic_layer()]. #' @param context_layer An optional [context_layer()]. #' @param ... These dots are for future extensions and must be empty. @@ -28,7 +28,8 @@ #' instructions = "Use the organization's fiscal-year conventions." #' ) #' ``` -#' @param network Whether the `run_r` session has network access. One of +#' @param network Whether the model-authored R session has network access. One +#' of #' `"none"` (the default) or `"full"`. The session uses OS sandboxing on #' Linux and macOS. On unsupported hosts, local development can opt in to #' best-effort R guardrails with @@ -51,6 +52,26 @@ #' role is viewer cannot read traces even when named here; trace readers #' need at least a publisher account. #' +#' @section Agent tools: +#' Depending on its semantic layer, context layer, and data sources, a commons +#' agent receives some combination of these tools: +#' +#' * `search_pool` searches trusted calculations and semantic models. +#' * `search_catalog` searches a warehouse catalog. +#' * `call_measure` invokes an R measure. +#' * `call_metrics` invokes governed or warehouse-native metrics. +#' * `call_calculation` invokes an exact trusted query. +#' * `search_context` retrieves relevant business context. +#' * `describe_table` inspects a table or semantic model. +#' * `run_sql` executes a read-only SQL query. +#' * `run_r` analyzes results and renders plots with model-authored R code. +#' +#' These model-facing tools are implementation details. Their constructors are +#' intentionally not exported, and their names, arguments, availability, and +#' behavior may change without notice. Application code should configure an +#' agent through `commons()` and its layer constructors rather than depend on +#' individual tools. +#' #' @return An [ellmer::Chat] subclass. #' #' @examples diff --git a/pkg-r/R/data-source.R b/pkg-r/R/data-source.R index 96e841fc..8493d2d8 100644 --- a/pkg-r/R/data-source.R +++ b/pkg-r/R/data-source.R @@ -12,8 +12,8 @@ #' * A `pins` board, e.g. [pins::board_connect()], is read into the same #' in-process database: each pin in `tables` becomes a table. Pin names are #' validated against the board at construction (a single listing call), but -#' each pin is downloaded only when its table is first used---by the -#' `describe_table` tool, a SQL query that references it, or a measure that +#' each pin is downloaded only when its table is first used---by table +#' inspection, a SQL query that references it, or a measure that #' takes the source's connection. [commons_server()] starts a background #' process right after startup that downloads the remaining pins into the #' local pins cache, so a first use typically only reads an @@ -44,14 +44,14 @@ #' model's definition when the agent describes or uses it. Explicitly #' selected models are read and validated when the data source is created. #' Snowflake semantic variables and Databricks metric-view parameters are -#' passed as typed JSON arguments to `call_metrics`. Databricks wildcard -#' members require concrete column metadata from the warehouse. -#' Native semantic models are available through `search_pool`, -#' `describe_table`, and `call_metrics`, but are not registered as physical -#' tables. +#' passed as typed JSON arguments when the agent invokes them. Databricks +#' wildcard members require concrete column metadata from the warehouse. +#' Native semantic models are available through trusted-calculation search, +#' table inspection, and metric invocation, but are not registered as +#' physical tables. #' Snowflake verified queries are exposed separately as exact trusted -#' calculations through `search_pool` and `call_calculation`; their SQL is -#' executed as stored rather than parsed to infer dependencies. +#' calculations; their SQL is executed as stored rather than parsed to infer +#' dependencies. #' An exact physical-table selection also imports associated models when #' every physical dependency is selected. Public relationships, facts, #' filters, and instructions become table-scoped first-touch and retrieval @@ -76,27 +76,27 @@ #' are included in the system prompt. These fields are the place for rules #' that span tables and for guidance on which tables answer which kinds of #' questions. -#' * The first time a conversation touches a table---via the `describe_table` -#' tool or a SQL query---the table's full dictionary entry rides along with +#' * The first time a conversation touches a table---via table inspection or a +#' SQL query---the table's full dictionary entry rides along with #' the tool result: its prose, documented columns, relationships, and -#' definitions of glossary terms it references. `describe_table` merges -#' documented columns with the table's live schema. +#' definitions of glossary terms it references. Table inspection merges +#' documented columns with the live schema. #' * For Snowflake and Databricks sources, a fully qualified dictionary table #' name matches the same selected relation. A relative name is accepted when #' it matches only one selected relation. Authored prose takes precedence, #' while warehouse column types remain authoritative. #' * When the agent also has a [context_layer()], the dictionary's prose is -#' indexed for the `search_context` tool. +#' indexed for context retrieval. #' #' A table's entry can also declare `definitions`: named expressions in the #' [data-dict expression language](https://data-dict.tidyverse.org/expressions.html). #' commons validates their inferred types and references, compiles them for -#' the source's SQL backend, and lets the model apply them as `{{name}}` -#' tokens in `run_sql` or through `call_metrics`. Definitions are delivered +#' the source's SQL backend, and lets the model use them as `{{name}}` tokens in +#' custom SQL or, for metrics, invoke them directly. Definitions are delivered #' through all three channels above. #' #' @section Trust: -#' The `run_sql` tool runs only read-only `SELECT` queries; statements that +#' The SQL query tool runs only read-only `SELECT` queries; statements that #' would modify data or schema (`INSERT`, `UPDATE`, `DROP`, and similar) are #' rejected before reaching the database. For the in-process DuckDB built from #' data frames, commons additionally disables extension loading and filesystem diff --git a/pkg-r/R/measures.R b/pkg-r/R/measures.R index dcb5f1bf..7bebc83b 100644 --- a/pkg-r/R/measures.R +++ b/pkg-r/R/measures.R @@ -19,9 +19,9 @@ #' such as `string[]`. Without a declaration, commons infers the type from the #' default, falling back to `string`. #' -#' Measure and helper source is visible in `run_r`; evaluating a measure's name -#' there prints its definition. Function environments, connections, and -#' credentials are not shared with that session. +#' Measure and helper source is visible to model-authored R code; evaluating a +#' measure's name there prints its definition. Function environments, +#' connections, and credentials are not shared with that session. #' #' @section Measure arguments: #' A measure function can take two kinds of arguments: @@ -138,8 +138,9 @@ expand_measures <- function(args, env = rlang::caller_env()) { #' [ellmer::ContentToolResult]. Its `value` is sent to the model and its #' `extra$display` controls the shinychat display. When the display includes #' HTML, Markdown, or text, the model is told that the result is already visible -#' to the user. An optional `extra$data` value is made available to `run_r` and -#' removed from the result before it is returned to ellmer. +#' to the user. An optional `extra$data` value is made available for further +#' model-authored R analysis and removed from the result before it is returned +#' to ellmer. #' #' @param name Measure name. #' @param description What the measure computes. diff --git a/pkg-r/inst/skills/commons/references/data-dictionaries.md b/pkg-r/inst/skills/commons/references/data-dictionaries.md index f76ee408..2440ee7c 100644 --- a/pkg-r/inst/skills/commons/references/data-dictionaries.md +++ b/pkg-r/inst/skills/commons/references/data-dictionaries.md @@ -35,7 +35,7 @@ Prioritize information the agent needs to interpret and query the source: - Column meanings, types, units, values, ranges, examples, and constraints. - Relationships demonstrated by schemas, trusted code, or verified exploration. -Treat the dictionary as curated context, not a copy of the source schema. Be mindful of its total size and do not document every column by default. If a column entry would only repeat a name or type available from the live schema, omit it unless the agent needs that column as a dimension or `where` operand in `call_metrics`. Prioritize columns whose meaning, units, valid values, caveats, relationships, or analytical role are not evident from the schema. +Treat the dictionary as curated context, not a copy of the source schema. Be mindful of its total size and do not document every column by default. If a column entry would only repeat a name or type available from the live schema, omit it unless the agent needs that column for grouping or filtering a trusted metric calculation. Prioritize columns whose meaning, units, valid values, caveats, relationships, or analytical role are not evident from the schema. Prefer the dictionary over `context_layer()` for information that belongs to a dataset, table, column, or relationship. Keep one authoritative copy of each fact and do not add claims that the available evidence does not support. @@ -44,7 +44,7 @@ Prefer the dictionary over `context_layer()` for information that belongs to a d commons delivers dictionary content progressively: - Dataset-level descriptions and details are available in the system prompt, along with as many glossary terms as fit within its size cap. Remaining glossary terms are searchable and arrive on first touch when a table entry references them. -- The first time a conversation uses a table, its description, details, documented columns, relationships, and governed definitions are delivered with the tool result. `describe_table` combines the dictionary entry with the live schema. +- The first time a conversation uses a table, its description, details, documented columns, relationships, and governed definitions are delivered with the tool result. Table inspection combines the dictionary entry with the live schema. - Dictionary prose and governed definitions are indexed for context search. Column details remain first-touch content rather than a second searchable copy. Put broadly applicable source guidance at the dataset level and table-specific guidance on the table. This keeps ambient context small while making detailed information available when it becomes relevant. @@ -73,7 +73,7 @@ tables: Each definition requires `name` and `expr`. `label`, `description`, and `details` are optional. commons validates the expression's inferred type and references, determines whether it is a filter, metric, or derived row value, and compiles it for the source's SQL backend. Document every referenced column and its type so the expression can be checked. Definition names must not shadow columns. Definitions on the same table may refer to one another by name, but references must not form a cycle. -The agent can apply definitions as `{{name}}` tokens in SQL. When the same name exists on multiple tables in a query, qualify it as `{{table::name}}`. Metrics can also be called through `call_metrics`. Add a definition only when its business meaning and computation come from trusted existing material and the user confirms any consequential choice. Do not invent a calculation merely to complete the dictionary. +The agent can apply definitions as `{{name}}` tokens in SQL. When the same name exists on multiple tables in a query, qualify it as `{{table::name}}`. Metrics can also be invoked as trusted calculations. Add a definition only when its business meaning and computation come from trusted existing material and the user confirms any consequential choice. Do not invent a calculation merely to complete the dictionary. ## Validate @@ -83,4 +83,4 @@ Use data-dict validation where useful, then verify the commons integration: - Compare documented columns and types with the live schema. - Confirm relationships and definition expressions use real tables and columns. - Construct the data sources and agent to catch expression parsing, type, reference, source-mapping, and backend-compilation errors. -- Exercise each definition against the live source with `run_sql` or `call_metrics`, as appropriate. Construction compiles but does not execute definitions, so representative queries are still needed to catch runtime and data-dependent errors. +- Exercise each definition against the live source with custom SQL or a trusted metric calculation, as appropriate. Construction compiles but does not execute definitions, so representative queries are still needed to catch runtime and data-dependent errors. diff --git a/pkg-r/man/commons.Rd b/pkg-r/man/commons.Rd index 83e52ba8..3e1c0442 100644 --- a/pkg-r/man/commons.Rd +++ b/pkg-r/man/commons.Rd @@ -23,8 +23,8 @@ ignored, with a warning; use \code{instructions} to add to commons' prompt.} \item{data_sources}{A \code{\link[=data_source]{data_source()}}, or a named list of them. Measures can take a source's connection as an argument named after the source; see -\code{\link[=semantic_layer]{semantic_layer()}}. When there are several sources, the \code{run_sql} and -\code{describe_table} tools take a source's name as a \code{source} argument.} +\code{\link[=semantic_layer]{semantic_layer()}}. When there are several sources, the SQL query and +table inspection tools take a source's name as a \code{source} argument.} \item{semantic_layer}{An optional \code{\link[=semantic_layer]{semantic_layer()}}.} @@ -42,7 +42,8 @@ system prompt, as a single string or the path to a text or Markdown file. ) }\if{html}{\out{}}} -\item{network}{Whether the \code{run_r} session has network access. One of +\item{network}{Whether the model-authored R session has network access. One +of \code{"none"} (the default) or \code{"full"}. The session uses OS sandboxing on Linux and macOS. On unsupported hosts, local development can opt in to best-effort R guardrails with @@ -71,8 +72,8 @@ need at least a publisher account.} An \link[ellmer:Chat]{ellmer::Chat} subclass. } \description{ -\code{commons()} creates an \link[ellmer:Chat]{ellmer::Chat} subclass with tools for a semantic -layer, context search, table inspection, and SQL queries. +\code{commons()} creates an \link[ellmer:Chat]{ellmer::Chat} subclass with tools for trusted +calculations, context search, table inspection, and code execution. } \details{ The provider and model come from \code{client}; commons sets its own system prompt @@ -80,6 +81,29 @@ and tools. Use \code{agent$chat()} to ask questions, \code{\link[=commons_theme] \code{\link[=commons_server]{commons_server()}} to embed the agent in Shiny, and \code{\link[vitals:generate]{vitals::generate()}} to use the agent as a vitals solver. } +\section{Agent tools}{ + +Depending on its semantic layer, context layer, and data sources, a commons +agent receives some combination of these tools: +\itemize{ +\item \code{search_pool} searches trusted calculations and semantic models. +\item \code{search_catalog} searches a warehouse catalog. +\item \code{call_measure} invokes an R measure. +\item \code{call_metrics} invokes governed or warehouse-native metrics. +\item \code{call_calculation} invokes an exact trusted query. +\item \code{search_context} retrieves relevant business context. +\item \code{describe_table} inspects a table or semantic model. +\item \code{run_sql} executes a read-only SQL query. +\item \code{run_r} analyzes results and renders plots with model-authored R code. +} + +These model-facing tools are implementation details. Their constructors are +intentionally not exported, and their names, arguments, availability, and +behavior may change without notice. Application code should configure an +agent through \code{commons()} and its layer constructors rather than depend on +individual tools. +} + \examples{ \dontrun{ # A measure over local data computes directly in R. diff --git a/pkg-r/man/data_source.Rd b/pkg-r/man/data_source.Rd index 5024dc7c..9c72c582 100644 --- a/pkg-r/man/data_source.Rd +++ b/pkg-r/man/data_source.Rd @@ -28,14 +28,14 @@ Namespace selections retain lightweight model metadata, then read a model's definition when the agent describes or uses it. Explicitly selected models are read and validated when the data source is created. Snowflake semantic variables and Databricks metric-view parameters are -passed as typed JSON arguments to \code{call_metrics}. Databricks wildcard -members require concrete column metadata from the warehouse. -Native semantic models are available through \code{search_pool}, -\code{describe_table}, and \code{call_metrics}, but are not registered as physical -tables. +passed as typed JSON arguments when the agent invokes them. Databricks +wildcard members require concrete column metadata from the warehouse. +Native semantic models are available through trusted-calculation search, +table inspection, and metric invocation, but are not registered as +physical tables. Snowflake verified queries are exposed separately as exact trusted -calculations through \code{search_pool} and \code{call_calculation}; their SQL is -executed as stored rather than parsed to infer dependencies. +calculations; their SQL is executed as stored rather than parsed to infer +dependencies. An exact physical-table selection also imports associated models when every physical dependency is selected. Public relationships, facts, filters, and instructions become table-scoped first-touch and retrieval @@ -69,8 +69,8 @@ when the data isn't already in a database. \item A \code{pins} board, e.g. \code{\link[pins:board_connect]{pins::board_connect()}}, is read into the same in-process database: each pin in \code{tables} becomes a table. Pin names are validated against the board at construction (a single listing call), but -each pin is downloaded only when its table is first used---by the -\code{describe_table} tool, a SQL query that references it, or a measure that +each pin is downloaded only when its table is first used---by table +inspection, a SQL query that references it, or a measure that takes the source's connection. \code{\link[=commons_server]{commons_server()}} starts a background process right after startup that downloads the remaining pins into the local pins cache, so a first use typically only reads an @@ -93,30 +93,30 @@ agent three ways: are included in the system prompt. These fields are the place for rules that span tables and for guidance on which tables answer which kinds of questions. -\item The first time a conversation touches a table---via the \code{describe_table} -tool or a SQL query---the table's full dictionary entry rides along with +\item The first time a conversation touches a table---via table inspection or a +SQL query---the table's full dictionary entry rides along with the tool result: its prose, documented columns, relationships, and -definitions of glossary terms it references. \code{describe_table} merges -documented columns with the table's live schema. +definitions of glossary terms it references. Table inspection merges +documented columns with the live schema. \item For Snowflake and Databricks sources, a fully qualified dictionary table name matches the same selected relation. A relative name is accepted when it matches only one selected relation. Authored prose takes precedence, while warehouse column types remain authoritative. \item When the agent also has a \code{\link[=context_layer]{context_layer()}}, the dictionary's prose is -indexed for the \code{search_context} tool. +indexed for context retrieval. } A table's entry can also declare \code{definitions}: named expressions in the \href{https://data-dict.tidyverse.org/expressions.html}{data-dict expression language}. commons validates their inferred types and references, compiles them for -the source's SQL backend, and lets the model apply them as \code{{{name}}} -tokens in \code{run_sql} or through \code{call_metrics}. Definitions are delivered +the source's SQL backend, and lets the model use them as \code{{{name}}} tokens in +custom SQL or, for metrics, invoke them directly. Definitions are delivered through all three channels above. } \section{Trust}{ -The \code{run_sql} tool runs only read-only \code{SELECT} queries; statements that +The SQL query tool runs only read-only \code{SELECT} queries; statements that would modify data or schema (\code{INSERT}, \code{UPDATE}, \code{DROP}, and similar) are rejected before reaching the database. For the in-process DuckDB built from data frames, commons additionally disables extension loading and filesystem diff --git a/pkg-r/man/measure.Rd b/pkg-r/man/measure.Rd index b40f3ce1..ed932e78 100644 --- a/pkg-r/man/measure.Rd +++ b/pkg-r/man/measure.Rd @@ -39,8 +39,9 @@ For full control over a result, \code{fn} can return an \link[ellmer:ContentToolResult]{ellmer::ContentToolResult}. Its \code{value} is sent to the model and its \code{extra$display} controls the shinychat display. When the display includes HTML, Markdown, or text, the model is told that the result is already visible -to the user. An optional \code{extra$data} value is made available to \code{run_r} and -removed from the result before it is returned to ellmer. +to the user. An optional \code{extra$data} value is made available for further +model-authored R analysis and removed from the result before it is returned +to ellmer. } \examples{ table <- data.frame(term = c("Headache", "Nausea"), count = c(7, 5)) diff --git a/pkg-r/man/semantic_layer.Rd b/pkg-r/man/semantic_layer.Rd index fb034324..b4a8c3d2 100644 --- a/pkg-r/man/semantic_layer.Rd +++ b/pkg-r/man/semantic_layer.Rd @@ -31,9 +31,9 @@ Each \verb{@param} marks a model-supplied argument and can declare its type: such as \code{string[]}. Without a declaration, commons infers the type from the default, falling back to \code{string}. -Measure and helper source is visible in \code{run_r}; evaluating a measure's name -there prints its definition. Function environments, connections, and -credentials are not shared with that session. +Measure and helper source is visible to model-authored R code; evaluating a +measure's name there prints its definition. Function environments, +connections, and credentials are not shared with that session. } \section{Measure arguments}{ diff --git a/pkg-r/vignettes/commons.Rmd b/pkg-r/vignettes/commons.Rmd index 3e7dac31..454e7cc2 100644 --- a/pkg-r/vignettes/commons.Rmd +++ b/pkg-r/vignettes/commons.Rmd @@ -247,7 +247,7 @@ tables: There are three kinds of definitions. Definitions can participate in trusted metric calculations or be used in custom SQL:[^definition-sql] -| Kind | Example | Use in `call_metrics` | +| Kind | Example | Use in a trusted metric calculation | |---|---|---| | Metric | `SUM(n)` | Computes the metric | | Filter | `status = 'active'` | Restricts rows or provides a grouping dimension | diff --git a/pkg-r/vignettes/governance.Rmd b/pkg-r/vignettes/governance.Rmd index f8055dec..229b912b 100644 --- a/pkg-r/vignettes/governance.Rmd +++ b/pkg-r/vignettes/governance.Rmd @@ -24,19 +24,21 @@ This vignette explains the boundaries that commons provides and the responsibili A commons agent is an ellmer chat with a system prompt and a set of tools. The system prompt excerpts available data sources and business context. Tools let the model retrieve more context, call a trusted calculation, run a SQL query, or execute R code. +The exact set depends on the agent's layers and data sources. The current tool names and their lifecycle status are documented in [commons()]. + We assume that a model might make any request allowed by its tools. Application security should therefore not depend on the model following an instruction like "never reveal sensitive data." Instead, only give an agent access to data that the current user of the application is allowed to see. The system prompt, tool arguments, and tool results are also sent to the model provider, so the agent should not have access to data that you [do not trust that provider to process](https://posit.co/blog/trust-llm-tools). -## SQL code execution with `run_sql` +## SQL code execution ### Destructive actions -`run_sql` accepts a single statement beginning with `SELECT` or `WITH`. commons rejects stacked statements and statements beginning with operations such as `INSERT`, `UPDATE`, `DELETE`, `DROP`, or `GRANT`. For data frames and pins, which commons loads into its own DuckDB database, it also disables extension loading and external filesystem access. +The agent's SQL query tool accepts a single statement beginning with `SELECT` or `WITH`. commons rejects stacked statements and statements beginning with operations such as `INSERT`, `UPDATE`, `DELETE`, `DROP`, or `GRANT`. For data frames and pins, which commons loads into its own DuckDB database, it also disables extension loading and external filesystem access. These checks provide defense in depth, but they are not a SQL parser or a database sandbox. When you supply a DBI connection, commons queries that connection as-is. The primary safeguard against destructive SQL is therefore database-enforced read-only access. ### Data access -The `tables` argument to `data_source()` controls which tables commons describes to the model. It is not an authorization boundary: `run_sql` can query any object available to the connection. +The `tables` argument to `data_source()` controls which tables commons describes to the model. It is not an authorization boundary: model-authored SQL can query any object available to the connection. On Posit Connect, [viewer OAuth integrations](https://docs.posit.co/connect/admin/access-controls/) can give an interactive application the current viewer's Snowflake or Databricks credentials. If the application creates its connection from those credentials, the warehouse continues to enforce that viewer's existing access policies, including row- and column-level security. commons snapshots the connection's principal, active role, and namespace when it creates a Snowflake or Databricks data source, and rejects subsequent operations if that identity changes. @@ -46,11 +48,11 @@ When viewer credentials are not available, use a service account with access to ### Transmitting results to the R process -The SQL query runs through the DBI connection in the main application process. Its result is both returned to the model and registered as a handle in the R process, allowing the model to analyze it with `run_r`. +The SQL query runs through the DBI connection in the main application process. Its result is both returned to the model and registered as a handle in the R process, allowing the model to analyze it further with R. -## `run_r` and its R process +## R code execution -The code requested through `run_r` does not run in the main application process. Each commons agent instead gets a persistent R subprocess, created on first use. Its state remains available to later `run_r` calls made by the same agent. +Model-authored R code does not run in the main application process. Each commons agent instead gets a persistent R subprocess, created on first use. Its state remains available to later R analysis by the same agent. ### OS-level sandboxing @@ -60,7 +62,7 @@ The filesystem sandbox gives the subprocess read access to R, installed packages By default, the subprocess cannot create network sockets. An application author can opt in to unrestricted network access with `commons(network = "full")`; the filesystem sandbox remains in place, but R code can then contact services reachable from the deployment and transmit data from result handles. Only enable network access when that egress is required and acceptable. -For local development on macOS, commons applies a similar filesystem and network policy using Seatbelt. Windows has no OS-level `run_r` sandbox. Deployed Connect applications run on Linux and use the Linux mechanisms described above. +For local development on macOS, commons applies a similar filesystem and network policy using Seatbelt. Windows has no OS-level sandbox for model-authored R code. Deployed Connect applications run on Linux and use the Linux mechanisms described above. By default, commons refuses to create an agent when an OS-level sandbox is not available. An application author can enable a local-development fallback explicitly: @@ -80,7 +82,7 @@ Communication happens through [callr](https://callr.r-lib.org/). commons seriali ## Permissioning facts -The rows returned by SQL are not the only potentially sensitive information available to an agent. Dataset descriptions and glossary entries from `data-dict.yaml` are included in the system prompt. Table documentation and sample values are supplied when a table is first used, context documents are available through search, and measure source can be read by the model in `run_r`. +The rows returned by SQL are not the only potentially sensitive information available to an agent. Dataset descriptions and glossary entries from `data-dict.yaml` are included in the system prompt. Table documentation and sample values are supplied when a table is first used, context documents are available through search, and measure source can be read by model-authored R code. Only include facts and source code that may be shared with both the application's viewers and its model provider. If one audience should not see a fact, use separate applications with separate context or place the facts behind viewer credentials rather than asking the model to hide it from that audience. From aa833b197524a0cee38d4078348fc02f939a97ab Mon Sep 17 00:00:00 2001 From: Simon Couch Date: Thu, 27 Aug 2026 15:16:11 -0700 Subject: [PATCH 2/5] refine private tool documentation --- pkg-r/R/commons.R | 17 ++++++++--------- pkg-r/R/data-source.R | 19 ++++++------------- pkg-r/R/measures.R | 7 +++---- pkg-r/man/commons.Rd | 17 ++++++++--------- pkg-r/man/data_source.Rd | 19 ++++++------------- pkg-r/man/measure.Rd | 5 ++--- pkg-r/man/semantic_layer.Rd | 2 +- pkg-r/vignettes/commons.Rmd | 2 +- pkg-r/vignettes/governance.Rmd | 12 ++++++------ 9 files changed, 41 insertions(+), 59 deletions(-) diff --git a/pkg-r/R/commons.R b/pkg-r/R/commons.R index 432a5fba..88f501c4 100644 --- a/pkg-r/R/commons.R +++ b/pkg-r/R/commons.R @@ -13,8 +13,7 @@ #' ignored, with a warning; use `instructions` to add to commons' prompt. #' @param data_sources A [data_source()], or a named list of them. Measures #' can take a source's connection as an argument named after the source; see -#' [semantic_layer()]. When there are several sources, the SQL query and -#' table inspection tools take a source's name as a `source` argument. +#' [semantic_layer()]. #' @param semantic_layer An optional [semantic_layer()]. #' @param context_layer An optional [context_layer()]. #' @param ... These dots are for future extensions and must be empty. @@ -28,7 +27,7 @@ #' instructions = "Use the organization's fiscal-year conventions." #' ) #' ``` -#' @param network Whether the model-authored R session has network access. One +#' @param network Whether the agent's R session has network access. One #' of #' `"none"` (the default) or `"full"`. The session uses OS sandboxing on #' Linux and macOS. On unsupported hosts, local development can opt in to @@ -64,13 +63,13 @@ #' * `search_context` retrieves relevant business context. #' * `describe_table` inspects a table or semantic model. #' * `run_sql` executes a read-only SQL query. -#' * `run_r` analyzes results and renders plots with model-authored R code. +#' * `run_r` analyzes results and renders plots in the agent's R session. #' -#' These model-facing tools are implementation details. Their constructors are -#' intentionally not exported, and their names, arguments, availability, and -#' behavior may change without notice. Application code should configure an -#' agent through `commons()` and its layer constructors rather than depend on -#' individual tools. +#' These model-facing tools should be considered private. Their constructors +#' are intentionally not exported, and their names, arguments, availability, +#' and behavior may change without notice. Application code should configure +#' an agent through `commons()` and its layer constructors rather than depend +#' on individual tools. #' #' @return An [ellmer::Chat] subclass. #' diff --git a/pkg-r/R/data-source.R b/pkg-r/R/data-source.R index 8493d2d8..6e33a1f6 100644 --- a/pkg-r/R/data-source.R +++ b/pkg-r/R/data-source.R @@ -43,15 +43,8 @@ #' Namespace selections retain lightweight model metadata, then read a #' model's definition when the agent describes or uses it. Explicitly #' selected models are read and validated when the data source is created. -#' Snowflake semantic variables and Databricks metric-view parameters are -#' passed as typed JSON arguments when the agent invokes them. Databricks -#' wildcard members require concrete column metadata from the warehouse. -#' Native semantic models are available through trusted-calculation search, -#' table inspection, and metric invocation, but are not registered as -#' physical tables. -#' Snowflake verified queries are exposed separately as exact trusted -#' calculations; their SQL is executed as stored rather than parsed to infer -#' dependencies. +#' Databricks wildcard members require concrete column metadata from the +#' warehouse. #' An exact physical-table selection also imports associated models when #' every physical dependency is selected. Public relationships, facts, #' filters, and instructions become table-scoped first-touch and retrieval @@ -91,12 +84,12 @@ #' A table's entry can also declare `definitions`: named expressions in the #' [data-dict expression language](https://data-dict.tidyverse.org/expressions.html). #' commons validates their inferred types and references, compiles them for -#' the source's SQL backend, and lets the model use them as `{{name}}` tokens in -#' custom SQL or, for metrics, invoke them directly. Definitions are delivered -#' through all three channels above. +#' the source's SQL backend, and makes them available to trusted metric +#' calculations and custom SQL. Definitions are delivered through all three +#' channels above. #' #' @section Trust: -#' The SQL query tool runs only read-only `SELECT` queries; statements that +#' The agent runs only read-only `SELECT` queries; statements that #' would modify data or schema (`INSERT`, `UPDATE`, `DROP`, and similar) are #' rejected before reaching the database. For the in-process DuckDB built from #' data frames, commons additionally disables extension loading and filesystem diff --git a/pkg-r/R/measures.R b/pkg-r/R/measures.R index 7bebc83b..3442fefd 100644 --- a/pkg-r/R/measures.R +++ b/pkg-r/R/measures.R @@ -19,7 +19,7 @@ #' such as `string[]`. Without a declaration, commons infers the type from the #' default, falling back to `string`. #' -#' Measure and helper source is visible to model-authored R code; evaluating a +#' Measure and helper source is visible in the agent's R session; evaluating a #' measure's name there prints its definition. Function environments, #' connections, and credentials are not shared with that session. #' @@ -138,9 +138,8 @@ expand_measures <- function(args, env = rlang::caller_env()) { #' [ellmer::ContentToolResult]. Its `value` is sent to the model and its #' `extra$display` controls the shinychat display. When the display includes #' HTML, Markdown, or text, the model is told that the result is already visible -#' to the user. An optional `extra$data` value is made available for further -#' model-authored R analysis and removed from the result before it is returned -#' to ellmer. +#' to the user. An optional `extra$data` value is made available in the agent's +#' R session and removed from the result before it is returned to ellmer. #' #' @param name Measure name. #' @param description What the measure computes. diff --git a/pkg-r/man/commons.Rd b/pkg-r/man/commons.Rd index 3e1c0442..7f9439dd 100644 --- a/pkg-r/man/commons.Rd +++ b/pkg-r/man/commons.Rd @@ -23,8 +23,7 @@ ignored, with a warning; use \code{instructions} to add to commons' prompt.} \item{data_sources}{A \code{\link[=data_source]{data_source()}}, or a named list of them. Measures can take a source's connection as an argument named after the source; see -\code{\link[=semantic_layer]{semantic_layer()}}. When there are several sources, the SQL query and -table inspection tools take a source's name as a \code{source} argument.} +\code{\link[=semantic_layer]{semantic_layer()}}.} \item{semantic_layer}{An optional \code{\link[=semantic_layer]{semantic_layer()}}.} @@ -42,7 +41,7 @@ system prompt, as a single string or the path to a text or Markdown file. ) }\if{html}{\out{}}} -\item{network}{Whether the model-authored R session has network access. One +\item{network}{Whether the agent's R session has network access. One of \code{"none"} (the default) or \code{"full"}. The session uses OS sandboxing on Linux and macOS. On unsupported hosts, local development can opt in to @@ -94,14 +93,14 @@ agent receives some combination of these tools: \item \code{search_context} retrieves relevant business context. \item \code{describe_table} inspects a table or semantic model. \item \code{run_sql} executes a read-only SQL query. -\item \code{run_r} analyzes results and renders plots with model-authored R code. +\item \code{run_r} analyzes results and renders plots in the agent's R session. } -These model-facing tools are implementation details. Their constructors are -intentionally not exported, and their names, arguments, availability, and -behavior may change without notice. Application code should configure an -agent through \code{commons()} and its layer constructors rather than depend on -individual tools. +These model-facing tools should be considered private. Their constructors +are intentionally not exported, and their names, arguments, availability, +and behavior may change without notice. Application code should configure +an agent through \code{commons()} and its layer constructors rather than depend +on individual tools. } \examples{ diff --git a/pkg-r/man/data_source.Rd b/pkg-r/man/data_source.Rd index 9c72c582..1451836f 100644 --- a/pkg-r/man/data_source.Rd +++ b/pkg-r/man/data_source.Rd @@ -27,15 +27,8 @@ selections import metric views, as native trusted metrics and dimensions. Namespace selections retain lightweight model metadata, then read a model's definition when the agent describes or uses it. Explicitly selected models are read and validated when the data source is created. -Snowflake semantic variables and Databricks metric-view parameters are -passed as typed JSON arguments when the agent invokes them. Databricks -wildcard members require concrete column metadata from the warehouse. -Native semantic models are available through trusted-calculation search, -table inspection, and metric invocation, but are not registered as -physical tables. -Snowflake verified queries are exposed separately as exact trusted -calculations; their SQL is executed as stored rather than parsed to infer -dependencies. +Databricks wildcard members require concrete column metadata from the +warehouse. An exact physical-table selection also imports associated models when every physical dependency is selected. Public relationships, facts, filters, and instructions become table-scoped first-touch and retrieval @@ -109,14 +102,14 @@ indexed for context retrieval. A table's entry can also declare \code{definitions}: named expressions in the \href{https://data-dict.tidyverse.org/expressions.html}{data-dict expression language}. commons validates their inferred types and references, compiles them for -the source's SQL backend, and lets the model use them as \code{{{name}}} tokens in -custom SQL or, for metrics, invoke them directly. Definitions are delivered -through all three channels above. +the source's SQL backend, and makes them available to trusted metric +calculations and custom SQL. Definitions are delivered through all three +channels above. } \section{Trust}{ -The SQL query tool runs only read-only \code{SELECT} queries; statements that +The agent runs only read-only \code{SELECT} queries; statements that would modify data or schema (\code{INSERT}, \code{UPDATE}, \code{DROP}, and similar) are rejected before reaching the database. For the in-process DuckDB built from data frames, commons additionally disables extension loading and filesystem diff --git a/pkg-r/man/measure.Rd b/pkg-r/man/measure.Rd index ed932e78..cfa84e56 100644 --- a/pkg-r/man/measure.Rd +++ b/pkg-r/man/measure.Rd @@ -39,9 +39,8 @@ For full control over a result, \code{fn} can return an \link[ellmer:ContentToolResult]{ellmer::ContentToolResult}. Its \code{value} is sent to the model and its \code{extra$display} controls the shinychat display. When the display includes HTML, Markdown, or text, the model is told that the result is already visible -to the user. An optional \code{extra$data} value is made available for further -model-authored R analysis and removed from the result before it is returned -to ellmer. +to the user. An optional \code{extra$data} value is made available in the agent's +R session and removed from the result before it is returned to ellmer. } \examples{ table <- data.frame(term = c("Headache", "Nausea"), count = c(7, 5)) diff --git a/pkg-r/man/semantic_layer.Rd b/pkg-r/man/semantic_layer.Rd index b4a8c3d2..bcd80663 100644 --- a/pkg-r/man/semantic_layer.Rd +++ b/pkg-r/man/semantic_layer.Rd @@ -31,7 +31,7 @@ Each \verb{@param} marks a model-supplied argument and can declare its type: such as \code{string[]}. Without a declaration, commons infers the type from the default, falling back to \code{string}. -Measure and helper source is visible to model-authored R code; evaluating a +Measure and helper source is visible in the agent's R session; evaluating a measure's name there prints its definition. Function environments, connections, and credentials are not shared with that session. } diff --git a/pkg-r/vignettes/commons.Rmd b/pkg-r/vignettes/commons.Rmd index 454e7cc2..676ffc74 100644 --- a/pkg-r/vignettes/commons.Rmd +++ b/pkg-r/vignettes/commons.Rmd @@ -255,7 +255,7 @@ There are three kinds of definitions. Definitions can participate in trusted met commons infers the definition kind from its expression. Aggregate and constant expressions are categorized as metrics, row-level Boolean expressions as filters, and other row-level expressions as derived values. -[^definition-sql]: In custom SQL, the agent refers to a definition using its `{{name}}` token. commons expands the token to SQL compiled for the data source. Because this is still custom SQL, the provenance outcome is `Cited` or `Untrusted`, rather than `Verified answer`. +[^definition-sql]: commons expands definitions used in custom SQL to expressions compiled for the data source. Because this is still custom SQL, the provenance outcome is `Cited` or `Untrusted`, rather than `Verified answer`. See the [DevRel Agent `data-dict.yaml`](https://github.com/posit-dev/devrel-agent/blob/main/dictionaries/devrel.data-dict.yaml) for examples of definitions. diff --git a/pkg-r/vignettes/governance.Rmd b/pkg-r/vignettes/governance.Rmd index 229b912b..b31ed50b 100644 --- a/pkg-r/vignettes/governance.Rmd +++ b/pkg-r/vignettes/governance.Rmd @@ -32,13 +32,13 @@ We assume that a model might make any request allowed by its tools. Application ### Destructive actions -The agent's SQL query tool accepts a single statement beginning with `SELECT` or `WITH`. commons rejects stacked statements and statements beginning with operations such as `INSERT`, `UPDATE`, `DELETE`, `DROP`, or `GRANT`. For data frames and pins, which commons loads into its own DuckDB database, it also disables extension loading and external filesystem access. +The agent can execute a single SQL statement beginning with `SELECT` or `WITH`. commons rejects stacked statements and statements beginning with operations such as `INSERT`, `UPDATE`, `DELETE`, `DROP`, or `GRANT`. For data frames and pins, which commons loads into its own DuckDB database, it also disables extension loading and external filesystem access. These checks provide defense in depth, but they are not a SQL parser or a database sandbox. When you supply a DBI connection, commons queries that connection as-is. The primary safeguard against destructive SQL is therefore database-enforced read-only access. ### Data access -The `tables` argument to `data_source()` controls which tables commons describes to the model. It is not an authorization boundary: model-authored SQL can query any object available to the connection. +The `tables` argument to `data_source()` controls which tables commons describes to the model. It is not an authorization boundary: SQL written by the agent can query any object available to the connection. On Posit Connect, [viewer OAuth integrations](https://docs.posit.co/connect/admin/access-controls/) can give an interactive application the current viewer's Snowflake or Databricks credentials. If the application creates its connection from those credentials, the warehouse continues to enforce that viewer's existing access policies, including row- and column-level security. commons snapshots the connection's principal, active role, and namespace when it creates a Snowflake or Databricks data source, and rejects subsequent operations if that identity changes. @@ -52,7 +52,7 @@ The SQL query runs through the DBI connection in the main application process. I ## R code execution -Model-authored R code does not run in the main application process. Each commons agent instead gets a persistent R subprocess, created on first use. Its state remains available to later R analysis by the same agent. +R code written by the agent does not run in the main application process. Each commons agent instead gets a persistent R subprocess, created on first use. Its state remains available to later R analysis by the same agent. ### OS-level sandboxing @@ -62,7 +62,7 @@ The filesystem sandbox gives the subprocess read access to R, installed packages By default, the subprocess cannot create network sockets. An application author can opt in to unrestricted network access with `commons(network = "full")`; the filesystem sandbox remains in place, but R code can then contact services reachable from the deployment and transmit data from result handles. Only enable network access when that egress is required and acceptable. -For local development on macOS, commons applies a similar filesystem and network policy using Seatbelt. Windows has no OS-level sandbox for model-authored R code. Deployed Connect applications run on Linux and use the Linux mechanisms described above. +For local development on macOS, commons applies a similar filesystem and network policy using Seatbelt. Windows has no OS-level sandbox for the agent's R session. Deployed Connect applications run on Linux and use the Linux mechanisms described above. By default, commons refuses to create an agent when an OS-level sandbox is not available. An application author can enable a local-development fallback explicitly: @@ -72,7 +72,7 @@ options(commons.allow_unsafe_fallback = TRUE) commons always uses OS-level sandboxing when it is available. This option cannot disable or bypass it; it only permits commons to fall back to best-effort R guardrails when no OS-level sandbox is available. -The fallback places checks around ordinary R functions while model-authored code evaluates. It limits filesystem reads to R, installed packages, and the worker directory, limits writes to the worker directory, denies subprocess functions, and follows the requested network policy. These checks reduce accidental access and damage, but native code and other R mechanisms can bypass them. They are not a sandbox or security boundary and should not be enabled in a deployment. +The fallback places checks around ordinary R functions while code evaluates in the agent's R session. It limits filesystem reads to R, installed packages, and the worker directory, limits writes to the worker directory, denies subprocess functions, and follows the requested network policy. These checks reduce accidental access and damage, but native code and other R mechanisms can bypass them. They are not a sandbox or security boundary and should not be enabled in a deployment. ### Communication with the main app process @@ -82,7 +82,7 @@ Communication happens through [callr](https://callr.r-lib.org/). commons seriali ## Permissioning facts -The rows returned by SQL are not the only potentially sensitive information available to an agent. Dataset descriptions and glossary entries from `data-dict.yaml` are included in the system prompt. Table documentation and sample values are supplied when a table is first used, context documents are available through search, and measure source can be read by model-authored R code. +The rows returned by SQL are not the only potentially sensitive information available to an agent. Dataset descriptions and glossary entries from `data-dict.yaml` are included in the system prompt. Table documentation and sample values are supplied when a table is first used, context documents are available through search, and measure source can be read in the agent's R session. Only include facts and source code that may be shared with both the application's viewers and its model provider. If one audience should not see a fact, use separate applications with separate context or place the facts behind viewer credentials rather than asking the model to hide it from that audience. From 3c57f8a710af40e8da534d13b8a5b916deaf4c6e Mon Sep 17 00:00:00 2001 From: Simon Couch Date: Thu, 27 Aug 2026 15:19:04 -0700 Subject: [PATCH 3/5] trim internal details from help pages --- pkg-r/R/chat.R | 12 +++---- pkg-r/R/context-layer.R | 7 ++-- pkg-r/R/data-source.R | 59 ++++++++++++---------------------- pkg-r/R/measures.R | 11 +++---- pkg-r/R/trajectory-review.R | 15 +++++---- pkg-r/man/commons_app.Rd | 12 +++---- pkg-r/man/context_layer.Rd | 7 ++-- pkg-r/man/data_source.Rd | 56 +++++++++++--------------------- pkg-r/man/measure.Rd | 11 +++---- pkg-r/man/trajectory_review.Rd | 15 +++++---- 10 files changed, 80 insertions(+), 125 deletions(-) diff --git a/pkg-r/R/chat.R b/pkg-r/R/chat.R index 0230a959..4276af84 100644 --- a/pkg-r/R/chat.R +++ b/pkg-r/R/chat.R @@ -11,13 +11,11 @@ #' @return A [shiny::shinyApp()] object. #' #' @section Citations and provenance: -#' The server verifies each `` the model writes against its -#' own context, measure definitions, and data documentation as the answer -#' streams, and rewrites verified citations inline as server-authored -#' `` elements. Citation details name the -#' trusted source. A provenance marker `` follows -#' the answer when it was produced by a governed calculation, or when a -#' fallback answer cites nothing verified. +#' The server verifies citations against trusted calculations, context, and data +#' documentation as the answer streams. Verified citations appear inline, with +#' details that name the trusted source. A provenance marker follows the answer +#' when it was produced by a governed calculation, or when a fallback answer +#' cites nothing verified. #' #' @examples #' \dontrun{ diff --git a/pkg-r/R/context-layer.R b/pkg-r/R/context-layer.R index dc70cbdf..8aaa8bf9 100644 --- a/pkg-r/R/context-layer.R +++ b/pkg-r/R/context-layer.R @@ -3,11 +3,10 @@ #' A context layer contains text that helps a [commons()] agent interpret its #' data source. #' -#' Files are chunked and indexed with \pkg{ragnar} when the agent first -#' searches its context. Facts that should be in every prompt belong in the -#' `instructions` passed to [commons()], not here. +#' Context is retrieved when relevant. Facts needed in every conversation +#' belong in the `instructions` passed to [commons()], not here. #' -#' @param files Character vector of paths to text/markdown files to index. +#' @param files Character vector of paths to text or Markdown files. #' #' @return A `commons_context_layer` object. #' diff --git a/pkg-r/R/data-source.R b/pkg-r/R/data-source.R index 6e33a1f6..ca707edf 100644 --- a/pkg-r/R/data-source.R +++ b/pkg-r/R/data-source.R @@ -12,17 +12,13 @@ #' * A `pins` board, e.g. [pins::board_connect()], is read into the same #' in-process database: each pin in `tables` becomes a table. Pin names are #' validated against the board at construction (a single listing call), but -#' each pin is downloaded only when its table is first used---by table -#' inspection, a SQL query that references it, or a measure that -#' takes the source's connection. [commons_server()] starts a background -#' process right after startup that downloads the remaining pins into the -#' local pins cache, so a first use typically only reads an -#' already-downloaded file. A table reflects the pin's +#' each pin is downloaded only when its table is first used. +#' [commons_server()] starts a background process right after startup that +#' downloads the remaining pins into the local pins cache, so a first use +#' typically only reads an already-downloaded file. A table reflects the pin's #' value at first use and is not refreshed for the lifetime of the data -#' source; if a pin can't be read (e.g. a network failure), the error -#' surfaces at that first use and the read is retried on the next one. -#' -#' The resulting object gives the agent a DBI connection plus a table registry. +#' source; if a pin can't be read (e.g. a network failure), the error surfaces +#' at that first use and the read is retried on the next one. #' #' @param ... A single DBI connection, a single `pins` board, or named data #' frames to register as tables. When passing data frames, each name becomes @@ -40,15 +36,13 @@ #' current schema. A Databricks `hive_metastore` selection must include a #' schema. Snowflake selections import semantic views, and Databricks #' selections import metric views, as native trusted metrics and dimensions. -#' Namespace selections retain lightweight model metadata, then read a -#' model's definition when the agent describes or uses it. Explicitly -#' selected models are read and validated when the data source is created. +#' Namespace selections read model definitions lazily. Explicitly selected +#' models are read and validated when the data source is created. #' Databricks wildcard members require concrete column metadata from the #' warehouse. #' An exact physical-table selection also imports associated models when -#' every physical dependency is selected. Public relationships, facts, -#' filters, and instructions become table-scoped first-touch and retrieval -#' context; private members remain hidden. +#' every physical dependency is selected. Only public relationships, facts, +#' filters, and instructions are exposed to the agent. #' #' For a board, a named character vector of pins to read: the names become #' table names, and the values are pin names passed to [pins::pin_read()]. @@ -62,31 +56,20 @@ #' @section Data dictionaries: #' A data dictionary describes a data source's tables and columns: what each #' table's rows represent, what its columns mean, allowed values and units, -#' how tables join, and definitions of domain terms. Its content reaches the -#' agent three ways: +#' how tables join, and definitions of domain terms. commons uses it to provide +#' business context and governed definitions to the agent. See +#' `vignette("commons", package = "commons")` for guidance on writing one. #' -#' * The dataset-level `description` and `details`, along with the glossary, -#' are included in the system prompt. These fields are the place for rules -#' that span tables and for guidance on which tables answer which kinds of -#' questions. -#' * The first time a conversation touches a table---via table inspection or a -#' SQL query---the table's full dictionary entry rides along with -#' the tool result: its prose, documented columns, relationships, and -#' definitions of glossary terms it references. Table inspection merges -#' documented columns with the live schema. -#' * For Snowflake and Databricks sources, a fully qualified dictionary table -#' name matches the same selected relation. A relative name is accepted when -#' it matches only one selected relation. Authored prose takes precedence, -#' while warehouse column types remain authoritative. -#' * When the agent also has a [context_layer()], the dictionary's prose is -#' indexed for context retrieval. +#' For Snowflake and Databricks sources, a fully qualified dictionary table +#' name matches the same selected relation. A relative name is accepted when +#' it matches only one selected relation. Authored prose takes precedence, +#' while warehouse column types remain authoritative. #' #' A table's entry can also declare `definitions`: named expressions in the #' [data-dict expression language](https://data-dict.tidyverse.org/expressions.html). #' commons validates their inferred types and references, compiles them for #' the source's SQL backend, and makes them available to trusted metric -#' calculations and custom SQL. Definitions are delivered through all three -#' channels above. +#' calculations and custom SQL. #' #' @section Trust: #' The agent runs only read-only `SELECT` queries; statements that @@ -96,9 +79,9 @@ #' access. These are safeguards, not a sandbox: when you supply your own #' connection, still open it in read-only mode where the backend supports it. #' Snowflake and Databricks sources snapshot the principal, active role, and -#' namespace at creation, then reject catalog and governed execution after -#' those values change. Authored and native semantic material is exposed only -#' after a zero-row query succeeds for the current principal. +#' namespace at creation, then reject catalog access and trusted calculations +#' after those values change. Authored and native semantic material is exposed +#' only after a zero-row query succeeds for the current principal. #' #' @return A `commons_data_source` object. #' diff --git a/pkg-r/R/measures.R b/pkg-r/R/measures.R index 3442fefd..b761845b 100644 --- a/pkg-r/R/measures.R +++ b/pkg-r/R/measures.R @@ -130,16 +130,13 @@ expand_measures <- function(args, env = rlang::caller_env()) { #' supply. #' #' Two return types receive special display handling: ggplots and [gt::gt()] -#' tables are shown directly to the user in the opened measure result. The model -#' is told that the plot or table has already been shown, so it can interpret the -#' result without repeating it. +#' tables are shown directly to the user in the opened measure result. #' #' For full control over a result, `fn` can return an #' [ellmer::ContentToolResult]. Its `value` is sent to the model and its -#' `extra$display` controls the shinychat display. When the display includes -#' HTML, Markdown, or text, the model is told that the result is already visible -#' to the user. An optional `extra$data` value is made available in the agent's -#' R session and removed from the result before it is returned to ellmer. +#' `extra$display` controls the shinychat display. An optional `extra$data` +#' value is made available in the agent's R session and removed from the result +#' before it is returned to ellmer. #' #' @param name Measure name. #' @param description What the measure computes. diff --git a/pkg-r/R/trajectory-review.R b/pkg-r/R/trajectory-review.R index 3aa5c840..24ac732d 100644 --- a/pkg-r/R/trajectory-review.R +++ b/pkg-r/R/trajectory-review.R @@ -34,15 +34,16 @@ #' @section Transcript contents: #' The transcript uses the same commons and shinychat renderer as live #' conversations, preserving recorded messages and tool activity. Provenance -#' markers are reconstructed from recorded provenance tags, but inline citations -#' are not recreated. Generated review documents list the recorded citation -#' decisions separately. +#' markers are reconstructed from recorded provenance outcomes, but inline +#' citations are not recreated. Generated review documents list the recorded +#' citation decisions separately. #' -#' Search-pool results are omitted because later tool calls record any selected -#' measure; other tool results are limited to 50 lines or 20,000 characters. +#' Trusted-calculation discovery results are omitted because later activity +#' records any selected calculation; other tool results are limited to 50 lines +#' or 20,000 characters. #' -#' Trust filters use each answer's provenance tag exactly as [trajectory_read()] -#' recorded it. Missing or conflicting records are omitted rather than inferred. +#' Trust filters use each answer's recorded provenance outcome. Missing or +#' conflicting records are omitted rather than inferred. #' #' Logged calls that aren't part of the agent's question-and-answer record are #' excluded from the viewer. These include shinychat's conversation-title diff --git a/pkg-r/man/commons_app.Rd b/pkg-r/man/commons_app.Rd index d8daec32..6a84c022 100644 --- a/pkg-r/man/commons_app.Rd +++ b/pkg-r/man/commons_app.Rd @@ -22,13 +22,11 @@ assemble the UI and server yourself with \code{\link[=commons_theme]{commons_the } \section{Citations and provenance}{ -The server verifies each \verb{} the model writes against its -own context, measure definitions, and data documentation as the answer -streams, and rewrites verified citations inline as numbered, -server-authored \verb{} elements. Citation details name the -trusted source. A provenance marker in a compact \verb{} follows -the answer when it was produced by a governed calculation, or when a -fallback answer cites nothing verified. +The server verifies citations against trusted calculations, context, and data +documentation as the answer streams. Verified citations appear inline, with +details that name the trusted source. A provenance marker follows the answer +when it was produced by a governed calculation, or when a fallback answer +cites nothing verified. } \examples{ diff --git a/pkg-r/man/context_layer.Rd b/pkg-r/man/context_layer.Rd index e7949783..bc027993 100644 --- a/pkg-r/man/context_layer.Rd +++ b/pkg-r/man/context_layer.Rd @@ -7,7 +7,7 @@ context_layer(files = character()) } \arguments{ -\item{files}{Character vector of paths to text/markdown files to index.} +\item{files}{Character vector of paths to text or Markdown files.} } \value{ A \code{commons_context_layer} object. @@ -17,9 +17,8 @@ A context layer contains text that helps a \code{\link[=commons]{commons()}} age data source. } \details{ -Files are chunked and indexed with \pkg{ragnar} when the agent first -searches its context. Facts that should be in every prompt belong in the -\code{instructions} passed to \code{\link[=commons]{commons()}}, not here. +Context is retrieved when relevant. Facts needed in every conversation +belong in the \code{instructions} passed to \code{\link[=commons]{commons()}}, not here. } \examples{ path <- tempfile(fileext = ".md") diff --git a/pkg-r/man/data_source.Rd b/pkg-r/man/data_source.Rd index 1451836f..4aafd617 100644 --- a/pkg-r/man/data_source.Rd +++ b/pkg-r/man/data_source.Rd @@ -24,15 +24,13 @@ table and view in that namespace. Leaving \code{tables} unset selects the current schema. A Databricks \code{hive_metastore} selection must include a schema. Snowflake selections import semantic views, and Databricks selections import metric views, as native trusted metrics and dimensions. -Namespace selections retain lightweight model metadata, then read a -model's definition when the agent describes or uses it. Explicitly -selected models are read and validated when the data source is created. +Namespace selections read model definitions lazily. Explicitly selected +models are read and validated when the data source is created. Databricks wildcard members require concrete column metadata from the warehouse. An exact physical-table selection also imports associated models when -every physical dependency is selected. Public relationships, facts, -filters, and instructions become table-scoped first-touch and retrieval -context; private members remain hidden. +every physical dependency is selected. Only public relationships, facts, +filters, and instructions are exposed to the agent. For a board, a named character vector of pins to read: the names become table names, and the values are pin names passed to \code{\link[pins:pin_read]{pins::pin_read()}}.} @@ -62,49 +60,33 @@ when the data isn't already in a database. \item A \code{pins} board, e.g. \code{\link[pins:board_connect]{pins::board_connect()}}, is read into the same in-process database: each pin in \code{tables} becomes a table. Pin names are validated against the board at construction (a single listing call), but -each pin is downloaded only when its table is first used---by table -inspection, a SQL query that references it, or a measure that -takes the source's connection. \code{\link[=commons_server]{commons_server()}} starts a background -process right after startup that downloads the remaining pins into the -local pins cache, so a first use typically only reads an -already-downloaded file. A table reflects the pin's +each pin is downloaded only when its table is first used. +\code{\link[=commons_server]{commons_server()}} starts a background process right after startup that +downloads the remaining pins into the local pins cache, so a first use +typically only reads an already-downloaded file. A table reflects the pin's value at first use and is not refreshed for the lifetime of the data -source; if a pin can't be read (e.g. a network failure), the error -surfaces at that first use and the read is retried on the next one. +source; if a pin can't be read (e.g. a network failure), the error surfaces +at that first use and the read is retried on the next one. } - -The resulting object gives the agent a DBI connection plus a table registry. } \section{Data dictionaries}{ A data dictionary describes a data source's tables and columns: what each table's rows represent, what its columns mean, allowed values and units, -how tables join, and definitions of domain terms. Its content reaches the -agent three ways: -\itemize{ -\item The dataset-level \code{description} and \code{details}, along with the glossary, -are included in the system prompt. These fields are the place for rules -that span tables and for guidance on which tables answer which kinds of -questions. -\item The first time a conversation touches a table---via table inspection or a -SQL query---the table's full dictionary entry rides along with -the tool result: its prose, documented columns, relationships, and -definitions of glossary terms it references. Table inspection merges -documented columns with the live schema. -\item For Snowflake and Databricks sources, a fully qualified dictionary table +how tables join, and definitions of domain terms. commons uses it to provide +business context and governed definitions to the agent. See +\code{vignette("commons", package = "commons")} for guidance on writing one. + +For Snowflake and Databricks sources, a fully qualified dictionary table name matches the same selected relation. A relative name is accepted when it matches only one selected relation. Authored prose takes precedence, while warehouse column types remain authoritative. -\item When the agent also has a \code{\link[=context_layer]{context_layer()}}, the dictionary's prose is -indexed for context retrieval. -} A table's entry can also declare \code{definitions}: named expressions in the \href{https://data-dict.tidyverse.org/expressions.html}{data-dict expression language}. commons validates their inferred types and references, compiles them for the source's SQL backend, and makes them available to trusted metric -calculations and custom SQL. Definitions are delivered through all three -channels above. +calculations and custom SQL. } \section{Trust}{ @@ -116,9 +98,9 @@ data frames, commons additionally disables extension loading and filesystem access. These are safeguards, not a sandbox: when you supply your own connection, still open it in read-only mode where the backend supports it. Snowflake and Databricks sources snapshot the principal, active role, and -namespace at creation, then reject catalog and governed execution after -those values change. Authored and native semantic material is exposed only -after a zero-row query succeeds for the current principal. +namespace at creation, then reject catalog access and trusted calculations +after those values change. Authored and native semantic material is exposed +only after a zero-row query succeeds for the current principal. } \examples{ diff --git a/pkg-r/man/measure.Rd b/pkg-r/man/measure.Rd index cfa84e56..5ea95d38 100644 --- a/pkg-r/man/measure.Rd +++ b/pkg-r/man/measure.Rd @@ -31,16 +31,13 @@ supply. } \details{ Two return types receive special display handling: ggplots and \code{\link[gt:gt]{gt::gt()}} -tables are shown directly to the user in the opened measure result. The model -is told that the plot or table has already been shown, so it can interpret the -result without repeating it. +tables are shown directly to the user in the opened measure result. For full control over a result, \code{fn} can return an \link[ellmer:ContentToolResult]{ellmer::ContentToolResult}. Its \code{value} is sent to the model and its -\code{extra$display} controls the shinychat display. When the display includes -HTML, Markdown, or text, the model is told that the result is already visible -to the user. An optional \code{extra$data} value is made available in the agent's -R session and removed from the result before it is returned to ellmer. +\code{extra$display} controls the shinychat display. An optional \code{extra$data} +value is made available in the agent's R session and removed from the result +before it is returned to ellmer. } \examples{ table <- data.frame(term = c("Headache", "Nausea"), count = c(7, 5)) diff --git a/pkg-r/man/trajectory_review.Rd b/pkg-r/man/trajectory_review.Rd index e47e55b2..2b6f237e 100644 --- a/pkg-r/man/trajectory_review.Rd +++ b/pkg-r/man/trajectory_review.Rd @@ -77,15 +77,16 @@ human-readable transcript for joint human-agent review. The transcript uses the same commons and shinychat renderer as live conversations, preserving recorded messages and tool activity. Provenance -markers are reconstructed from recorded provenance tags, but inline citations -are not recreated. Generated review documents list the recorded citation -decisions separately. +markers are reconstructed from recorded provenance outcomes, but inline +citations are not recreated. Generated review documents list the recorded +citation decisions separately. -Search-pool results are omitted because later tool calls record any selected -measure; other tool results are limited to 50 lines or 20,000 characters. +Trusted-calculation discovery results are omitted because later activity +records any selected calculation; other tool results are limited to 50 lines +or 20,000 characters. -Trust filters use each answer's provenance tag exactly as \code{\link[=trajectory_read]{trajectory_read()}} -recorded it. Missing or conflicting records are omitted rather than inferred. +Trust filters use each answer's recorded provenance outcome. Missing or +conflicting records are omitted rather than inferred. Logged calls that aren't part of the agent's question-and-answer record are excluded from the viewer. These include shinychat's conversation-title From 1b0c51b9e38e12663b309613dbc4efb37ee8d128 Mon Sep 17 00:00:00 2001 From: Simon Couch Date: Thu, 27 Aug 2026 15:25:55 -0700 Subject: [PATCH 4/5] address second documentation review --- pkg-r/R/commons.R | 6 ++++-- pkg-r/R/trajectory-review.R | 6 +++--- pkg-r/man/commons.Rd | 6 ++++-- pkg-r/man/trajectory_review.Rd | 6 +++--- pkg-r/vignettes/governance.Rmd | 2 -- 5 files changed, 14 insertions(+), 12 deletions(-) diff --git a/pkg-r/R/commons.R b/pkg-r/R/commons.R index 88f501c4..1df9abc5 100644 --- a/pkg-r/R/commons.R +++ b/pkg-r/R/commons.R @@ -1,7 +1,9 @@ #' Create a commons agent #' -#' `commons()` creates an [ellmer::Chat] subclass with tools for trusted -#' calculations, context search, table inspection, and code execution. +#' `commons()` creates an [ellmer::Chat] subclass with tools and prompting that +#' allow the agent to navigate its data sources, semantic layer, and context +#' layer. Depending on the agent's choice of tools, responses can be +#' deterministically tagged as high-trust or low-trust. #' #' The provider and model come from `client`; commons sets its own system prompt #' and tools. Use `agent$chat()` to ask questions, [commons_theme()] and diff --git a/pkg-r/R/trajectory-review.R b/pkg-r/R/trajectory-review.R index 24ac732d..bcc293e9 100644 --- a/pkg-r/R/trajectory-review.R +++ b/pkg-r/R/trajectory-review.R @@ -34,7 +34,7 @@ #' @section Transcript contents: #' The transcript uses the same commons and shinychat renderer as live #' conversations, preserving recorded messages and tool activity. Provenance -#' markers are reconstructed from recorded provenance outcomes, but inline +#' markers are reconstructed from recorded provenance tags, but inline #' citations are not recreated. Generated review documents list the recorded #' citation decisions separately. #' @@ -42,8 +42,8 @@ #' records any selected calculation; other tool results are limited to 50 lines #' or 20,000 characters. #' -#' Trust filters use each answer's recorded provenance outcome. Missing or -#' conflicting records are omitted rather than inferred. +#' Trust filters use each answer's provenance tag exactly as [trajectory_read()] +#' recorded it. Missing or conflicting records are omitted rather than inferred. #' #' Logged calls that aren't part of the agent's question-and-answer record are #' excluded from the viewer. These include shinychat's conversation-title diff --git a/pkg-r/man/commons.Rd b/pkg-r/man/commons.Rd index 7f9439dd..b3aab9eb 100644 --- a/pkg-r/man/commons.Rd +++ b/pkg-r/man/commons.Rd @@ -71,8 +71,10 @@ need at least a publisher account.} An \link[ellmer:Chat]{ellmer::Chat} subclass. } \description{ -\code{commons()} creates an \link[ellmer:Chat]{ellmer::Chat} subclass with tools for trusted -calculations, context search, table inspection, and code execution. +\code{commons()} creates an \link[ellmer:Chat]{ellmer::Chat} subclass with tools and prompting that +allow the agent to navigate its data sources, semantic layer, and context +layer. Depending on the agent's choice of tools, responses can be +deterministically tagged as high-trust or low-trust. } \details{ The provider and model come from \code{client}; commons sets its own system prompt diff --git a/pkg-r/man/trajectory_review.Rd b/pkg-r/man/trajectory_review.Rd index 2b6f237e..d8811dc9 100644 --- a/pkg-r/man/trajectory_review.Rd +++ b/pkg-r/man/trajectory_review.Rd @@ -77,7 +77,7 @@ human-readable transcript for joint human-agent review. The transcript uses the same commons and shinychat renderer as live conversations, preserving recorded messages and tool activity. Provenance -markers are reconstructed from recorded provenance outcomes, but inline +markers are reconstructed from recorded provenance tags, but inline citations are not recreated. Generated review documents list the recorded citation decisions separately. @@ -85,8 +85,8 @@ Trusted-calculation discovery results are omitted because later activity records any selected calculation; other tool results are limited to 50 lines or 20,000 characters. -Trust filters use each answer's recorded provenance outcome. Missing or -conflicting records are omitted rather than inferred. +Trust filters use each answer's provenance tag exactly as \code{\link[=trajectory_read]{trajectory_read()}} +recorded it. Missing or conflicting records are omitted rather than inferred. Logged calls that aren't part of the agent's question-and-answer record are excluded from the viewer. These include shinychat's conversation-title diff --git a/pkg-r/vignettes/governance.Rmd b/pkg-r/vignettes/governance.Rmd index b31ed50b..7f4a5549 100644 --- a/pkg-r/vignettes/governance.Rmd +++ b/pkg-r/vignettes/governance.Rmd @@ -24,8 +24,6 @@ This vignette explains the boundaries that commons provides and the responsibili A commons agent is an ellmer chat with a system prompt and a set of tools. The system prompt excerpts available data sources and business context. Tools let the model retrieve more context, call a trusted calculation, run a SQL query, or execute R code. -The exact set depends on the agent's layers and data sources. The current tool names and their lifecycle status are documented in [commons()]. - We assume that a model might make any request allowed by its tools. Application security should therefore not depend on the model following an instruction like "never reveal sensitive data." Instead, only give an agent access to data that the current user of the application is allowed to see. The system prompt, tool arguments, and tool results are also sent to the model provider, so the agent should not have access to data that you [do not trust that provider to process](https://posit.co/blog/trust-llm-tools). ## SQL code execution From 669eda43701771683c062abd5524063f83fff964 Mon Sep 17 00:00:00 2001 From: Simon Couch Date: Thu, 27 Aug 2026 16:41:08 -0700 Subject: [PATCH 5/5] refine agent documentation --- pkg-r/R/commons.R | 6 ++++-- pkg-r/man/commons.Rd | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/pkg-r/R/commons.R b/pkg-r/R/commons.R index 1df9abc5..dd1eff91 100644 --- a/pkg-r/R/commons.R +++ b/pkg-r/R/commons.R @@ -3,7 +3,8 @@ #' `commons()` creates an [ellmer::Chat] subclass with tools and prompting that #' allow the agent to navigate its data sources, semantic layer, and context #' layer. Depending on the agent's choice of tools, responses can be -#' deterministically tagged as high-trust or low-trust. +#' deterministically classified as based on a trusted calculation, cited, or +#' untrusted. #' #' The provider and model come from `client`; commons sets its own system prompt #' and tools. Use `agent$chat()` to ask questions, [commons_theme()] and @@ -65,7 +66,8 @@ #' * `search_context` retrieves relevant business context. #' * `describe_table` inspects a table or semantic model. #' * `run_sql` executes a read-only SQL query. -#' * `run_r` analyzes results and renders plots in the agent's R session. +#' * `run_r` executes R code to analyze results and render plots in the agent's +#' R session. #' #' These model-facing tools should be considered private. Their constructors #' are intentionally not exported, and their names, arguments, availability, diff --git a/pkg-r/man/commons.Rd b/pkg-r/man/commons.Rd index b3aab9eb..665ea8f0 100644 --- a/pkg-r/man/commons.Rd +++ b/pkg-r/man/commons.Rd @@ -74,7 +74,8 @@ An \link[ellmer:Chat]{ellmer::Chat} subclass. \code{commons()} creates an \link[ellmer:Chat]{ellmer::Chat} subclass with tools and prompting that allow the agent to navigate its data sources, semantic layer, and context layer. Depending on the agent's choice of tools, responses can be -deterministically tagged as high-trust or low-trust. +deterministically classified as based on a trusted calculation, cited, or +untrusted. } \details{ The provider and model come from \code{client}; commons sets its own system prompt @@ -95,7 +96,8 @@ agent receives some combination of these tools: \item \code{search_context} retrieves relevant business context. \item \code{describe_table} inspects a table or semantic model. \item \code{run_sql} executes a read-only SQL query. -\item \code{run_r} analyzes results and renders plots in the agent's R session. +\item \code{run_r} executes R code to analyze results and render plots in the agent's +R session. } These model-facing tools should be considered private. Their constructors