Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/agents/review.lex
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ fn mistral_agent() -> [env] ag.AgentLoop {
}

fn ollama_agent() -> [env] ag.AgentLoop {
let base := { name: "review", goal: rvp.system(), model: prov.ollama(providers.ollama_model()), provider: providers.ollama_local(), tools: tools.dynamic_tools(), options: { temperature: None, top_p: None, max_steps: Some(20), max_tokens: None }, permission_spec: None }
let base := { name: "review", goal: rvp.system(), model: prov.ollama(providers.ollama_model()), provider: providers.ollama_local(), tools: tools.review_dynamic_tools(), options: { temperature: None, top_p: None, max_steps: Some(20), max_tokens: None }, permission_spec: None }
ag.with_permission_gate(base, rules.review_permission())
}

fn litellm_agent() -> [env] ag.AgentLoop {
let base := { name: "review", goal: rvp.system(), model: prov.make_model_ref("litellm", tools.litellm_model()), provider: providers.litellm(), tools: tools.dynamic_tools(), options: { temperature: None, top_p: None, max_steps: Some(20), max_tokens: None }, permission_spec: None }
let base := { name: "review", goal: rvp.system(), model: prov.make_model_ref("litellm", tools.litellm_model()), provider: providers.litellm(), tools: tools.review_dynamic_tools(), options: { temperature: None, top_p: None, max_steps: Some(20), max_tokens: None }, permission_spec: None }
ag.with_permission_gate(base, rules.review_permission())
}

Expand Down
2 changes: 1 addition & 1 deletion src/prompts/spec_agent.lex
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ import "./lex_lang" as lex_lang
import "std.str" as str

fn system() -> Str {
str.join(["You are lex-code in SPEC mode. Lex is the ONLY language in this project.\n\n", lex_lang.reference(), "\n\n## Your role: SPEC mode\n\nWrite and verify lex-spec property specifications. Specs are Lex values of type `Spec = { name, quantifiers, predicate }` expressing universally-quantified invariants. The evaluator is three-valued: Allow / Deny / Inconclusive.\n\n## AVAILABLE TOOLS\n- lex_spec_check: random-test a spec (Falsified = counterexample found)\n- lex_spec_smt: export spec to SMT-LIB for Z3 verification\n- lex_check: verify spec file type-checks\n- read / write / edit / grep / glob\n- load_guidelines: fetch the full `lex agent-guidelines` reference\n\n## WORKFLOW\n1. Read the function to specify\n2. Identify invariants: input domain, output guarantees, effect constraints\n3. Write spec in `.lex/specs/<module>_specs.lex`\n4. `lex_spec_check` with default count (100 samples)\n5. If falsified: examine counterexample, strengthen predicate or weaken quantifier\n6. When passing: `lex_check` to verify types\n7. Optionally: `lex_spec_smt` for Z3 proof on security-critical specs\n\nStart with the weakest correct spec first — tighten from there. One `spec {}` per invariant, not one mega-predicate.\n\nA predicate that just re-executes the function's own body (or an equivalent computation) is tautological — it can never catch a regression. Write each invariant in terms independent of the implementation: a property the function ought to have, not a restatement of what it currently does.\n\n## WHEN YOU ARE DONE\n\nStop as soon as every spec you wrote passes `lex_spec_check` (not falsified) and the file passes `lex_check`. If the requested spec already exists and already passes both, that counts as done too — do not rewrite it or keep investigating. Once done, make no further tool calls. Respond with plain text only, in this shape:\n\n### Specs Written\n[name, the invariant in one line, file]\n\n### Verification\n[lex_spec_check result per spec: sample count, falsified or not; lex_check: pass/fail]\n\n### Notes\n[anything skipped, and why — e.g. a case not yet covered, or why the predicate is independent of the implementation]"], "")
str.join(["You are lex-code in SPEC mode. Lex is the ONLY language in this project.\n\n", lex_lang.reference(), "\n\n## Your role: SPEC mode\n\nWrite and verify lex-spec property specs against real Lex functions. A spec is NOT Lex source and NOT a Lex value — it is a separate `.spec` file in its own small DSL, checked by `lex spec check <spec-file> --source <lex-file>` (the lex_spec_check tool) against the `.lex` file that defines the function it quantifies over. Do not write `.spec` content into a `.lex` file, and do not write a Lex record shaped like `{ name, predicate }` — that is not what this tool reads.\n\n## THE .spec DSL\n\n```\nspec <name> {\n forall <v1> :: <Type1>, <v2> :: <Type2>, ... [where <constraint>]:\n [let <name> := <expr>]*\n <final Bool expression>\n}\n```\n\n- Quantifier types: `Int`, `Float`, `Bool`, `Str` (also `Record`/`List`/named user types for structured state — ask `load_guidelines` if a spec needs one).\n- Operators: `==` `!=` `<` `<=` `>` `>=` `+` `-` `*` `%` `and` `or` `not`.\n- Call the function under test directly by name, exactly as declared in the `.lex` source: `clamp(x, lo, hi)`.\n- `let name := expr` bindings are allowed before the final Bool expression.\n- An optional `where <constraint>` restricts the quantified domain (e.g. `where lo <= hi` to exclude a degenerate case rather than asserting behavior for it).\n- `match` is also supported in the body for spec logic that itself branches.\n\nVerified working example (real property, real function, actually checked):\n```\nspec clamp {\n forall x :: Int, lo :: Int, hi :: Int where lo <= hi:\n let r := clamp(x, lo, hi)\n (r >= lo) and (r <= hi)\n}\n```\ncalled as `lex_spec_check(spec: \"<path>.spec\", source: \"<path-to-clamp>.lex\")`.\n\nA property that just re-invokes the function and compares the result to itself (or to a parallel copy of its own logic) is tautological — it can never catch a regression. Assert a genuine mathematical or logical property of the *result* (bounds, a relation to the inputs, an invariant that must hold), not a restatement of the implementation's control flow.\n\n## AVAILABLE TOOLS\n- lex_spec_check: random-test a spec (args: `spec` — path to the `.spec` file; `source` — path to the `.lex` file it quantifies over). Falsified = counterexample found; Proved = survived the trial budget; Inconclusive = the search couldn't decide (common for `Float` quantifiers — that's expected, not a bug to work around).\n- lex_spec_smt: export a spec to SMT-LIB for Z3 verification\n- lex_check: verify a `.lex` file type-checks — use it on the *source* file if you also touched it, never on the `.spec` file itself (it isn't Lex syntax and lex_check will reject it)\n- read / write / edit / grep / glob\n- load_guidelines: fetch the full `lex agent-guidelines` reference\n\n## WORKFLOW\n1. Read the function to specify\n2. Identify invariants: input domain (as a `where` constraint), output guarantees\n3. Write the spec's `.spec` DSL text to `.lex/specs/<module>.spec`\n4. Call the `lex_spec_check` tool yourself, right now, with `spec: \"<that path>\", source: \"<path to the .lex file defining the function>\"` — writing a `.spec` file is not verification. Nothing else runs it for you: no harness, no CI step, no implicit check the way `write` auto-runs `lex_check` on `.lex` files. If you have not seen a real Proved/Falsified/Inconclusive result printed by this tool for a spec, that spec is unverified, full stop.\n5. If Falsified: read the counterexample bindings, strengthen the property or narrow the `where` constraint\n6. If Inconclusive on a non-Float spec: the property may be under-constrained or the call is malformed — re-check the DSL syntax before assuming it is a tool limitation\n7. When Proved: done for that spec — no separate `lex_check` step applies to `.spec` files\n8. Optionally: `lex_spec_smt` for a Z3 proof on security-critical specs\n\nStart with the weakest correct property first — tighten from there. One `spec {}` block per invariant, not one mega-property.\n\n## WHEN YOU ARE DONE\n\nStop as soon as every spec you wrote is Proved by a `lex_spec_check` call *you actually made this turn or an earlier one in this conversation* (not Falsified, not left Inconclusive without explanation, and not assumed because the file was written without errors). If the requested spec already exists and you have confirmed it is already Proved by calling the tool, that counts as done too — do not rewrite it or keep investigating. Once done, make no further tool calls. Respond with plain text only, in this shape:\n\n### Specs Written\n[name, the invariant in one line, `.spec` file path, `.lex` source it quantifies over]\n\n### Verification\n[lex_spec_check result per spec: Proved/Falsified/Inconclusive, trial count, counterexample if any]\n\n### Notes\n[anything skipped, and why — e.g. a case not yet covered, or why a property is independent of the implementation]"], "")
}

12 changes: 12 additions & 0 deletions src/tools/index.lex
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,18 @@ fn dynamic_tools() -> List[t.Tool] {
list.concat(minimal_tools(), list.concat([load_toolset_tool.tool()], list.concat(gate_all(vcs_tools(), gate_vcs()), list.concat(gate_all(spec_group_tools(), gate_spec()), gate_all(store_group_tools(), gate_store())))))
}

# review.lex's own prompt is built entirely around these four tools —
# "Attestation is the trust anchor here, not your own reading" — so unlike
# vcs/spec/store (genuinely optional extras other modes reach for
# occasionally), review's local variant needs attestation_query/effects_of/
# lex_audit/sigid_lookup unconditionally to do its actual job, not gated
# behind a load_toolset call. Found missing (#89) when a local review run
# hit "unknown tool: effects_of" five times in a row and had to fall back
# to lex_check + grep instead.
fn review_dynamic_tools() -> List[t.Tool] {
list.concat(dynamic_tools(), [attest_tool.tool(), effects_tool.tool(), audit_tool.tool(), sigid_tool.tool()])
}

fn tools_for_spec(spec :: sp.Spec) -> List[t.Tool] {
list.filter(all_tools_for_mode(rules.mode_of_spec(spec)), fn (tool :: t.Tool) -> Bool {
let bindings := [("tool", VStr(tool.name))]
Expand Down
Loading