Repository navigation
Release 1.1.0: NetLogo 7.0.4, bug fixes, model catalog, decision API with Jev - #85
Merged
Merged
Conversation
- netLogoVersion 7.0.4, extension plugin 7.0.4, sbt 1.10.1, version 1.1.0 - Pin org.nlogo:netlogo to netLogoVersion: plugin 7.0.4 pulls netlogo-prims-json 0.1.1, which depends on NetLogo 7.1.0-alpha1 and otherwise evicts 7.0.4 - Pin scalatest 3.2.19 to match NetLogo's tests jar - Add a copy of org.nlogo.util.AnyFunSuiteEx, missing from netlogo-7.0.4-tests.jar, so TestLanguage suites compile (#45) - NetLogo 7.0.4 is compiled for Java 21: CI, build.sh and docs move to JDK 21 - Unwrap ujson 4.4 trace exceptions so JSON parse errors stay readable - Zip extras fail the build when missing, flatten curated demos, and ship a tracked placeholder config.txt instead of the local one (#67)
- GeminiProvider splits system messages out of contents and sends them as top-level systemInstruction.parts[0].text, joined with a blank line when there are several (#28) - contents now holds only user/model turns in order; no systemInstruction key when there are no system messages - generationConfig (structured output, thinkingConfig) is unchanged - StructuredOutputRequestSpec: two tests covering systemInstruction text, role order, and the no-system case
- ModelRegistry keeps the bundled catalog separately; loadOverride always rebuilds from it instead of merging into the current map (#29) - A directory with no, empty, or broken models-override.yaml now drops any earlier override - Add ModelRegistry.clearOverride(); llm:load-config and llm:list-models call it when the model is unsaved (no model path) - Add ModelRegistrySpec covering load A then B, broken override, and clearOverride - Risk: low; only override state handling changes, bundled catalog behaviour is unchanged
- Add scripts/check-zip.sh: asserts the packageZip output holds llm.jar, docs/README.md, demos/config-reference.txt, demos/config.txt and the flattened demos (color-sharing, social-deduction-game, topology-tournament, provider-sensitivity), and that demos/config.txt matches demos/config-placeholder.txt. - Run sbt packageZip and the check after tests in .github/workflows/ci.yml. - Regression guard for #67 (#67-check); no Scala changes.
- Parse OpenAI-compatible content and finish_reason null-safely so a refusal body (content: null plus "refusal") no longer throws before usage is read (#82) - Add refusal: Option[String] to Choice, with ChatResponse.refusal - Map Anthropic stop_reason "refusal" to the same field - Every chat primitive (chat, chat-async, chat-all, chat-with-template, choose, chat-with-schema, chat-json, chat-with-thinking) records usage, then fails with "model refused: <text>" and commits nothing to history - Tests: refusal parsing in UsageParsingSpec; __TEST_REFUSE marker in DeterministicTestProvider with two tests.txt cases Refs #82
…format - Fail non-429 HTTP errors with a typed HttpStatusException carrying the status code - On HTTP 400 whose body names response_format/json_schema/json_object/structured or unsupported, resend once with responseFormat = None - Print a one-time NOTE per (provider, model) via notifyResponseFormatUnsupported - Other 400s (bad key, unknown model) still fail after one send - Add SchemaFallbackSpec covering the fallback and the non-fallback 400 Refs #81
- Treat HTTP 500/502/503/504, Anthropic 529 and connect/read/timeout failures as transient in BaseHttpProvider.executeWithRetry, using the same RetryPolicy backoff, jitter and budget as 429. Other 4xx still fail on the first attempt. - Generalise the wait notice (notifyRetryWait) to name the provider, status or transport error, delay and attempt number. - Final errors name the provider, HTTP status or transport error, a short reason and one next step; Ollama asks whether the server runs. - RetrySpec: 500,500,200 now succeeds; 529/502/503/504 retried; 4xx not retried; connect failures and timeouts retried; friendly messages. - docs/SETUP.md: retry keys cover all transient errors. Refs #48-retry (#48 remainder, #60) fix: report unknown hosts and Gemini bad keys accurately - Detect UnresolvedAddressException/UnknownHostException in the cause chain; java.net.http raises it as a ConnectException, so a mistyped base URL was reported as "connection refused" and retried. It now fails at once with "(unknown host)" and points at the base URL setting. - Treat an error body containing API_KEY_INVALID (Gemini's HTTP 400) as an invalid key so the message names the API key setting. - Rename the wait counters to retryWaitCount/retryWaitTotalMs/ resetRetryWaitStats, since they now count 5xx, 529 and network waits too. - Add RetrySpec cases for both message fixes.
- Add ConfigStore.resolveBaseUrl: provider-specific key, then generic base_url, then provider default - Use it for BaseHttpProvider request URL and throttle key, GeminiProvider, and OllamaProvider reachability/model listing - Stop seeding base_url in ConfigStore.withDefaults, ProviderFactory.getDefaultConfig and provider initializeDefaults, so any value present was set by the user - llm:set-provider (ProviderFactory.applyProviderSwitch) moves a generic base_url to the previous provider's own key and clears it, so it never leaks to the new provider - isOllamaReachable (LLMExtension.probeOllama) probes the resolved Ollama URL instead of always localhost:11434 - Add BaseUrlResolutionSpec (BackendStub) covering OpenAI, Gemini, Anthropic and Ollama resolution Refs #25
- Add a volatile run generation counter, bumped by clearAll under both historyLock and usageLock before the maps are cleared - chat, chat-async, chat-all, chat-with-template, choose, chat-with-schema, chat-json and chat-with-thinking capture the generation at launch - commitExchange and recordUsage take that generation and skip the write inside their lock when it is stale; a stale async reporter still returns its value to the caller - tests.txt: LLMChatAsyncAfterClearAllWritesNothing and LLMChatAsyncAfterClearAllSkipsUnawaitedReply Fixes #80
- executeWithRetry read only the provider-specific key, so after #25 an error could name the default URL while requests went to base_url
… Manager - JDK 21 and sbt 1.10.1 requirements, isSnapshot handling, zip check script - NetLogo-Libraries steps against the 7.0 branch, NetLogo org URLs
- models.yaml takes an optional per-provider default: (mapping form with default:/models:, plain lists still accepted); ModelRegistry.defaultModel reads it and falls back to the ProviderDescriptor default - optional user catalog at ~/.netlogo-llm/models.yaml (override with the llm.models.user.file property or NETLOGO_LLM_MODELS) merged additively over the bundled catalog; a broken file warns and is ignored; models-override.yaml still layers on top and keeps the lower default when it names none - drop the hardcoded FALLBACK_CONFIG; if models.yaml cannot load, each provider falls back to its descriptor default only - every provider class, ProviderFactory and ConfigStore.withDefaults read the default through ModelRegistry, so llm:set-model / model= precedence is unchanged - refresh the catalog: defaults gpt-6-luna, claude-haiku-5-5, gemini-3.5-flash-lite, openai/gpt-oss-20b (groq, openrouter), Qwen/Qwen3.8-Flash, llama3.2; retired and retiring IDs removed - treat OpenAI gpt-5.x/gpt-6.x as reasoning models (developer role, max_completion_tokens, no temperature), keeping o-series handling - Gemini 3+ gets thinkingLevel and never thinkingLevel with thinkingBudget; 2.5 gets thinkingBudget only - ModelCatalogSpec covers defaults, fallback, user catalog and layering; RegistryLock serializes suites that mutate the global registry Refs #64 docs: document the model catalog and refresh stale model IDs - CONFIGURATION.md: "Model Catalog" section covering default:, the user catalog at ~/.netlogo-llm/models.yaml (llm.models.user.file / NETLOGO_LLM_MODELS), and how models-override.yaml layers on top - API-REFERENCE.md, PROVIDER-GUIDE.md, SETUP.md, EXAMPLES.md, README.md, USAGE.md: new defaults and current model IDs; GPT-5.x/6.x reasoning format and Gemini 3 thinkingLevel notes - demos/config-reference.txt and config-placeholder.txt: current IDs - LLMExtension comment no longer names the old OpenAI default Refs #64
…backend - Add vendor-neutral DecisionQuestion (choice, score, yes-no) and DecisionResult models, and a DecisionProvider trait that any decision-style API implements - Add llm:decide prompt spec reporting [answer probability distribution]; prompt only, no history, usage recorded per agent with the run generation; spec validated before any request - Add llm:decide-all mirroring llm:chat-all: concurrent sends through the throttle, one wait, per-agent [agent result ok?] triples - Add TypeSafeProvider (POST /v1/systemone, bearer auth) mapping choice, score and noul answers, usage and input-token cost; register "typesafe" with jev-latest, jev-preview, jev-1.13.0, jev-1.14.0 - llm:choose uses the native choice question on a decision provider; the free-text primitives raise a clear error on typesafe; providers without a decision API raise a clear error from llm:decide / llm:decide-all - Tests: TypeSafeProviderSpec (BackendStub request/response mapping, 401), DeterministicDecisionTestProvider and tests.txt cases for decide, decide-all, validation, choose and the text-primitive error - Docs: API reference decision section, provider guide, configuration guide and config-reference typesafe block
- T12 covers llm:decide (yes-no, choice, score types and distributions), llm:choose on a decision provider, llm:decide-all, cost tracking, and the refusal of free-text chat - typesafe added to the provider chooser; it runs only T12 - Replace the retired claude-3-5-haiku-latest alternate model Verified live: 13/13 against jev-latest; 20-turtle decide-all in 0.18 s
…gets Release follow-ups (followups items 1-3, 6): - llm:choose on a DecisionProvider applies llm:decide's choice-spec rules (1-255 distinct non-empty options) before sending, and raises a clear error when the provider picks an option outside the list - user catalog and models-override.yaml sections add their own default: to their model list, so a default named only under default: is valid and no "not in the known model list" warning fires - OpenAI-format reasoning requests with max_tokens still at the provider default send max_completion_tokens=4000; a user-set value is unchanged - an empty reply with finish_reason=length raises an error telling the user to raise max_tokens instead of returning an empty string - llm:decide-all on a provider without a decision API already raises the "has no decision API" message (covered by LLMDecideNeedsDecisionProvider) - tests: LLMChooseOnDecisionProviderValidatesOptions, ModelCatalogSpec default-not-in-list cases, ReasoningBudgetSpec - docs: API-REFERENCE, CONFIGURATION, PROVIDER-GUIDE docs: add TypeSafe setup and refresh stale Together and OpenRouter model IDs Release follow-ups (followups item 4): - docs/SETUP.md: typesafe in the provider list, a TypeSafe (Jev) setup section, and its provider-specific defaults - docs/EXAMPLES.md, docs/TESTING.md: deepseek-ai/DeepSeek-R1 -> deepseek-ai/DeepSeek-V4.1-Flash - OpenRouterProvider and TogetherProvider doc comments use current IDs chore: replace retired model IDs in demo config and test model Release follow-ups (followups item 5): - demos/color-sharing/config.txt: model comment lists use current OpenAI, Anthropic and Gemini IDs from models.yaml - demos/tests/tests.nlogox: anthropic/claude-3.5-haiku -> anthropic/claude-haiku-5.5 in the OpenRouter vendor-prefix test (string literals only; compile gate passes on NetLogo 7.0.4) chore: replace remaining retired model IDs in demo configs - demos/config: Anthropic and Gemini examples use claude-haiku-5-5 and gemini-3.5-flash-lite - demos/tests/config.txt.example: Gemini, OpenRouter and Together model lists and the Together default use current IDs from models.yaml - demos/baba-is-ai/README.md: config snippet uses current IDs fix: honour user-set max_tokens on reasoning models and count empty replies - ConfigStore tracks seeded defaults (setDefault/isDefault); seeds in withDefaults, provider switch and provider init use it, and ProviderFactory.copyConfig keeps the marker when building a provider - OpenAICompatibleProvider lifts max_completion_tokens to 4000 only when max_tokens is still the seeded default, so an explicit max_tokens=1000 is sent as 1000 - The empty-reply-at-token-limit error moves from response parsing to the primitives (rejectEmptyReply after recordUsage), so billed tokens still reach llm:usage; the message names the output tokens used - DeterministicTestProvider __TEST_TRUNCATE hook, tests.txt case and ReasoningBudgetSpec cases; PROVIDER-GUIDE.md and models.yaml comment updated
- demos/el-farol-decisions: 150 persona patrons decide each night with llm:decide-all (crowd forecast score, go-out yes-no, bar choice) and either sample from Jev's probabilities or take the top pick - text LLM backend uses llm:choose per patron to show the latency gap - monitors for ms and cost per night, total cost and failed calls; plots for attendance vs capacity, mean P(go) vs fraction out, and forecast - embedded headless-check experiment, run live on Jev in both modes - config.txt.example and config-text.txt.example; real configs stay ignored - ship the model in the release zip (build.sbt, check-zip.sh, .gitignore) - link the demo from API-REFERENCE and PROVIDER-GUIDE fix: harden El Farol demo config and text backend - Open the model code with two ABOUTME lines. - Text backend clears each patron's chat history before llm:choose, so the prompt alone carries memory and per-night cost stays flat. - Print the first per-call error once per night on the text backend. - Setup rejects a config.txt whose provider is not typesafe for the Jev backend, with the exact lines to add; the zip placeholder config gains a commented TypeSafe block and the Info tab and README explain it. - go reloads the config when the backend chooser changed since setup.
- Bars get a sign row with name, crowd vs capacity and a FULL marker that patrons never cover; a street with a centre line; homes on a lawn with a live stayed-home count - show-observations? prints the prompt the active backend actually sends - A missing config file explains exactly which lines to write, so zip users without the .example files are not stuck
- 1 - 0.56 showed as 0.43999999999999995 in llm:decide results; compute it with BigDecimal so monitors read 0.44
- docs/README.md lists Groq and TypeSafe (Jev), the decision and batching primitives, and a maintainer section: layout, adding a provider or a decision backend, catalog upkeep, tests, and the features decided against - docs/TESTING.md documents the e2e harness (T1-T12) and its headless run - Extension Manager long description mentions Jev and typed decisions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Final release of the extension: NetLogo 7.0.4 build, bug fixes, a refreshed model catalog, and a generic decision API with TypeSafe (Jev) as its first backend.
Build
org.nlogo:netlogoto 7.0.4, since plugin 7.0.4 pulls netlogo-prims-json 0.1.1, which depends on 7.1.0-alpha1 and otherwise evicts 7.0.4.org.nlogo.util.AnyFunSuiteEx, whichnetlogo-7.0.4-tests.jaromits. This is the cause of CI broken on NetLogo 7.0.4: Scala 3.7 compiler crash compiling TestLanguage suites #45, not a Scala compiler bug.config.txt. CI checks the zip contents withscripts/check-zip.sh.Fixes
clear-allno longer write history or usage into the new run.llm:chooseand the structured primitives resend once withoutresponse_formatwhen a model rejects it.model refused: ....base_urlconfigs are not honored consistently #25 one base-URL resolver; legacybase_urlis honoured and never leaks to another provider.systemInstruction.Features
models.yaml(default:); optional user catalog at~/.netlogo-llm/models.yaml. The catalog is refreshed for current models (Claude 5.5, GPT-6, Gemini 3.x, Groq, Together, OpenRouter), and the hardcoded fallback list is removed. Adds the GPT-5/6 reasoning request format and Gemini 3thinkingLevel.llm:decide/llm:decide-all: typed decisions (choice, score, yes-no) reporting[answer probability distribution], through a vendor-neutralDecisionProvidertrait. TypeSafe (Jev) is the first backend;llm:chooseworks on it too.demos/el-farol-decisions: sampling from probabilities versus always taking the top pick.Testing
sbt test: 361 passing (deterministic, no network).llm:decide-allin 0.18 s).Closes #45, closes #64, closes #67, closes #80, closes #81, closes #82, closes #25, closes #28, closes #29, closes #60.