Repository navigation
feat: add llm:chat-all for batched per-agent calls - #83
Merged
Merged
Conversation
One request per agent, sent together, one wait. A model that runs `ask turtles [ llm:chat ... ]` waits for each turtle in turn; chat-all takes about as long as the slowest single reply. - `llm:chat-all agentset [ -> prompt ]` reports [agent reply ok?] triples; the prompt reporter runs as each agent, like ask - optional schema argument reuses the structured-output path and returns parsed [key value] lists per agent - a failed call is reported in that agent's triple and never aborts the batch; no history is committed for it - every prompt is built before any request is sent, so a prompt error costs no requests - requests go through each agent's provider, so throttling, retry, profiles, history and usage all apply per agent - the batch waits while replies keep arriving and gives up only after a full budget with no reply, so a hung provider costs one budget, not one per agent; late replies and halted waits never write history Closes #53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #53.
What it does
Every agent in a set asks the LLM its own question at the same time, and the model waits once for all the replies.
ask turtles [ llm:chat ... ]waits for each turtle in turn;llm:chat-alltakes about as long as the slowest single reply.ask, so it reads that agent's own variables.[agent reply ok?]. A failed call only marks its own entryfalse, carrying the error message; the rest of the batch still completes.llm:chat-with-schema), so each reply arrives as NetLogo values.Design notes
parseObjectSchemahelper;llm:chat-with-schema's error messages are unchanged.Verification
sbt test: 268/268, including 14 newtests.txtcases. They cover triples, prompts running as each agent, per-agent history and usage, profiles, failure isolation, schema parsing and bad schemas, non-string prompts, an empty agentset, concurrency, a hung batch costing one budget, progress extending the wait, and rejecting extra schema arguments.openai/gpt-oss-20b): 48/48, including the new T11 (6 turtles through a throttle cap of 3, plus a schema batch).ask turtles [ llm:chat ]versus 1.22 s withllm:chat-all.chat-allcalls the same provider method thatThrottleIntegrationSpeccovers, and T11 exercises a cap live.Docs
docs/API-REFERENCE.md: a quick-reference row, a### llm:chat-allsection, and a "Batched Calls" usage pattern.