feat(desktop): add Together AI as a provider option - #3055
Conversation
Together speaks stock OpenAI Chat Completions, so it needs no new wire
dialect in buzz-agent — only the preset that relay-mesh already
established: store `together` as the provider and derive
BUZZ_AGENT_PROVIDER=openai plus Together's base URL at spawn.
The credential stays TOGETHER_API_KEY everywhere the user can see it
(picker, readiness gate, model discovery) and is mapped onto
OPENAI_COMPAT_API_KEY only at spawn, so nothing asks for a key the
dialog never offers. Both OpenAI keys are scrubbed before the mapped one
is written back, so an unrelated key in the env cannot reach Together.
Model discovery needed two accommodations: Together answers /v1/models
with a bare array rather than OpenAI's {"data": [...]} envelope, and its
catalog mixes ~2k endpoints across every modality, so only entries
declaring type "chat" are offered.
Signed-off-by: jordan <jkail@together.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
Verification update — I ran this against the live Together API with a real key, plus the full local CI suite. Live, through the production code paths (not reimplementations):
UI — confirmed in the running app that "Together AI" lands in the Agent defaults dropdown between OpenAI-compatible and Buzz shared compute, and that selecting it asks for a "Together AI API Key" (required) rather than the OpenAI label, with Model required and discovery running. Suites — Three smoke failures showed up in the full parallel run and are not from this change — I checked each against
No test scaffolding is included in the PR; the live harnesses above were temporary and the branch is unchanged since it was opened. |
Only buzz-agent gets BUZZ_AGENT_PROVIDER rewritten to Together's OpenAI-compatible transport at spawn. Nothing rewrites GOOSE_PROVIDER, and goose ships no provider under this id, so a goose agent configured with Together would persist GOOSE_PROVIDER=together and fail at startup. Gate the option on the runtime inside getPersonaProviderOptions rather than at each picker, so none of the four call sites has to remember the exclusion. The (current) tail still renders, so an agent already saved with Together keeps showing its value instead of appearing unset. This mirrors the existing relay-mesh treatment in AgentConfigFields, which hides Buzz shared compute for the same reason. Signed-off-by: jordan <jkail@together.ai> Co-authored-by: Cursor <cursoragent@cursor.com>
2 participants
Summary
Adds Together AI to the provider picker in Agent defaults, agent templates, and per-agent config.
Together speaks stock OpenAI Chat Completions, so this needs no new wire dialect in
buzz-agent. It follows the preset patternrelay-mesh("Buzz shared compute") already established: the desktop storestogetheras the provider and derivesBUZZ_AGENT_PROVIDER=openaiplus Together's base URL at spawn, in a newmanaged_agents/together.rs. TheProviderenum,Llm::complete, and the effort tables are untouched.Three things follow from Together naming its own credential and shipping an unusually large catalog:
TOGETHER_API_KEYeverywhere the user can see it — picker, readiness gate, model discovery — and is mapped ontoOPENAI_COMPAT_API_KEYonly at spawn. Requiring the mapped name would ask for a key the dialog never offers. Both OpenAI keys are scrubbed before the mapped one is written back, so a key left in the env or inherited from the shell cannot reach Together's ingress./v1/modelsreturns a bare array, not OpenAI's{"data": [...]}envelope, soOpenAiModelListResponsebecame an untagged enum accepting both shapes.moonshotai/Kimi-K2.6defeat the id-shape heuristic used for OpenAI. Discovery filters on the declaredtype == "chat"and prefers Together'sdisplay_namefor labels.Together requires an explicit model — it serves well over a thousand chat models and picks no default — so the picker offers no "Default model" entry, same as Anthropic and OpenAI.
Three
check-file-sizes.mjsceilings are ratcheted (agent_models.rs+50,readiness.rs+74,runtime.rs+23), each with a comment explaining the growth. The preset logic itself lives in the new file; what lands in the existing ones is the spawn call site, the credential arms, and the discovery branch, which cannot move.Related issue
None found. The closest existing work is #1975 (first-class OpenRouter support), which is deliberately a much heavier change — OpenRouter needs a real
Providerenum variant for its body mutations,max_tokensspelling, and attribution headers. Together needs none of that, so this stays a preset. The two PRs touch overlapping files (agentConfigOptions.tsx,readiness.rs,agent_models.rs,check-file-sizes.mjs) and will likely need a trivial merge resolution, whichever lands second.Testing
buzz-agentbinary over ACP stdio using exactly the envapply_together_envderives —initialize→session/new→session/promptcompleted withstopReason: end_turnonmoonshotai/Kimi-K2.6. Separately confirmed tool calling works over that transport (finish_reason: tool_calls).chat, returned as a bare array — the two behaviors the parsing and filtering changes exist for.TOGETHER_API_KEYand not the mapped OpenAI key, bare-array parsing, and chat-only filtering.cargo test(1791 passed),pnpm test(3633 passed),pnpm typecheck,pnpm check,cargo fmt --check,cargo clippyclean. Builds verified both with and without themesh-llmfeature.Together is scoped to
buzz-agent. I originally left the Goose side as an open question, then checked: Goose ships no Together provider (it exists only in an unmerged fork PR, and there under the idtogetherai), whilebuzz-personapipes the provider id straight intoGOOSE_PROVIDER. Offering it for Goose would therefore persistGOOSE_PROVIDER=togetherand fail at startup, so the second commit hides it for provider-selecting runtimes other thanbuzz-agent— the same treatmentAgentConfigFieldsalready givesrelay-mesh. An agent already saved with Together still renders its value via the(current)tail. The credential requirement stays keyed on the provider for both runtimes, keeping the Rust/TS mirror intact.Made with Cursor