diff --git a/README.md b/README.md index 4c910b3..8083773 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,15 @@ using each provider's standard variable name: | Anthropic | `claude` | `anthropic/claude-sonnet-4-6` | `ANTHROPIC_API_KEY` | | Perplexity | `perplexity` | `perplexity/sonar-pro` | `PERPLEXITY_API_KEY` | | OpenAI | `openai` | `openai/gpt-4.1` | `OPENAI_API_KEY` | +| Groq | `groq` | `groq/llama-3.3-70b-versatile` | `GROQ_API_KEY` | +| DeepSeek | `deepseek` | `deepseek/deepseek-chat` | `DEEPSEEK_API_KEY` | +| Mistral | `mistral` | `mistral/mistral-large-latest` | `MISTRAL_API_KEY` | +| Together | `together` | `together/meta-llama/Llama-3.3-70B-Instruct-Turbo` | `TOGETHER_API_KEY` | + +Every first-class provider above is a **direct vendor key to a direct vendor +endpoint** — conclave never routes through an aggregator. Any other +OpenAI-compatible vendor (including aggregators/routers, which are deliberately +*not* first-class) remains usable config-only via an `endpoints:` entry. Set whichever you have: @@ -211,11 +220,12 @@ The suite mocks the httpx transport, so it needs no network and no API keys. ## Extending: custom OpenAI-compatible providers conclave's provider layer is an adapter registry over a single `httpx` transport -(`resolve_adapter` in `src/conclave/adapters/`). The five first-class providers are +(`resolve_adapter` in `src/conclave/adapters/`). The first-class providers are adapters; adding a *new* provider family is one registration. Adding any **OpenAI-compatible** endpoint (a local server, a gateway, another vendor's -`/chat/completions`) needs **no code** — just an `endpoints:` entry in your config that -names the base URL and the env-var that holds its key: +`/chat/completions`, or an aggregator/router you choose to use) needs **no code** — +just an `endpoints:` entry in your config that names the base URL and the env-var +that holds its key: ```yaml endpoints: diff --git a/docs/PRODUCT_DESIGN_DOCUMENT.md b/docs/PRODUCT_DESIGN_DOCUMENT.md index 3178fe7..665ce93 100644 --- a/docs/PRODUCT_DESIGN_DOCUMENT.md +++ b/docs/PRODUCT_DESIGN_DOCUMENT.md @@ -185,18 +185,26 @@ live in `registry.DEFAULT_MODELS` / `registry.PROVIDER_ENV_VARS` and are overrid | Anthropic | `claude` | `anthropic/claude-sonnet-4-6` | `ANTHROPIC_API_KEY` | BUILT | | Perplexity | `perplexity` | `perplexity/sonar-pro` | `PERPLEXITY_API_KEY` | BUILT | | OpenAI | `openai` | `openai/gpt-4.1` | `OPENAI_API_KEY` | BUILT | +| Groq | `groq` | `groq/llama-3.3-70b-versatile` | `GROQ_API_KEY` | BUILT | +| DeepSeek | `deepseek` | `deepseek/deepseek-chat` | `DEEPSEEK_API_KEY` | BUILT | +| Mistral | `mistral` | `mistral/mistral-large-latest` | `MISTRAL_API_KEY` | BUILT | +| Together | `together` | `together/meta-llama/Llama-3.3-70B-Instruct-Turbo` | `TOGETHER_API_KEY` | BUILT | | *(any provider known to an adapter)* | *raw id as name* | *passed through verbatim* | *adapter's provider env var* | SUPPORTED (untyped) | | *(any OpenAI-compatible endpoint)* | *config `endpoints:` entry* | *your model id* | *the endpoint's `api_key_env`* | SUPPORTED (config-only) | -**Default synthesizer:** `claude`. **Default council** (when none is configured): all five -known providers. Because `resolve_model_id()` passes unknown names through verbatim, a user +All nine first-class providers are **direct vendor key -> direct vendor endpoint** (no +aggregator/router), per §11. The Groq/DeepSeek/Mistral/Together additions (issue #5) are +OpenAI-compatible, each served by `OpenAICompatAdapter` with a verified URL and key env-var +(no native adapter required). Aggregators/routers (e.g. OpenRouter) are deliberately *not* +promoted — they stay config-only via `endpoints:`, keeping no-middleman intact. + +**Default synthesizer:** `claude`. **Default council:** all nine known providers when none is +configured. Because `resolve_model_id()` passes unknown names through verbatim, a user can already add a council member by raw id whose prefix an adapter recognizes (e.g. `openai/gpt-4o`) without a code change; it just won't have a static key-presence check (treated as "attempt and catch"). A wholly new OpenAI-compatible vendor needs no code at all — a `config.yml` `endpoints:` entry (base URL + `api_key_env`) makes it a first-class member -via `OpenAICompatAdapter` (see §6). - -Expanding the *first-class* provider list (more friendly-name defaults) is Roadmap, §9. +via `OpenAICompatAdapter` (see §6). Further first-class defaults remain Roadmap, §9 #3. --- @@ -241,7 +249,7 @@ CLI (cli.py, typer+rich) Library (from conclave import Council) | `transport.py` | The single async network boundary: `post_json` — one httpx call site for the whole highway. Nothing else in conclave touches the network. | | `adapters/__init__.py` | `resolve_adapter(model_id, config)` — the provider registry and **extension seam**. Maps a model-id prefix (or a config `endpoints:` entry) to the adapter that serves it. Adding a provider family = one registration here; adding an OpenAI-compatible endpoint = config-only. | | `adapters/base.py` | The `ProviderAdapter` protocol, `ProviderError`, and `redact()` — the secret-scrubber applied to every error string before it reaches `ModelAnswer.error`. | -| `adapters/openai_compat.py` | `OpenAICompatAdapter` — serves openai / xai / perplexity and any custom OpenAI-compatible endpoint. Per-provider full completions URL (note: Perplexity has no `/v1` segment). | +| `adapters/openai_compat.py` | `OpenAICompatAdapter` — serves openai / xai / perplexity / groq / deepseek / mistral / together and any custom OpenAI-compatible endpoint. Per-provider full completions URL (note: Perplexity has no `/v1` segment; Groq nests its OpenAI surface under `/openai/v1`). | | `adapters/anthropic.py` | `AnthropicAdapter` — native `POST /v1/messages` (`x-api-key` + `anthropic-version`); system prompt hoisted to the top-level `system` field; `max_tokens` required (default 4096); parses `content[].text` and `input_tokens`/`output_tokens`. | | `adapters/gemini.py` | `GeminiAdapter` — native `generateContent` (`x-goog-api-key`); OpenAI roles mapped (assistant→model), `systemInstruction` hoisted, `generationConfig.{temperature,maxOutputTokens}`; parses `usageMetadata`. | | `registry.py` | Single source of truth for friendly-name → model-id defaults and provider → env-var mapping. Key *presence* logic only — never key values. | @@ -263,10 +271,11 @@ CLI (cli.py, typer+rich) Library (from conclave import Council) depending on an LLM SDK. `resolve_adapter` (`adapters/__init__.py`) maps each model id to a `ProviderAdapter`; every adapter serializes its provider's request shape and hands it to the **one** network call site, `transport.post_json` (`transport.py`). Three adapters cover the -five first-class providers: `OpenAICompatAdapter` (openai/xai/perplexity, OpenAI-style -`/chat/completions`), `AnthropicAdapter` (native `/v1/messages`, system-prompt hoist, -required `max_tokens`), and `GeminiAdapter` (native `generateContent`, role mapping, -`usageMetadata`). Extension is deliberately cheap: a **new provider family** is one +nine first-class providers: `OpenAICompatAdapter` +(openai/xai/perplexity/groq/deepseek/mistral/together, OpenAI-style `/chat/completions`), +`AnthropicAdapter` (native `/v1/messages`, system-prompt hoist, required `max_tokens`), and +`GeminiAdapter` (native `generateContent`, role mapping, `usageMetadata`). Extension is +deliberately cheap: a **new provider family** is one registration in `adapters/__init__.py`; a **new OpenAI-compatible endpoint** (local server, gateway, another vendor) is **config-only** — a `~/.conclave/config.yml` `endpoints:` entry giving a base URL and the env-var *name* of its key, served by `OpenAICompatAdapter` with no @@ -284,7 +293,8 @@ LLM-SDK dependency.** Packaged with hatchling; console script `conclave = concla **Shipped in v0.1:** - `synthesize` and `raw` modes (fan-out, partial results, synthesizer merge). -- 5 first-class providers + pass-through for any model id an adapter recognizes. +- First-class providers (5 at v0.1; 9 since issue #5) + pass-through for any model id an + adapter recognizes. - BYO-keys via env-var name only; graceful skip of missing-key members. - Concurrent fan-out with per-call timeout and temperature. - Structured `CouncilResult` with latency, token usage, per-model error capture. @@ -307,9 +317,9 @@ LLM-SDK dependency.** Packaged with hatchling; console script `conclave = concla - **LiteLLM removed.** Replaced by conclave's own provider highway: an `httpx` async transport (`transport.py`) behind a per-provider adapter registry (`adapters/`). `httpx` is now the only network dependency; there is no LLM-SDK dependency (see §6). -- Three adapters cover the five first-class providers (`OpenAICompatAdapter` for - openai/xai/perplexity, native `AnthropicAdapter`, native `GeminiAdapter`); `resolve_adapter` - is the extension seam. +- Three adapters cover the first-class providers (`OpenAICompatAdapter` for + openai/xai/perplexity and the issue-#5 additions groq/deepseek/mistral/together, native + `AnthropicAdapter`, native `GeminiAdapter`); `resolve_adapter` is the extension seam. - **Custom OpenAI-compatible endpoints** via a config `endpoints:` section — config-only, no code change. - **Key-leak hardening landed:** provider/transport error strings are scrubbed by `redact()` @@ -354,10 +364,18 @@ Ordered roughly by strategic value to the origin use case and to mcp-warden. preserving fixed-rounds behavior exactly. Recorded on `CouncilResult.converged` + `convergence_score`; `converge_threshold` is part of the debate cache key so a converged run and a fixed run never collide. Kept in the list, struck through, for traceability. -3. **More first-class providers** — additional friendly-name defaults (e.g. more OpenAI, +3. ~~**More first-class providers** — additional friendly-name defaults (e.g. more OpenAI, Anthropic, Google, and open-weights endpoints). New OpenAI-compatible vendors are already config-only via `endpoints:`; this item is about promoting common ones to typed defaults - (and adding native adapters where a provider isn't OpenAI-compatible). + (and adding native adapters where a provider isn't OpenAI-compatible).~~ **LANDED** + (issue #5): promoted four direct-key, OpenAI-compatible vendors to typed first-class + defaults — `groq`, `deepseek`, `mistral`, `together` — each with a doc-verified completions + URL, key env-var, and default model id, registered in the single source of truth + (`registry.OPENAI_COMPAT_PROVIDERS`) so `PROVIDER_ENV_VARS`, the adapter URL table, the + import-time drift guard, `redact()` coverage, and `conclave providers` stay consistent. No + native adapter was needed (all four are OpenAI-compatible). Aggregators/routers were + intentionally excluded (config-only via `endpoints:`) to preserve no-middleman (§11). Further + defaults stay open under §12 #5. Struck through for traceability. 4. **Caching** — optional result cache keyed on (prompt, council, mode, model ids) to make repeated/eval runs cheap. Must remain off by default and never persist keys. 5. **Streaming** — stream member answers and/or the synthesis to the terminal/library. diff --git a/src/conclave/adapters/openai_compat.py b/src/conclave/adapters/openai_compat.py index 157dfe3..2d976c7 100644 --- a/src/conclave/adapters/openai_compat.py +++ b/src/conclave/adapters/openai_compat.py @@ -4,8 +4,9 @@ shape (``{model, messages, temperature}`` in, ``choices[0].message.content`` out) is served by a single :class:`OpenAICompatAdapter` instance, parameterized by its full completions URL and env var name(s). conclave ships instances for **openai**, -**xai**, and **perplexity**; the same class powers any user-supplied -OpenAI-compatible endpoint declared in config. +**xai**, **perplexity**, **groq**, **deepseek**, **mistral**, and **together** +(all direct vendor key -> direct vendor endpoint); the same class powers any +user-supplied OpenAI-compatible endpoint declared in config. Per-provider full URLs live in :data:`OPENAI_COMPAT_URLS` so the verified endpoints sit in one place. Env-var names are sourced from @@ -18,11 +19,18 @@ from .base import ProviderError, status_error # Verified per-provider full completions URLs. Note Perplexity has NO ``/v1`` -# segment; xAI and OpenAI do. These are the authoritative endpoints. +# segment while xAI/OpenAI do, and Groq nests its OpenAI surface under +# ``/openai/v1``. These mirror :data:`conclave.registry.OPENAI_COMPAT_PROVIDERS` +# (the source of truth) -- the import-time drift guard fails loudly if they +# desync. Every entry is a direct vendor endpoint (no aggregator/router). OPENAI_COMPAT_URLS: dict[str, str] = { "openai": "https://api.openai.com/v1/chat/completions", "xai": "https://api.x.ai/v1/chat/completions", "perplexity": "https://api.perplexity.ai/chat/completions", + "groq": "https://api.groq.com/openai/v1/chat/completions", + "deepseek": "https://api.deepseek.com/v1/chat/completions", + "mistral": "https://api.mistral.ai/v1/chat/completions", + "together": "https://api.together.xyz/v1/chat/completions", } diff --git a/src/conclave/registry.py b/src/conclave/registry.py index 0121d2b..f3dc663 100644 --- a/src/conclave/registry.py +++ b/src/conclave/registry.py @@ -66,12 +66,18 @@ class OpenAICompatProvider: # Friendly name -> default provider-prefixed model id. Overridable via ~/.conclave/config.yml. +# Every entry is a DIRECT vendor key to a DIRECT vendor endpoint (no aggregator/router) -- +# the no-middleman positioning in PRODUCT_DESIGN_DOCUMENT.md §11 is load-bearing. DEFAULT_MODELS: dict[str, str] = { "grok": "xai/grok-4.3", "gemini": "gemini/gemini-2.5-pro", "claude": "anthropic/claude-sonnet-4-6", "perplexity": "perplexity/sonar-pro", "openai": "openai/gpt-4.1", + "groq": "groq/llama-3.3-70b-versatile", + "deepseek": "deepseek/deepseek-chat", + "mistral": "mistral/mistral-large-latest", + "together": "together/meta-llama/Llama-3.3-70B-Instruct-Turbo", } # SINGLE SOURCE OF TRUTH for built-in OpenAI-compatible providers: prefix -> @@ -91,6 +97,27 @@ class OpenAICompatProvider: completions_url="https://api.perplexity.ai/chat/completions", env_vars=("PERPLEXITY_API_KEY",), ), + # Groq's OpenAI-compatible surface lives under an /openai/v1 path prefix + # (https://console.groq.com/docs/openai). Direct vendor key, direct endpoint. + "groq": OpenAICompatProvider( + completions_url="https://api.groq.com/openai/v1/chat/completions", + env_vars=("GROQ_API_KEY",), + ), + # DeepSeek's /v1 segment is OpenAI-SDK compatibility sugar (no version meaning) + # and is accepted (https://api-docs.deepseek.com/). + "deepseek": OpenAICompatProvider( + completions_url="https://api.deepseek.com/v1/chat/completions", + env_vars=("DEEPSEEK_API_KEY",), + ), + "mistral": OpenAICompatProvider( + completions_url="https://api.mistral.ai/v1/chat/completions", + env_vars=("MISTRAL_API_KEY",), + ), + # Together's canonical REST host is api.together.xyz (https://docs.together.ai). + "together": OpenAICompatProvider( + completions_url="https://api.together.xyz/v1/chat/completions", + env_vars=("TOGETHER_API_KEY",), + ), } # Native (non OpenAI-compatible) providers: prefix -> candidate env var names. diff --git a/tests/conftest.py b/tests/conftest.py index 904aa0e..96831af 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -94,5 +94,9 @@ def clear_keys(monkeypatch) -> None: "ANTHROPIC_API_KEY", "PERPLEXITY_API_KEY", "OPENAI_API_KEY", + "GROQ_API_KEY", + "DEEPSEEK_API_KEY", + "MISTRAL_API_KEY", + "TOGETHER_API_KEY", ): monkeypatch.delenv(var, raising=False) diff --git a/tests/test_cli.py b/tests/test_cli.py index ccf745c..0b14a0c 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -331,3 +331,47 @@ def test_providers_command_lists_keys_without_values(monkeypatch, patch_cli_conf assert "grok" in result.output assert "XAI_API_KEY" in result.output assert "super-secret-value" not in result.output + + +def test_providers_command_lists_new_first_class_providers(monkeypatch, tmp_path): + """`conclave providers` includes the issue-#5 direct-key providers + env vars. + + Uses the real default config (no patched cli.load_config) so the registry's + DEFAULT_MODELS drive the table; an empty CONCLAVE_CONFIG path means the + built-in defaults are what appear. + """ + monkeypatch.setenv("CONCLAVE_CONFIG", str(tmp_path / "missing.yml")) + # The env-var column only renders a NAME when a key is present (otherwise it + # shows '-'), so set each new provider's key. The values are obvious fakes and + # must never appear in the output (BYO-keys name-only posture). + secrets = { + "GROQ_API_KEY": "groq-secret-value", + "DEEPSEEK_API_KEY": "deepseek-secret-value", + "MISTRAL_API_KEY": "mistral-secret-value", + "TOGETHER_API_KEY": "together-secret-value", + } + for var, val in secrets.items(): + monkeypatch.setenv(var, val) + # Force a wide console so Rich does not ellipsize the (long) env-var / model + # columns; the module-level console fixes its width at import time, so swap in + # a wide one. Otherwise the assertions would test rendering width, not content. + from rich.console import Console + + monkeypatch.setattr(cli, "console", Console(width=200)) + from conclave.config import clear_config_cache + + clear_config_cache() + + result = runner.invoke(cli.app, ["providers"]) + assert result.exit_code == 0 + for name, env_var in ( + ("groq", "GROQ_API_KEY"), + ("deepseek", "DEEPSEEK_API_KEY"), + ("mistral", "MISTRAL_API_KEY"), + ("together", "TOGETHER_API_KEY"), + ): + assert name in result.output + assert env_var in result.output + # No secret VALUE ever appears (only the env-var NAME does). + for val in secrets.values(): + assert val not in result.output diff --git a/tests/test_providers.py b/tests/test_providers.py index d652da8..c35dd60 100644 --- a/tests/test_providers.py +++ b/tests/test_providers.py @@ -13,6 +13,8 @@ from __future__ import annotations +from pathlib import Path + import pytest import conclave.config as config_mod @@ -76,6 +78,133 @@ def test_resolve_adapter_unknown_prefix_raises(): resolve_adapter("mystery/model") +# --------------------------------------------------------------------------- # +# First-class direct-key OpenAI-compatible providers (issue #5) +# --------------------------------------------------------------------------- # + +# Each tuple: (friendly name, model id, completions URL, env var, bare model id +# the adapter must send to the vendor). All are direct vendor key -> direct +# vendor endpoint (no aggregator) per PRODUCT_DESIGN_DOCUMENT.md §11. Endpoints, +# env-var names, and default model ids were verified against live vendor docs. +NEW_PROVIDERS = [ + ( + "groq", + "groq/llama-3.3-70b-versatile", + "https://api.groq.com/openai/v1/chat/completions", + "GROQ_API_KEY", + "llama-3.3-70b-versatile", + ), + ( + "deepseek", + "deepseek/deepseek-chat", + "https://api.deepseek.com/v1/chat/completions", + "DEEPSEEK_API_KEY", + "deepseek-chat", + ), + ( + "mistral", + "mistral/mistral-large-latest", + "https://api.mistral.ai/v1/chat/completions", + "MISTRAL_API_KEY", + "mistral-large-latest", + ), + ( + "together", + "together/meta-llama/Llama-3.3-70B-Instruct-Turbo", + "https://api.together.xyz/v1/chat/completions", + "TOGETHER_API_KEY", + # Only the first '/' is the provider prefix; the rest is the vendor's id. + "meta-llama/Llama-3.3-70B-Instruct-Turbo", + ), +] + + +@pytest.mark.parametrize("name,model_id,url,env_var,bare", NEW_PROVIDERS) +def test_new_provider_resolves_to_openai_compat_adapter(name, model_id, url, env_var, bare): + """friendly name -> default model id -> OpenAI-compat adapter + URL + env var.""" + from conclave.config import load_config + from conclave.registry import DEFAULT_MODELS + + cfg = load_config(path=Path("/nonexistent/conclave.yml")) + assert cfg.resolve_model_id(name) == model_id + assert DEFAULT_MODELS[name] == model_id + + adapter = resolve_adapter(model_id) + assert isinstance(adapter, OpenAICompatAdapter) + assert adapter.completions_url == url + assert adapter.env_vars == (env_var,) + # The bare model id sent to the vendor strips only the provider prefix. + assert adapter._bare_model(model_id) == bare + + +@pytest.mark.parametrize("name,model_id,url,env_var,bare", NEW_PROVIDERS) +def test_new_provider_env_var_mapping(name, model_id, url, env_var, bare): + """The single-source derivation maps each new provider to its key env var.""" + from conclave.registry import PROVIDER_ENV_VARS, key_source, required_env_vars + + assert required_env_vars(model_id) == [env_var] + assert PROVIDER_ENV_VARS[name] == [env_var] + assert key_source(model_id) is None # no key set in a clean test env + + +@pytest.mark.parametrize("name,model_id,url,env_var,bare", NEW_PROVIDERS) +async def test_new_provider_call_model_success(monkeypatch, name, model_id, url, env_var, bare): + """End-to-end call through a new provider yields a usable ModelAnswer.""" + monkeypatch.setenv(env_var, "sk-newprovider-test") + monkeypatch.setenv("CONCLAVE_CONFIG", "/nonexistent/conclave.yml") + + captured = {} + + async def fake_post_json(post_url, headers, json_body, timeout): + captured["url"] = post_url + captured["headers"] = headers + captured["body"] = json_body + return 200, { + "choices": [{"message": {"content": f"hello from {name}"}}], + "usage": {"prompt_tokens": 3, "completion_tokens": 2, "total_tokens": 5}, + } + + monkeypatch.setattr("conclave.transport.post_json", fake_post_json) + + answer = await call_model(name, model_id, [{"role": "user", "content": "hi"}]) + assert answer.ok + assert answer.answer == f"hello from {name}" + assert answer.usage is not None + assert answer.usage.total_tokens == 5 + assert answer.error is None + assert captured["url"] == url + assert captured["headers"]["Authorization"] == "Bearer sk-newprovider-test" + assert captured["body"]["model"] == bare + + +@pytest.mark.parametrize("name,model_id,url,env_var,bare", NEW_PROVIDERS) +async def test_new_provider_missing_key_is_error(monkeypatch, name, model_id, url, env_var, bare): + """A new provider with no key set yields a clean, non-raising error naming the var.""" + monkeypatch.delenv(env_var, raising=False) + monkeypatch.setenv("CONCLAVE_CONFIG", "/nonexistent/conclave.yml") + + answer = await call_model(name, model_id, [{"role": "user", "content": "hi"}]) + assert not answer.ok + assert answer.answer is None + assert env_var in answer.error + + +@pytest.mark.parametrize("name,model_id,url,env_var,bare", NEW_PROVIDERS) +def test_redact_scrubs_new_provider_key_value(monkeypatch, name, model_id, url, env_var, bare): + """redact() masks each new provider's key value out of an error string. + + Uses an unprefixed, no-recognized-shape value so ONLY the single-source + name-based derivation (PROVIDER_ENV_VARS -> redact) can scrub it. Proves the + new env vars are covered automatically with no edit to base.redact. + """ + fake_key = f"{name}FAKEsecret_unprefixed_0123456789" + monkeypatch.setenv(env_var, fake_key) + + cleaned = redact(f"{name}: HTTP 401: invalid api key: {fake_key}") + assert fake_key not in cleaned + assert "[REDACTED]" in cleaned + + # --------------------------------------------------------------------------- # # call_model end-to-end with transport patched # --------------------------------------------------------------------------- #