Skip to content

Add Nebul as provider (4 curated chat models + sync) - #7871

Open
wynandhuizinga wants to merge 2 commits into
anomalyco:devfrom
NEBUL-AI:wh/feat/add-nebul-provider-v3
Open

wynandhuizinga wants to merge 2 commits into
anomalyco:devfrom
NEBUL-AI:wh/feat/add-nebul-provider-v3

Conversation

@wynandhuizinga

@wynandhuizinga wynandhuizinga commented Sep 23, 2026 •

Copy link
Copy Markdown

Add https://nebul.com as a provider with four curated chat models, plus an hourly sync module.

Supersedes #7400 (same provider, deliberately trimmed to the four flagship models below for a smaller review surface). Ref #6879.

Models added (all base_model + override-only against existing lab metadata — no new lab files needed):

Served ID Lab base Cost (in/out/cache-read, USD/MTok) Context
zai-org/GLM-5.3 zhipuai/glm-5.3 1.47 / 4.62 / 0.35 1,048,576
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 nvidia/nemotron-3-super-120b-a12b 0.32 / 0.69 / 0.08 1,000,000
Qwen/Qwen3.5-397B-A17B alibaba/qwen3.5-397b-a17b 0.63 / 3.78 / 0.15 262,144 (inherited)
mistralai/Mistral-Large-3-675B-Instruct-2512 mistral/mistral-large-2512 0.60 / 1.73 / 0.15 inherited

Curation policy

The sync treats the catalog as authoritative for these four entries' live cost/context, but it can never grow or shrink the shipped set:

  • skipCreates: true — only the four hand-authored models ship; any other in-scope chat model is reported via the skipped notice and left out.
  • deleteMissing: false — a curated model absent from the catalog (e.g. zai-org/GLM-5.3 has been intermittently absent) is retained and surfaced via missingNotice rather than silently deleted.
  • trackMissingModels: false — no missing-model issues for in-scope chat models this provider deliberately does not curate.

Sources

Data Verified against
Costs and per-model context Live GET https://api.inference.nebul.io/v1/model/info (input_cost_per_1m_tokens, output_cost_per_1m_tokens, cache_read_input_cost_per_1m_tokens, max_input_tokens), re-checked 2026-09-24
Nemotron 3 Super & Qwen3.5 397B effort values Live probes (POST /v1/chat/completions, 2026-09-23): reasoning_effort accepts none|minimal|low|medium|high|xhigh|max — all levels return 200; thinking is on by default, none disables it. The catalog's supports_reasoning = false for both IDs is a false negative
GLM-5.3 effort values low|high|max Live catalog reasoning_efforts + parity with the lab (providers/zai) and OpenRouter entries for the same base model
Wire fields Nebul docs: reasoning_effort param only — no on/off toggle, no reasoning budget; traces arrive in reasoning_content (hence interleaved)

Reasoning controls policy

  • Effort-only, per the probe/doc evidence above; no toggles, no budgets invented for this API.
  • Sync preserves authored, probe-verified options exactly (they outrank the advertised reasoning_efforts, which have been observed wrong on this host). If a reasoner advertises nothing and has nothing authored, sync fails closed with a missing-reasoning-controls error instead of inventing values.

Sync safety

  • All four files carry leading header comments and are re-verified by sync rather than blindly rewritten: sync keeps existing entries when the host's pricing/context is temporarily null and when a served alias stops resolving to lab metadata.
  • Whole-catalog faults fail closed in parseModels before any file is written or deleted: empty response, no entry matching the chat-model filter, or a catalog smaller than half the known-good in-scope size.
  • Out-of-scope entries (embeddings/rerankers by model_type, document-OCR by name, display_tags Guard Model / Content Safety / Private / Internal, server-side-superseded IDs) skip silently.

Checklist

  • Compliant logo.svg (currentColor, 0 0 24 24 viewBox)
  • Non-lab models use base_model; all four lab files already exist upstream
  • Provider files override-only (USD/MTok costs, reasoning options, limits)
  • bun validate passes
  • bun test packages/core/test/nebul.test.ts — 21/21 pass
  • bun packages/core/script/sync-models.ts nebul --dry-run — 4 unchanged, 0 created/updated/removed

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [critical] [violation] packages/core/src/sync/providers/nebul.ts:9 - Check: Sync fetch URL must match the authoritative catalog endpoint. Why: Code fetches https://api.inference.nebul.io/model/info but PR body, TOML headers, and provider.toml api all use https://api.inference.nebul.io/v1/model/info; the missing /v1 will fail the hourly sync. Action: Fix API_ENDPOINT to the /v1/model/info URL and verify with a dry-run.
  • [high] [possible mistake] providers/nebul/models/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16.toml:12 - Check: Relay effort sets must copy lab + same-surface peers; extra levels need host docs or live meaningful effect. Why: Lab is toggle-only and peers are narrower (openrouter toggle+low|medium+budget, requesty none|low|medium|high|max+budget, edenai none|low|medium, kenari low|medium); no peer shows minimal+xhigh for this model, and Qwen/Qwen3.5-397B-A17B.toml:12 similarly exceeds lab (toggle+budget), openrouter (toggle only), and ovhcloud/digitalocean (none|low|medium|high) on minimal/xhigh/max. A 200 OK acceptance probe alone does not prove graded effect or that none disables traces. Action: Provide docs excerpt listing supported reasoning_effort values and probe excerpts showing differing traces / none disabling, or narrow to the verified set.
  • [high] [violation] packages/core/src/sync/providers/nebul.ts:191 - Check: Sync must fail closed on unverified reasoning controls, not auto-create from advertised lists. Why: buildReasoningOptions falls back to advertised reasoning_efforts for new entries, but the PR proves the catalog unreliable (supports_reasoning=false false negatives, Mistral-Medium-3.5 advertised low|medium|high|max but only high works); with 4 of ~12 catalog models committed, hourly sync will auto-create ~8 unreviewed models with potentially wrong controls. Action: Throw MissingReasoningOptionsError for new reasoners without authored probe-verified options (or set skipCreates/missingModelID flow), or commit all in-scope models with verified options.

@wynandhuizinga

Copy link
Copy Markdown
Author

All three findings addressed in ddf28e82.

1. Endpoint (nebul.ts:9): live-verified 2026-09-23: both https://api.inference.nebul.io/model/info and https://api.inference.nebul.io/v1/model/info return 200 with the identical catalog (with and without auth), so the sync would not have failed — but the bare path was inconsistent with provider.toml and the PR body. Aligned to /v1/model/info.

2. Effort sets: narrowed both nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16.toml and Qwen/Qwen3.5-397B-A17B.toml to none|low|medium|high|max, matching the widest same-surface peer baseline (requesty). minimal/xhigh will return in a follow-up with per-level distinct-effect probe evidence.

3. Fail-open advertised-efforts fallback (nebul.ts:191): fixed. New reasoners without hand-authored reasoning_options now throw MissingReasoningOptionsError (runner warns + skips); the catalog's advertised list is never copied into a synced entry. Dry-run against the live catalog after the change: 0 created, 0 updated, 0 removed, 4 unchanged, with 9 reasoners skipped pending hand-authored probe-verified options (incl. mistralai/Mistral-Medium-3.5-128B, whose advertised low|medium|high|max is disproven by the recorded probes — only high works).

bun test (nebul suite 20/20, incl. a new fail-closed test) and validate both pass.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/nebul/models/Qwen/Qwen3.5-397B-A17B.toml:13 - Check: Reasoning-options baseline must be lab + same-surface peer set; extra levels beyond lab/peers need host docs or live meaningful effect. Why: Lab (providers/alibaba/models/qwen3.5-397b-a17b.toml) is toggle+budget_tokens and established relay peers (providers/ovhcloud, providers/scaleway, providers/crof, providers/digitalocean/models/qwen3.5-397b-a17b.toml) are none|low|medium|high — none include max. A 200 OK probe alone does not prove a distinct reasoning effect and contradicts the file's own narrowing rationale for minimal/xhigh. Action: Drop max to ["none", "low", "medium", "high"] to match peers, or keep it only with distinct-effect evidence (e.g., differing reasoning-token counts/outputs for max vs high plus doc reference) cited in the PR body/header.

@wynandhuizinga

Copy link
Copy Markdown
Author

Qwen3.5 finding addressed in cde5df80: dropped max — providers/nebul/models/Qwen/Qwen3.5-397B-A17B.toml is now ["none", "low", "medium", "high"], matching the established relay-peer baseline (ovhcloud/scaleway/crof/digitalocean all carry none|low|medium|high; the lab entry itself is toggle+budget_tokens). Header comment updated to cite the peer set and note that minimal/xhigh/max lacked distinct-effect evidence. (NVIDIA-Nemotron-3-Super-120B-A12B-BF16.toml keeps max deliberately: requesty's entry for the same model carries none|low|medium|high|max.) bun test (nebul suite 20/20) and validate pass.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 23, 2026
@wynandhuizinga
wynandhuizinga force-pushed the wh/feat/add-nebul-provider-v3 branch from cde5df8 to a583b83 Compare September 24, 2026 11:06
@github-actions github-actions Bot removed the reviewer: ready Automated review found no actionable items label Sep 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [violation] packages/core/src/sync/providers/nebul.ts:137 - Check: Sync parsing must not fail the whole run on an unused source field. Why: translateModel deliberately never copies reasoning_efforts because probes proved the catalog list unreliable, yet EffortValues strict enum makes NebulResponse.parse throw on any future unknown value, failing hourly sync and blocking cost/context refreshes for the curated models. Action: Loosen the field to accept unknown strings (e.g. z.array(z.string())) since it is never synced, and update the rejects unknown reasoning effort values test in packages/core/test/nebul.test.ts to assert unknown values are accepted/ignored.

Add the Nebul provider (NEBUL-AI) with exactly four curated chat models,
each as a base_model override:

- zai-org/GLM-5.3
- nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
- Qwen/Qwen3.5-397B-A17B
- mistralai/Mistral-Large-3-675B-Instruct-2512

The sync module reads the unauthenticated /v1/model/info catalog as
authoritative for the curated entries' cost/context, but is constrained so
it can never grow or shrink the shipped set: skipCreates keeps other
in-scope chat models out, deleteMissing: false retains a curated model that
drops out of the catalog (surfaced via missingNotice), and
trackMissingModels: false avoids missing-model issues for models this
provider deliberately does not curate.

Reasoning options are only ever hand-authored from live probes; the
catalog's advertised reasoning_efforts are probe-proven unreliable, and a
reasoner without authored options fail-closes. Whole-catalog faults are
rejected before any file is written or deleted.
@wynandhuizinga
wynandhuizinga force-pushed the wh/feat/add-nebul-provider-v3 branch from a583b83 to 93fb473 Compare September 24, 2026 11:47
@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 24, 2026
Rewrite the comments added with the Nebul provider for simple, active-voice
English: short sentences, no semicolons or em-dashes, no speculative modals.
No code or data changes.
@github-actions github-actions Bot removed the reviewer: ready Automated review found no actionable items label Sep 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant