Skip to content

feat: add Avian as LLM provider - #494

Closed
avianion wants to merge 6770 commits into
CodebuffAI:mainfrom
avianion:feat/add-avian-provider
Closed

feat: add Avian as LLM provider#494
avianion wants to merge 6770 commits into
CodebuffAI:mainfrom
avianion:feat/add-avian-provider

Conversation

@avianion

@avianion avianion commented Apr 4, 2026

Copy link
Copy Markdown

Summary

Register Avian model IDs and provider metadata so the codebase recognizes avian/* as valid model names.

Changes

  • Add avian/deepseek-v3.2, avian/kimi-k2.5, avian/glm-5, avian/minimax-m2.5 to the ModelName type (both copies)
  • Add avian to ALLOWED_MODEL_PREFIXES
  • Add avianModels constant, spread into models and providerModelNames
  • Add avian entry to providerDomains and getLogoForModel

What this does NOT include

This PR only adds type/constant registration. The actual provider module (avian.ts), env var (AVIAN_API_KEY), and chat completions routing are not included and would need to be added in a follow-up PR for these models to actually work end-to-end.

Models

Model Notes
avian/deepseek-v3.2 DeepSeek V3 (0324)
avian/kimi-k2.5 Moonshot Kimi K2.5
avian/glm-5 Zhipu GLM-5
avian/minimax-m2.5 MiniMax M2.5

ghost commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

One thing I'd double-check before merging: web/src/llm-api/avian.ts currently bills as soon as any streamed chunk contains usage. In this repo, the newer provider implementations in web/src/llm-api/canopywave.ts and web/src/llm-api/siliconflow.ts added a billedAlready + isFinalChunk(...) guard because some providers emit cumulative usage on non-final chunks or can repeat a terminal usage frame.

Without that guard here, Avian could insert duplicate BigQuery rows and consume credits more than once for the same response. I'd be inclined to copy the CanopyWave/SiliconFlow pattern and strip usage from non-final chunks before forwarding them to the SDK.

@avianion

ghost commented Apr 9, 2026

Copy link
Copy Markdown
Author

Addressed feedback: added billedAlready + isFinalChunk() guard to prevent duplicate BigQuery rows and credit charges. Follows the same pattern used in canopywave.ts and siliconflow.ts — usage is now stripped from non-final chunks and billing only triggers once on the final chunk with finish_reason.

github-actions[bot] and others added 25 commits May 4, 2026 01:52
The pre-init module legitimately needs to set process.env so the embedded
wasm path reaches every copy of init-node.ts (the SDK's pre-built bundle
inlines its own copy).
- cli/src/pre-init/tree-sitter-wasm.ts: silence TS error for the bun-only
  `with { type: 'file' }` import (TS resolves the .wasm via the package's
  exports map and has no loader for binary assets).
- cli/src/__tests__/integration-tmux.test.ts: explicitly clear
  FREEBUFF_MODE from the tmux global env before running. A prior freebuff
  build or `bun run dev:freebuff` in the same tmux server leaves it set,
  which made the help-output test see the freebuff CLI variant (no
  `--agent` flag) instead of codebuff.
- web/jest.config.cjs: fix react/react-dom moduleNameMapper paths — they
  pointed at `web/node_modules/react` but bun hoists react to the
  workspace root.
- web/jest.setup.js: polyfill TextEncoder/TextDecoder, ReadableStream,
  Request/Response/Headers/fetch from Node + undici. JSDOM lacks these
  globals, and undici (loaded transitively via `next/server`) needs them
  at module-load time.
Mirrors the existing input on freebuff-release.yml so prod releases
can be built from a specific commit while still bumping version on
latest main. Used to roll back to a known-good commit when main is
broken.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix (d79d758) embedded the wasm path on process.env and
let init-node.ts resolve it via fs.existsSync. That worked on Linux/mac
but kept crashing on Windows: bun --compile reports the embedded asset
as `B:\~BUN\root\tree-sitter.wasm`, and fs.existsSync returns false for
that path even though fs.readFileSync succeeds. resolveTreeSitterWasm
fell through every branch and threw "Internal error: tree-sitter.wasm
not found".

Read the bytes once in pre-init via fs.readFileSync (which works on
Windows bunfs) and stash them on globalThis. init-node.ts now passes
them straight to Parser.init({ wasmBinary }), bypassing locateFile and
filesystem path resolution entirely.

globalThis is the cross-bundle channel: the SDK pre-built bundle inlines
its own copy of init-node.ts, so a module-level variable in this package
isn't visible to the singleton initialized via the SDK. The path-based
fallback is preserved for external SDK consumers that don't pre-load.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both --version smoke tests passed on Windows even though the binary
crashed for users: commander exits the process synchronously, before the
Parser.init promise has a chance to reject. Three changes to close the
gap:

- cli/scripts/smoke-binary.ts: portable script that spawns the binary,
  lets it run for 5s, kills it, and asserts the captured stdout/stderr
  doesn't contain earlyFatalHandler markers ("Fatal error during
  startup", "Internal error: tree-sitter.wasm not found", unhandled
  rejections, missing modules). Wired into the release-build smoke step
  for every platform and into the freebuff-e2e build smoke step.

- freebuff/e2e/tests/startup.e2e.test.ts: wait for "Pick a model to
  start" to render instead of just non-empty output. The model selector
  only appears once the binary survived module init (Parser.init
  included), the auth/session API call returned, and the React tree
  mounted, so a half-rendered crash splash no longer satisfies the
  assertion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous smoke test and e2e test both checked for known error markers
("Fatal error during startup", etc.). That misses anything we didn't
think to add — novel error messages, silent crashes, hangs, segfaults
that produce no stderr.

Switch both to a positive signal: assert the binary actually rendered a
known boot screen. If something goes wrong we don't anticipate, the
boot text never appears and the test fails with a clear "binary never
reached a known boot screen" diagnostic. Negative pattern matches stay
for clearer error messages on regressions of bugs we've already seen.

- cli/scripts/smoke-binary.ts: gate pass/fail on at least one of N boot
  signals appearing in stdout/stderr (chat surface header, login modal,
  freebuff queue states, freebuff country-block screen, chat input
  prompt). Verified locally: passes on real binaries, fails on a stub
  that hangs without rendering.

- freebuff/e2e/tests/startup.e2e.test.ts: wait for the FREEBUFF ASCII
  logo's F+R crossbar pattern (`█████╗  ██████╔╝`). The logo renders
  for every valid boot state — including the country-block screen that
  GitHub Actions runners hit because their egress is flagged as
  anonymized network — so this assertion survives the geo gate that
  was tripping the previous "Pick a model to start" wait.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Freebuff 0.0.62 still crashed on Windows with the same "Internal error:
tree-sitter.wasm not found" — surfaced this time through the late
renderer-cleanup handler ("Unhandled rejection: error: ...") instead of
the early one, so it appeared *after* the logo had rendered. CI Windows
smoke passed because the rejection fires past the 5s kill timer (after
React mounts and the renderer is up), and even when it does fire, the
boot screen has already matched our positive signal.

Root cause: the previous fix's `fs.readFileSync(treeSitterWasmPath)` of
the bunfs path silently fails on Windows for some user environments,
its catch block falls through, globalThis stays unset, and init-node
then hits the broken path-based fallback. CI Windows happened to pass
fs.readFileSync — user Windows didn't.

Bypass the filesystem entirely: bake the wasm bytes into the JS source
as a base64 string literal that bun --compile bundles into the binary's
text segment. No runtime fs read, no path normalization, no platform
quirks.

- cli/src/pre-init/tree-sitter-wasm-bytes.ts: committed stub with empty
  base64. Dev mode and unit tests see this and fall through to
  code-map's path-based resolution (which works locally because
  node_modules/web-tree-sitter/tree-sitter.wasm exists).
- cli/scripts/build-binary.ts: overwrites the stub with the real bytes
  before `bun build --compile`, restores it after. `process.on('exit',
  restore)` is a backstop so a crash mid-build doesn't leave a multi-MB
  diff in the working tree.
- cli/src/pre-init/tree-sitter-wasm.ts: drop the `with { type: 'file' }`
  + readFileSync path, decode the embedded base64 directly.
- cli/scripts/smoke-binary.ts: bump the run window from 5s to 10s and
  match the late-handler form ("Unhandled rejection:" / "Uncaught
  exception:") in addition to the early one. The 0.0.62 regression
  fired *after* the boot screen rendered, so a positive boot signal
  alone isn't enough — we need to keep watching for fatal markers
  through the full window.

Verified locally: full bun --compile build embeds 205KB of wasm as
274KB of base64, stub is restored after build (and after a simulated
mid-build crash via the exit handler), binary boots cleanly to the
chat surface with no wasm errors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The tree-sitter wasm regression that crashed freebuff 0.0.62 only
manifested on real Windows. CI was Linux-only, macOS dev machines
behaved fine, and the Windows binary was only built+smoked at release
time (cli-release-build.yml). So the bug shipped twice before being
caught by user reports.

Add a windows-latest job to freebuff-e2e.yml that builds the freebuff
binary natively on Windows and runs the long smoke test against it.
The full tmux-based e2e matrix can't follow — Windows runners don't
ship tmux, and porting tmuxStart/tmuxSend would be substantial — but
smoke-binary.ts catches the failure mode that bit us: it spawns the
binary, waits long enough for the late renderer-cleanup rejection
handler to fire, and asserts both that no fatal markers appeared and
that the boot screen actually rendered.

Mirrors the Windows-specific bits from cli-release-build.yml's
build-windows-binary job: explicit `bun install --cwd cli` and the
@OpenTui workspace symlink fix, both needed because bun workspace
linking doesn't work reliably on Windows runners.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Freebuff 0.0.64 still crashed for users with the same wasm error even
though it was built from a commit that contained the base64 embed. The
runtime stack trace pointed at the path-resolution fallback in
init-node.ts:76, meaning the embed didn't reach the SDK bundle's
globalThis check at runtime — the binary fell through to fs.existsSync
which never works on Windows bunfs paths.

Two hardening passes so this can't ship silently again:

- cli/src/pre-init/tree-sitter-wasm.ts: hidden `--smoke-tree-sitter`
  flag, handled in the very first import. Calls Parser.init({ wasmBinary
  }) directly with the embedded base64 and exits 0/1. Lives here (not
  commander) on purpose — it tests *the embed*, not the broader init
  path that has a path-resolution fallback that would mask a broken
  embed by passing in dev mode.
- cli/scripts/build-binary.ts: post-bun-compile, scan the output binary
  for the wasm's base64 prefix. Build fails if the bytes didn't actually
  make it through bundling (e.g. bun dropping a huge string literal,
  bundle cache reading a stale empty stub). Always-on log of which path
  the wasm was resolved from so CI logs make the embed step diagnosable.
  More resilient resolve: search workspace root, cli/node_modules, and
  sdk/node_modules before falling back to createRequire — Windows CI's
  `bun install --cwd cli` lays out web-tree-sitter differently than
  a hoisted root install.
- packages/code-map/src/init-node.ts: accept bunfs paths
  (`/~BUN/root/...`) without an fs.existsSync check. fs.existsSync
  inconsistently returns false for bun --compile asset paths on Windows
  even though the runtime can read them, so the existing path-resolution
  fallback was permanently broken on Windows. Belt-and-braces: this
  makes the fallback work even if the embed step regresses.
- cli/scripts/smoke-binary.ts: run --smoke-tree-sitter as a deterministic
  pre-check before the long-window boot smoke. A broken embed fails fast
  with a clear "exit code 1, no boot ok marker" error instead of a 10s
  timeout that depends on render-loop timing.

Verified locally: build embeds 205KB wasm as 274KB base64, post-build
verification finds the prefix in the compiled binary, --smoke-tree-sitter
exits 0 with "tree-sitter smoke ok", full smoke passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…le' }`

The base64-in-source approach didn't survive `bun --compile` on Windows.
The CI build's `verifyTreeSitterWasmEmbedded` step caught it:

    Embedded tree-sitter.wasm from D:\a\...\tree-sitter.wasm (205488 bytes
      → 273984 chars base64)
    [343ms]  minify  -16.58 MB
    Embedded tree-sitter wasm prefix not found in D:\a\...\codebuff.exe.

So the embed step wrote the bytes to disk and bun read them, but the
274KB string literal didn't end up in the compiled output — likely
tree-shaken or transformed by the minifier on Windows. The same code
worked on macOS and Linux locally and in CI.

Switch to Bun's documented asset-embed mechanism: import the wasm with
`with { type: 'file' }`. Bun handles this through the bundler's asset
pipeline rather than as a generic string literal, and the resulting
binary contains the wasm bytes verbatim at a bunfs path.

- cli/src/pre-init/tree-sitter-wasm.ts: import the wasm path, set the
  env var (for the locateFile fallback), and try a synchronous read so
  Parser.init can take the wasmBinary fast path. If the read throws
  (some Windows configurations have done this), log loudly so user
  reports include the diagnostic, then fall through to the locateFile
  flow — which init-node.ts now accepts bunfs paths through, even when
  fs.existsSync misreports them.
- The --smoke-tree-sitter handler is now a top-level `await` instead
  of a fire-and-forget IIFE. Without that, commander.parse() ran
  synchronously in main() and failed on the unknown flag before the
  smoke handler could exit cleanly.
- cli/scripts/build-binary.ts: drop the base64 stub-overwrite step
  entirely. New verifyTreeSitterWasmEmbedded reads a 64-byte chunk
  from the *middle* of the source wasm and asserts it appears in the
  compiled binary — that proves *this specific* tree-sitter.wasm
  shipped, not just any wasm (OpenTUI also embeds tree-sitter language
  wasms, so a magic-bytes-only scan would false-pass).
- Delete cli/src/pre-init/tree-sitter-wasm-bytes.ts: no longer used.

Verified locally: build embeds tree-sitter.wasm via the file-attribute
import, post-build verification finds the source bytes at offset
77319353 of the compiled binary, --smoke-tree-sitter exits 0 with
"tree-sitter smoke ok (wasmBinary, 205488 bytes)".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Last attempt put the handler at top-level in the pre-init module behind
a top-level await, on the theory that ESM would pause subsequent module
evaluation until it resolved. That worked on macOS locally but not on
Windows in CI:

    smoke-binary: spawning ./codebuff.exe for 10s…
    error: tree-sitter smoke failed with exit code 1
    error: unknown option '--smoke-tree-sitter'

So commander.parse() ran before our handler exited, which means
top-level await is not actually blocking parent-module evaluation in
the bun --compile output on Windows (or it's getting transformed away
by `--production` minification).

Move the handler to the top of main() in cli/src/index.tsx, before
parseArgs(). At that point commander hasn't run yet, so we can short-
circuit cleanly. The pre-init module's only job is now to publish the
embedded wasm bytes (globalThis) and path (env var); the handler reads
those out of the same channels the production runtime uses.

Verified locally: ./codebuff --smoke-tree-sitter prints
"tree-sitter smoke ok (wasmBinary, 205488 bytes)" and exits 0; full
smoke-binary.ts run passes both the tree-sitter pre-check and the
boot-screen window.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
James Grugett and others added 18 commits May 23, 2026 23:13
Co-authored-by: James Grugett <jahooma@gmail.com>
Co-authored-by: James Grugett <jahooma@gmail.com>
Co-authored-by: James Grugett <jahooma@gmail.com>
Source: CodebuffAI/freebuff-private@c445ed638e2b0d1afcbe6c24c86c3e6db6475fc4
Source: CodebuffAI/freebuff-private@a27bb0ff5740a141c367de03fcfaa5ca44b9b1bb
Source: CodebuffAI/freebuff-private@cff529bfc31649d67277623edbc2df786d26601e
Source: CodebuffAI/freebuff-private@dea277573f29207d1a5e80e74a07ab0f69ea8ce9
Source: CodebuffAI/freebuff-private@509220440020545ed3b7a6aac22ac02a0d89a2fb
Source: CodebuffAI/freebuff-private@f27993427cd3903de82e7ae6dbc51e318b2e83ae
@codebuff-team

ghost commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution, but this falls under "adding a vendor," which is a product-direction decision the maintainers make, not something to introduce unilaterally via a PR — regardless of how clean the wiring is.

Separately, the diff itself doesn't match the PR description. The body claims a new web/src/llm-api/avian.ts provider module with streaming/non-streaming support, TTFT tracking, error handling, an AVIAN_API_KEY env var, and routing of avian/* chat completions — none of that appears in the actual diff. All that's here is the type/constant registration (ModelName, avianModels, providerModelNames, providerDomains). Without the actual provider implementation, these three files alone would break at runtime: avian/* models would be recognized as valid ModelNames and show up in provider metadata, but there's no code path to actually call the Avian API, so requests would fail or fall through incorrectly.

Also worth flagging: the diff sneaks in ...openCodeZenModels into the models spread in model-config.ts, which is unrelated to Avian and not mentioned anywhere in the PR description — this suggests the diff was generated from a messier branch state than what's described.

Finally, several of the listed models (kimi-k2.5, glm-5, minimax-m2.5) don't correspond to any models I'm aware of being publicly available — please double check these are real, shippable model IDs and not placeholders.

If you want to pursue this, it would need: the actual provider module, env schema entry, and chat-completions routing all in the same PR, and it would still need sign-off on adding a new vendor before it's actionable.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:out-of-scope Touches paths the public mirror does not accept labels Aug 19, 2026
@avianion

ghost commented Aug 22, 2026

Copy link
Copy Markdown
Author

Thanks for the detailed review. Addressed the concrete issues:

  1. Removed ...openCodeZenModels spread — that was an unrelated change that leaked in from the branch state. Removed in the latest commit.

  2. Updated PR description — now accurately reflects what the diff contains: type/constant registration only. The previous description was written for a larger changeset that didn't survive the merge conflicts. The provider module, env var, and routing would need a follow-up PR.

  3. Model IDs — these are real, publicly available models served via Avian's API:

    • kimi-k2.5 — Moonshot's Kimi K2.5
    • glm-5 — Zhipu's GLM-5
    • minimax-m2.5 — MiniMax M2.5

Understood that adding a new vendor is a product-direction decision. Happy to close this if the team decides against it.

@codebuff-team codebuff-team added the stale No activity after a maintainer request; queued for closing label Aug 22, 2026
@codebuff-team

ghost commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Marking this stale - there has been no activity here for 140 days. It will close in 7 days unless someone comments.

This is backlog upkeep, not a verdict on the pull request. A single comment keeps it open, and anything closed this way can be reopened.

@avianion

ghost commented Aug 26, 2026

Copy link
Copy Markdown
Author

Still active — happy to expand this into the full provider implementation (module, env schema, routing) if the team decides to move forward with adding Avian as a vendor.

@avianion

ghost commented Aug 30, 2026

Copy link
Copy Markdown
Author

Checking in — the code-level feedback has been addressed (removed the unrelated openCodeZenModels spread, corrected description, confirmed model IDs). Happy to make any further changes needed. Would love to get maintainer input on the vendor-approval question so we can move forward.

@codebuff-team codebuff-team removed the stale No activity after a maintainer request; queued for closing label Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:out-of-scope Touches paths the public mirror does not accept

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants