Skip to content

feat: add orcarouter llm2 extension#2206

Open
jinhaosong-source wants to merge 1 commit into
TEN-framework:mainfrom
jinhaosong-source:feat/orcarouter-llm2
Open

feat: add orcarouter llm2 extension#2206
jinhaosong-source wants to merge 1 commit into
TEN-framework:mainfrom
jinhaosong-source:feat/orcarouter-llm2

Conversation

@jinhaosong-source

@jinhaosong-source jinhaosong-source commented Jul 8, 2026

Copy link
Copy Markdown

What this PR does

Adds an OrcaRouter LLM2 provider extension: orcarouter_llm2_python.

OrcaRouter is an OpenAI-compatible meta-router — a single endpoint over many upstream models that also layers gateway-level zero-trust security for AI agents (scoped keys, guardrails, tool allow-lists, audit trail) and a routing DSL for per-request upstream selection. Because it speaks the OpenAI Chat Completions protocol, the extension mirrors gemini_llm2_python / openai_llm2_python: it reuses the OpenAI SDK and only changes the defaults (base_urlhttps://api.orcarouter.ai/v1, modelorcarouter/auto).

Disclosure: I'm an engineer on the OrcaRouter team.

Included

  • ai_agents/agents/ten_packages/extension/orcarouter_llm2_python/ — the extension (manifest.json, property.json, addon.py, extension.py, orcarouter.py, think_parser.py, __init__.py, pyproject.toml, requirements.txt, README.md).
  • tests/ — pure-logic unit tests for the streaming <think> reasoning parser (run anywhere) and config-default tests (skipped where ten_ai_base is unavailable), plus the standard conftest.py harness and config fixtures.
  • Registered in the voice-assistant example dependency list (additive — the default graph is unchanged).
  • ORCAROUTER_* documented in ai_agents/.env.example.

Features

  • Streaming, tool/function calling, and reasoning (<think> blocks and reasoning_content) work unchanged via the OpenAI-compatible protocol.
  • model="orcarouter/auto" lets OrcaRouter pick a live model per request (strategy configured in the OrcaRouter console).
  • Model-fallback chain via request parameters: {"models": ["openai/gpt-4o", "anthropic/claude-haiku-4.5"], "route": "fallback"}.
  • custom_headers forwards HTTP-Referer / X-Title for console attribution.

How I verified

Beyond the unit tests, I ran the extension's exact client path (OpenAI SDK with base_url + default_headers) against the live OrcaRouter API:

  • Streamingmodel="openai/gpt-4o" streamed content deltas correctly.
  • Tool calling — a get_weather tool was invoked as expected.
  • Fallbackextra_body={"models": [...], "route": "fallback"} resolved to a live model and returned a completion.
  • Reasoning params — confirmed OrcaRouter normalizes provider-specific parameter constraints server-side (sampling params like temperature are accepted even when a request is routed to a reasoning model), so the extension sends the standard OpenAI parameters unconditionally.

Notes

  • No changes to framework internals; this is a self-contained extension plus additive example/env wiring.
  • Conventional-commit title; commits authored under my GitHub account for the CLA.

OrcaRouter is an OpenAI-compatible meta-router, so this extension reuses
the OpenAI SDK and only changes defaults: base_url is
https://api.orcarouter.ai/v1 and model is orcarouter/auto. It supports
streaming, tool calling, reasoning output, and custom attribution
headers, and passes routing controls (models + route: fallback) through
request parameters.

Register it in the voice-assistant example dependency list and document
ORCAROUTER_* in .env.example.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant