Skip to content

Add CheaperInferenceLLM provider - #291

Open
aiapienthusiast wants to merge 1 commit into
ANative-Lab:mainfrom
aiapienthusiast:cheaperinference-provider
Open

aiapienthusiast wants to merge 1 commit into
ANative-Lab:mainfrom
aiapienthusiast:cheaperinference-provider

Conversation

@aiapienthusiast

Copy link
Copy Markdown

Adds CheaperInferenceLLM, a model class for Cheaper Inference.
It follows the same pattern as NovitaLLM (#268).

Cheaper Inference is an OpenAI-compatible LLM gateway.
One API key gives access to models from several labs.

The class subclasses OpenAILLM and only sets the client and the base URL.
Like NovitaLLM, it records token counts and sets the cost to 0, because LiteLLM has no prices for these models.

Usage:

from evoagentx.models import CheaperInferenceConfig, CheaperInferenceLLM

config = CheaperInferenceConfig(model="gpt-5.4-mini", cheaperinference_key="your-key")
llm = CheaperInferenceLLM(config=config)
Details

Files:

  • evoagentx/models/cheaperinference_model.py: new CheaperInferenceLLM (alias cheaperinference), base URL https://api.cheaperinference.com/v1.
  • evoagentx/models/model_configs.py: new CheaperInferenceConfig.
  • evoagentx/models/__init__.py: export.
  • docs/modules/llm.md, docs/zh/modules/llm.md: usage section.

Tests:

  • ruff check evoagentx: pass.
  • pytest tests/src/models: 36 passed.

Live check: gpt-5.4-mini returned a reply through CheaperInferenceLLM.generate() and async_generate().

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant