feat(openai): adaptive parameter omission and per-request bearer source

CHANGELOG.md

@@ -19,6 +19,15 @@
   `reasoning_effort` is omitted, since OpenRouter takes one or the other).
   Other OpenAI-compatible endpoints still drop the budget, and an explicit
   `reasoning` provider option still wins.
+- OpenAI-compatible chat: adaptive parameter omission. A 400 whose message
+  names an optional field the request sent (`temperature`, `top_p`, `seed`,
+  `max_tokens`, `max_completion_tokens`, `reasoning_effort`) drops that field,
+  retries once, and remembers the omission for that endpoint and model. The
+  evidence rule and the process-wide store are public as
+  `providers::omission::{parameter_blamed_by, remember_omit, is_omitted}`.
+- `providers::BearerSource` and `OpenAiModel::with_bearer_source`: a credential
+  read per request (honouring `AuthStyle`, invalidated on 401) so a rotating
+  token no longer forces rebuilding the model.
 - Anthropic: a request carrying both a reasoning `effort` and `budget_tokens`
   now keeps adaptive thinking with that effort, byte-identical to the
   effort-only request. Fixed-budget thinking is used only when no effort is

crates/tinyinference-llm/src/providers/mod.rs

@@ -41,6 +41,7 @@ mod types;
 // OpenAI-compatible endpoint. Providers with a different wire protocol would be
 // added behind their own Cargo feature.
 pub mod anthropic;
+pub mod omission;
 pub mod openai;
 // #[cfg(feature = "ollama")]    pub mod ollama;
 

crates/tinyinference-llm/src/providers/omission.rs

@@ -0,0 +1,161 @@
+//! Learned request-parameter omission: what an endpoint has told us, by name,
+//! that a model does not accept.
+//!
+//! A provider that rejects a request parameter with an HTTP 400 almost always
+//! names it (`Unsupported parameter: 'max_tokens'`, `Unsupported value:
+//! 'temperature' does not support 0.2 with this model`). An adapter can then drop
+//! that one parameter, retry once, and [`remember_omit`] it so every later
+//! request to the same endpoint and model leaves it off without spending the
+//! round-trip again. [`OpenAiModel`](crate::providers::openai::OpenAiModel) does
+//! this on its Chat Completions path; the functions are public so a host that
+//! builds its own wire body can share the same evidence rule and the same store.
+//!
+//! This is what keeps static capability tables (such as
+//! `OpenAiModel::with_temperature_unsupported_models` or the o-series
+//! `max_completion_tokens` rename) an optimisation rather than a dependency: a
+//! vendor that changes a restriction silently between releases is corrected by
+//! one wasted round-trip (a 400 is billed nothing), not by a library release.
+//!
+//! Ported from OpenCompany's `company/inference/dialect.rs` learning layer.
+//!
+//! # Scope of what is learned
+//!
+//! * **Only parameters we sent.** [`parameter_blamed_by`] looks for the names
+//!   the caller says went out, never a vendor error format, so it cannot go
+//!   stale and a 400 about a field we did not send never drops one we did.
+//! * **Keyed on endpoint and model.** A model id is not unique across gateways;
+//!   one gateway refusing `max_tokens` for `gpt-4o` says nothing about another.
+//!   The endpoint key keeps the base path (a path-routed gateway fronts several
+//!   upstreams) but drops the operation suffix, so `/chat/completions` and
+//!   `/responses` on one base share what was learnt.
+//! * **Process-wide and in-memory.** It is a cache, not a record. Losing it on
+//!   restart costs one round-trip per model, and a vendor that lifts a
+//!   restriction is not remembered as broken forever.
+
+use std::collections::HashSet;
+use std::sync::{OnceLock, RwLock};
+
+/// Phrases that mark a provider error as a statement about the request's
+/// *shape*. Without one, a body that merely mentions a parameter name (say,
+/// model output echoed into an error) could talk us into dropping it.
+const REJECTION_MARKERS: &[&str] = &[
+    "unsupported",
+    "not supported",
+    "unsupported_value",
+    "unsupported_parameter",
+    "unknown parameter",
+    "unrecognized",
+    "not permitted",
+    "invalid_request_error",
+    "is deprecated",
+    "does not support",
+    "extra inputs",
+    "unexpected",
+];
+
+/// Operation suffixes stripped from an endpoint before it keys the store:
+/// they name what was asked of a service, not which service answered.
+const OPERATIONS: &[&str] = &[
+    "/chat/completions",
+    "/completions",
+    "/responses",
+    "/messages",
+    "/models",
+];
+
+/// One learned omission: `(endpoint scope, lowercased model id, parameter)`.
+type OmissionKey = (String, String, String);
+
+/// Which parameter, if any, a provider rejection blames — chosen only from the
+/// parameters the request actually sent.
+///
+/// `body` is the provider's error text (message, code, or raw body); `sent` is
+/// the wire names that went out. Returns `None` when the text carries no
+/// rejection marker (it is not about the request's shape) or names none of
+/// `sent`. When several sent names appear, the longest wins, so a name that
+/// contains another is not mistaken for it.
+///
+/// Matching is against the *wire* names on purpose: after a rename
+/// (`max_tokens` → `max_completion_tokens`) the field a vendor rejects is the
+/// one that went out, not the one the caller asked with.
+pub fn parameter_blamed_by<'a, S: AsRef<str>>(body: &str, sent: &'a [S]) -> Option<&'a str> {
+    let body = body.to_ascii_lowercase();
+    if !REJECTION_MARKERS.iter().any(|marker| body.contains(marker)) {
+        return None;
+    }
+    let mut candidates: Vec<&str> = sent.iter().map(AsRef::as_ref).collect();
+    candidates.sort_by_key(|name| std::cmp::Reverse(name.len()));
+    candidates
+        .into_iter()
+        .find(|name| !name.is_empty() && body.contains(&name.to_ascii_lowercase()))
+}
+
+/// Records that `model` at `endpoint` rejects `parameter`, so the next request
+/// there omits it without spending a round-trip.
+///
+/// Keyed on the endpoint as well as the model because that is who answered:
+/// keyed on the model alone, one gateway's 400 would silently drop the
+/// parameter for every gateway serving that id — and dropping an output cap is
+/// a bill, not just a difference. The model id is compared case-insensitively.
+pub fn remember_omit(endpoint: &str, model: &str, parameter: &str) {
+    tracing::debug!(
+        target: "tinyinference::omission",
+        model,
+        parameter,
+        "[providers][omission] remembering a rejected request parameter"
+    );
+    learned_store()
+        .write()
+        .unwrap_or_else(|poisoned| poisoned.into_inner())
+        .insert(key(endpoint, model, parameter));
+}
+
+/// Whether `model` at `endpoint` is known to reject `parameter` — i.e. whether
+/// [`remember_omit`] recorded it for the same service in this process.
+pub fn is_omitted(endpoint: &str, model: &str, parameter: &str) -> bool {
+    learned_store()
+        .read()
+        .unwrap_or_else(|poisoned| poisoned.into_inner())
+        .contains(&key(endpoint, model, parameter))
+}
+
+fn key(endpoint: &str, model: &str, parameter: &str) -> OmissionKey {
+    (
+        endpoint_scope(endpoint),
+        model.to_ascii_lowercase(),
+        parameter.to_string(),
+    )
+}
+
+/// The part of a URL that identifies the service: scheme, host, port and base
+/// path, lowercased, with query, fragment and any [`OPERATIONS`] suffix removed.
+///
+/// Anything that does not parse as `scheme://rest` is used whole: a key that is
+/// too specific costs one round-trip, one that is too broad is the bug the
+/// endpoint scoping exists to prevent.
+pub(super) fn endpoint_scope(endpoint: &str) -> String {
+    let lowered = endpoint.trim().to_ascii_lowercase();
+    let Some((scheme, rest)) = lowered.split_once("://") else {
+        return lowered;
+    };
+    let authority_and_path = rest.split(['?', '#']).next().unwrap_or(rest);
+    let trimmed = authority_and_path.trim_end_matches('/');
+    if trimmed.is_empty() {
+        return lowered;
+    }
+    let base = OPERATIONS
+        .iter()
+        .find_map(|operation| trimmed.strip_suffix(operation))
+        .unwrap_or(trimmed);
+    format!("{scheme}://{}", base.trim_end_matches('/'))
+}
+
+/// The single cell the reader and the writer share.
+fn learned_store() -> &'static RwLock<HashSet<OmissionKey>> {
+    static LEARNED: OnceLock<RwLock<HashSet<OmissionKey>>> = OnceLock::new();
+    LEARNED.get_or_init(|| RwLock::new(HashSet::new()))
+}
+
+#[cfg(test)]
+#[path = "omission_tests.rs"]
+mod tests;

crates/tinyinference-llm/src/providers/omission_tests.rs

@@ -0,0 +1,124 @@
+use super::*;
+
+/// One endpoint for the tests that do not exercise scoping. Every test uses
+/// its own model id, because the learned store is process-wide.
+const AT: &str = "https://api.example/v1";
+
+#[test]
+fn a_rejection_blames_only_a_parameter_we_actually_sent() {
+    let sent = ["temperature", "max_tokens"];
+    assert_eq!(
+        parameter_blamed_by(
+            "Unsupported value: 'temperature' does not support 0.2 with this model.",
+            &sent
+        ),
+        Some("temperature")
+    );
+    // A field we did not send is not ours to drop.
+    assert_eq!(
+        parameter_blamed_by("Unsupported parameter: 'logit_bias'", &sent),
+        None
+    );
+    // And a 400 that is not about the request's shape teaches us nothing.
+    assert_eq!(
+        parameter_blamed_by("the temperature in Paris is 19 degrees", &sent),
+        None
+    );
+}
+
+#[test]
+fn a_rejection_names_the_wire_name_that_actually_went_out() {
+    // After a rename the wire name is not the caller's name, so the blame is
+    // matched against what was sent.
+    let sent = ["temperature", "max_completion_tokens"];
+    assert_eq!(
+        parameter_blamed_by(
+            "400: Unsupported parameter: 'max_completion_tokens' is not supported",
+            &sent
+        ),
+        Some("max_completion_tokens")
+    );
+    // Nothing we sent is named, so there is nothing to drop and no retry.
+    assert_eq!(
+        parameter_blamed_by(
+            "400: Extra inputs are not permitted: 'reasoning_effort'",
+            &sent
+        ),
+        None
+    );
+}
+
+#[test]
+fn the_longest_matching_name_is_blamed() {
+    // `max_completion_tokens` does not contain `max_tokens`, but a future pair
+    // that nests would otherwise blame the shorter name.
+    let sent = ["top", "top_p"];
+    assert_eq!(
+        parameter_blamed_by("Unsupported parameter: 'top_p'", &sent),
+        Some("top_p")
+    );
+}
+
+#[test]
+fn nothing_is_omitted_until_a_rejection_is_remembered() {
+    let model = "omission-learning-test-model-v1";
+    assert!(!is_omitted(AT, model, "top_p"));
+    remember_omit(AT, model, "top_p");
+    assert!(is_omitted(AT, model, "top_p"));
+    assert!(
+        !is_omitted(AT, model, "temperature"),
+        "only the rejected parameter is dropped"
+    );
+}
+
+#[test]
+fn a_learned_omission_ignores_model_id_case() {
+    let model = "Omission-Case-Test-Model";
+    remember_omit(AT, model, "seed");
+    assert!(is_omitted(AT, &model.to_ascii_lowercase(), "seed"));
+}
+
+#[test]
+fn a_rejection_is_learned_about_the_endpoint_that_made_it() {
+    // A model id is not unique across endpoints: one gateway refusing a
+    // parameter says nothing about another serving the same id.
+    let model = "omission-endpoint-scope-test-model-v1";
+    let one = "https://gateway-one.example/v1";
+    let two = "https://gateway-two.example/v1";
+
+    remember_omit(one, model, "max_tokens");
+    assert!(is_omitted(one, model, "max_tokens"));
+    assert!(
+        !is_omitted(two, model, "max_tokens"),
+        "the other gateway never rejected anything"
+    );
+
+    // Same service, different operation: one endpoint answering two ways.
+    for operation in [
+        "https://gateway-one.example/v1/chat/completions",
+        "https://gateway-one.example/v1/responses",
+        "https://gateway-one.example/v1/",
+        "https://GATEWAY-ONE.example/v1?client_version=1",
+    ] {
+        assert!(
+            is_omitted(operation, model, "max_tokens"),
+            "{operation} is the same service"
+        );
+    }
+
+    // A path-routed gateway is several services behind one origin.
+    assert!(
+        !is_omitted(
+            "https://gateway-one.example/vendor-b/v1",
+            model,
+            "max_tokens"
+        ),
+        "another upstream behind the same host has not rejected anything"
+    );
+}
+
+#[test]
+fn an_unparseable_endpoint_is_used_whole() {
+    assert_eq!(endpoint_scope("  Not A URL  "), "not a url");
+    assert_eq!(endpoint_scope("https://"), "https://");
+}

crates/tinyinference-llm/src/providers/openai/README.md

@@ -44,6 +44,14 @@ of the rest of TinyInference.
   Anthropic API), `groq`, `xai`, `openrouter`, `together`, `mistral`, `ollama`.
   Override the preset's default model with `.with_model(..)`.
 
+**Rotating credentials.** `.with_bearer_source(Arc<dyn providers::BearerSource>)`
+replaces the constructor's static key with a value read on **every** request
+(a projected platform token, a session JWT), so a rotation never needs a
+rebuild and the connection pool and learned state survive it. The value is sent
+per the configured `AuthStyle` exactly as the static key would be, `with_header`
+headers are still attached, a source yielding `None` sends no credential header,
+and a 401 calls `BearerSource::invalidate` so the next request re-reads.
+
 Accessors: `.model()`, `.provider()`, `.base_url()`.
 
 ## Model discovery
@@ -111,6 +119,33 @@ call, and only for the shape the request actually used. This means unmodified
 `from_env()` / preset instances "just work" against LM Studio and friends
 without any per-provider configuration.
 
+## Adaptive parameter omission
+
+Static knowledge of which model takes which parameter
+(`with_temperature_unsupported_models`, the o-series/`gpt-5`
+`max_completion_tokens` rename) goes stale as vendors change models. When a
+Chat Completions call fails with an HTTP 400 whose error **message** names one
+of the optional fields that request actually sent — `temperature`, `top_p`,
+`seed`, `max_tokens`, `max_completion_tokens`, `reasoning_effort` — next to a
+rejection phrase (`unsupported`, `not supported`, `unknown parameter`, ...):
+
+1. that one field is dropped and the request is retried **once** (unary and
+   streaming paths alike; at most one retry per call, shared with the
+   request-shape degradation above, which takes precedence);
+2. the omission is remembered process-wide for that endpoint and model, so every
+   later request there leaves the field off up front — across `OpenAiModel`
+   instances.
+
+Messages, tools, `stop` and `response_format` are never dropped, a
+context-overflow 400 is never read as a rejection (those routinely name
+`max_tokens`), and a field injected by the host's `on_payload` hook is never
+stripped. The learning key is the endpoint's scheme, host, port and base path
+(operation suffixes such as `/chat/completions` removed) plus the model id, so
+two gateways serving the same id do not share what one of them rejected. The
+evidence rule and the store live in `providers::omission`
+(`parameter_blamed_by`, `remember_omit`, `is_omitted`) for hosts that build
+their own wire bodies.
+
 ## Inline reasoning-tag extraction
 
 Reasoning models served through OpenAI-compatible local runtimes (qwen3 and