feat(server): OpenAI protocol coverage -- sampling extras, n, echo/suffix, usage details, tokenize/detokenize/metrics

[gdevenyi]

Summary

Closes the most common gaps between FreeToken's OpenAI-compatible surface and vLLM / SGLang / llama.cpp's server. docs/openai_api.md has the four-way table of endpoints, request parameters and response fields (honoured / accepted / 400), and lists what is deliberately not implemented.

Sampling extras (/v1/chat/completions and /v1/completions): min_p, presence_penalty, frequency_penalty (over generated tokens), repetition_penalty (prompt + generated, HF semantics), logit_bias (token id → bias, clamped to [-100, 100]), min_tokens (no EOS / stop token before N generated tokens), stop_token_ids, include_stop_str_in_output, skip_special_tokens (per request; default off here because the reasoning and tool parsers read the tags). Out-of-range values answer 400 with the field named.

Implemented as logits processors (engine/sample.py): Sampler.prepare builds a per-batch LogitsPlan on the host from the requests' SamplingParams and token histories, only for the rows that asked; apply_logits_processors applies it to a float32 copy of the logits before the sampling kernel (repetition → presence → frequency → logit_bias → min_tokens mask → min_p, vLLM's order). A batch without them takes exactly the old path, and sampling already runs outside the CUDA graph, so no capture changes. Cost measured on an Ada card with a 3k-token history: 0.97 ms per step for a batch that uses them (0.32 ms on the device), 0.08 ms on the plain path.

n (1..16) on both routes: one generation per choice submitted together (the prefix cache serves the shared prompt), results gathered; streams interleaved into one SSE response with the choice index, one usage chunk (prompt counted once, completions summed) and one [DONE]; a client disconnect aborts every uid of the fan-out.

Completions: echo (the prompt leads the text, or the first chunk), suffix as a fill-in-the-middle prompt on models whose vocabulary has the FIM tokens (Qwen family; 400 otherwise).

Chat: continue_final_message (the final assistant message is continued, no generation prompt), request_id echoed as the response id, seed / user accepted, system_fingerprint: null in responses and chunks.

Usage: completion_tokens_details.reasoning_tokens — the detokenizer counts the tokens up to and including the reasoning end tag and reports it on the finished reply.

Routes: POST /tokenize and /detokenize (also under /v1/; the vLLM / SGLang shape, messages render through the chat template so count equals a generation's prompt_tokens), GET /metrics (Prometheus text of /v1/stats), GET /version.

Not in this PR, on purpose: constrained decoding (response_format json / json_schema, grammars: needs a grammar engine on the sampling path), prompt logprobs and echo + logprobs (prefill logits), a per-request seed (one batched sampling kernel), embeddings / rerank / score / audio.

Test plan

• tests/engine/test_logits_processors.py (pure torch on the CPU: each processor, the plan builder, the greedy path), tests/tokenizer/test_detokenize_extras.py (stop-string keep, per-request skip_special_tokens, reasoning token count), tests/server/test_openai_extras.py (n fan-out non-stream and stream, echo, suffix with and without FIM tokens, prompt-major choice order, the extras reaching SamplingParams, 400s, continue_final_message, request_id, usage details, tokenize / detokenize, metrics exposition)
• tests/server, tests/tokenizer, tests/scheduler, tests/engine: 790 passed on the CPU
• GPU micro-check of the processors and the sampler with the real vocabulary (248,320) on RTX 6000 Ada
• Served Qwen3.8-Flash-Next (RadixArk NVFP4, offload backend, TP=1) from this branch and ran a live probe of every field and route: 24/24 (penalties change a repetitive output, logit_bias steers the answer, min_tokens forces 2 → 35 tokens, stop_token_ids on . stops at 18 tokens, include_stop_str_in_output keeps the stop word, min_p / seed / user accepted, n=2 non-stream and stream, echo, suffix as FIM (return a + b), continue_final_message, request_id as the response id, reasoning_tokens 39 of 44 completion tokens with thinking on, tokenize / detokenize round trip and a rendered-messages count equal to the generation's prompt_tokens, /metrics, /version, five 400s), 8-question quality probe unchanged (6/8), decode unchanged (61.7 tok/s single-stream / 166 at 8 concurrent, the same as main on that card)

🤖 Generated with Claude Code

https://claude.ai/code/session_0173pf9k9fSVtwbm3f898HDt

[gdevenyi]

Re-tested against current main (fb7f732, i.e. after the #418 / #427 / #426 quantization refactor) on 2 x RTX 6000 Ada, TP=2 box.

Method. This PR's head merged onto main, then the full pytest tests suite. The run is CUDA-hidden (CUDA_VISIBLE_DEVICES="") on purpose: this box is serving a model on both GPUs, and with them visible 65-95 GPU tests fail on main itself with AcceleratorError: out of memory, with the count swinging ~10 between identical runs. Hiding CUDA makes the result deterministic, so a failure-set difference against main means something. Baseline: main = 1205 passed, 350 skipped, 0 failed.

Result: 1235 passed, 350 skipped, no new failures.

The +30 over main are this PR's own tests, and they ran (not skipped).

🤖 Generated with Claude Code

https://claude.ai/code/session_0173pf9k9fSVtwbm3f898HDt