docs(apertus): document chat-template format#93
Merged
Conversation
PR 2 of the Apertus rollout (APERTUS_ROLLOUT.md). Researches the
actual chat-template format Apertus models expect, before PR 3
implements ApertusChatTemplate against it. Sources fetched from
HuggingFace `swiss-ai/Apertus-8B-Instruct-2509` 2026-05-01:
chat_template.jinja (14601 bytes), tokenizer_config.json,
special_tokens_map.json.
Findings (full spec lives in docs/specs/apertus-chat-template.md):
1. Apertus is NOT chatml-compatible. The previous
ModelRegistry.kt:66 setting `chatTemplateFamily="chatml"` was
a fallback guess; this PR corrects it to `"apertus"` so the
forthcoming PR 3 resolver branch dispatches correctly.
2. Apertus has its own role tokens:
<|system_start|>...<|system_end|>
<|developer_start|>...<|developer_end|>
<|user_start|>...<|user_end|>
<|assistant_start|>...<|assistant_end|> (EOS = <|assistant_end|>)
<|inner_prefix|>...<|inner_suffix|> (deliberation block)
<|tools_prefix|>[...]<|tools_suffix|> (tool calls)
No overlap with chatml/llama3/gemma. Needs its own
ApertusChatTemplate.kt + ApertusToolCallingSupport.kt.
3. Tools render as TypeScript-style type declarations, not JSON
Schema. The Jinja `render_tools` macro emits:
// <description>
type <name> = (_: { param: type }) => any;
PR 3 either ports the recursive type-lowering Jinja macro
verbatim into Kotlin, or keeps a Jinja runtime (Pebble/jinjava)
and just substitutes the tools array. The doc flags Jinja-runtime
as the lower-risk path given the macro's complexity (oneOf,
nullable, enum, array).
4. A developer block is emitted UNCONDITIONALLY after the system
message — it carries `Deliberation: enabled|disabled` and
`Tool Capabilities:` (either rendered tools or the literal
string `disabled`). PR 3's chat template must emit this even
when no tools are present.
5. When the caller doesn't supply a system message, the template
emits a default:
"You are Apertus, a helpful assistant created by the SwissAI
initiative.\nKnowledge cutoff: 2024-04\nCurrent date: <YYYY-MM-DD>"
The model is trained on this; PR 3 must mirror it.
6. Tool-call output format from the model:
<|tools_prefix|>[{"<tool>": <args_json>}, ...]<|tools_suffix|>
Single-key JSON objects in a JSON array; multiple parallel calls
stack with comma separation. Parser strategy spelled out in the
doc (PR 3 implements ApertusToolCallParserStrategy against this).
Changes in this PR:
- docs/specs/apertus-chat-template.md (NEW): the spec, with
comparison table vs chatml/llama3/gemma2, role-by-role token
table, tool format details, parser strategy outline, four
canonical parity test cases for PR 3 to assert byte-for-byte.
- llm-core/.../ModelRegistry.kt:66:
APERTUS("apertus", "Apertus", false, "chatml")
→ APERTUS("apertus", "Apertus", false, "apertus")
`supportsToolCalling=false` stays — PR 3 flips it once
ApertusChatTemplate + ApertusToolCallingSupport land. Updating
chatTemplateFamily early is harmless: ToolCallingSupportResolver
has no "apertus" branch yet, so the fallback path
(GenericToolCallingSupport) is unchanged.
- APERTUS_ROLLOUT.md: PR 2 checkbox ticked; status header advanced.
Verification:
- :llm-core:jvmTest green (no behavior change).
- Doc renders cleanly; ready for PR 3 to consume as the
specification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 2 of the Apertus rollout (see APERTUS_ROLLOUT.md). Researches the actual chat-template format Apertus models expect, before PR 3 implements
ApertusChatTemplateagainst it. Sources fetched from HuggingFaceswiss-ai/Apertus-8B-Instruct-25092026-05-01.Headline finding
Apertus is NOT chatml-compatible. The previous
ModelRegistry.kt:66settingchatTemplateFamily="chatml"was a fallback guess; this PR corrects it to"apertus".Apertus has its own role tokens (
<|system_start|>,<|developer_start|>,<|user_start|>,<|assistant_start|>,<|inner_prefix|>,<|tools_prefix|>— no overlap with chatml / llama3 / gemma), renders tools as TypeScript-style type declarations (not JSON Schema), emits a default system message + a developer block unconditionally, and uses a JSON-array tool-call format wrapped in<|tools_prefix|>...<|tools_suffix|>.Full spec at
docs/specs/apertus-chat-template.md(mirrors the existing markdown convention fromdocs/specs/spring-ai-adapter.md):ApertusToolCallParserStrategy(PR 3)ApertusChatTemplateHfParityTest(PR 3)Changes
docs/specs/apertus-chat-template.md(new) — the spec.llm-core/.../ModelRegistry.kt:66—chatTemplateFamily"chatml"→"apertus".supportsToolCalling=falsestays; PR 3 flips it once the chat template + parser ship.APERTUS_ROLLOUT.md— PR 2 checkbox ticked, status header advanced.Test plan
:llm-core:jvmTestgreen (chatTemplateFamily change has no behavior impact today —ToolCallingSupportResolverhas no"apertus"branch yet, so the fallback path is unchanged).Out of scope
ApertusChatTemplate.kt,ApertusToolCallingSupport.kt, registration inToolCallingSupportResolver.kt, flippingsupportsToolCalling=true— all PR 3.swiss-ai/Apertus-8B-Instruct-2509source files (chat_template.jinja,tokenizer_config.json,special_tokens_map.json), the canonical authority for the format.🤖 Generated with Claude Code