Skip to content

docs(skeep): SKEEP-006 status Implemented, link the input-frontends page from fromTokenizerJson #1351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michalharakal merged 1 commit into from
Oct 16, 2026
Merged

docs(skeep): SKEEP-006 status Implemented, link the input-frontends page from fromTokenizerJson #1351

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
= SKEEP-006: Input front ends and model identity
:description: SKaiNET proposal for one resolvable input front-end contract (tokenizers first, audio and image features next) keyed on a string-based model identity with an immutable family registry, so that every model type SKaiNET runs gets its preprocessing from the engine instead of re-deriving it downstream.

Status: Draft +
Status: Implemented +
Implementation: https://github.com/SKaiNET-developers/SKaiNET/pull/1332[#1332] (text lane, identity, resolver; engine 0.58.0), https://github.com/SKaiNET-developers/SKaiNET-transformers/pull/485[SKaiNET-transformers #485] (consumer, 0.58.0); the io-core compatibility aliases moved to `DeprecationLevel.ERROR` in 0.60.0 (https://github.com/SKaiNET-developers/SKaiNET/pull/1350[#1350]) and leave in 0.61. The audio (F4, F6) and image (F5) front ends are follow-up proposals. +
Audience: SKaiNET maintainers and contributors; SKaiNET-transformers maintainers (the first consumer, and the repository whose parallel tokenizer stack this proposal retires); SKaiNET-cartridge maintainers (the facade binds front ends to cartridge ports) +
Created: 2026-10-11 +
Tracking issue: https://github.com/SKaiNET-developers/SKaiNET/issues/1331[#1331] (implementation lanes: https://github.com/SKaiNET-developers/SKaiNET/issues/1321[#1321], https://github.com/SKaiNET-developers/SKaiNET-transformers/issues/471[SKaiNET-transformers #471], https://github.com/SKaiNET-developers/SKaiNET-transformers/issues/330[SKaiNET-transformers #330]) +
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/skeep/pages/index.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,6 @@ Every proposal should include:
| Schedules — structured concurrency for the compute layer (algorithm/schedule split)

| xref:skeep:006-input-frontends-and-model-identity.adoc[SKEEP-006]
| Draft
| Implemented
| Input front ends and model identity — one resolvable preprocessing contract (tokenizers first) keyed on a string-based family registry
|===
3 changes: 2 additions & 1 deletion .../skainet-frontend-text/src/commonMain/kotlin/sk/ainet/frontend/tokenizer/TextFrontEnds.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,8 @@ public object TextFrontEnds : FrontEndProvider {
* `tokenizer_config.json` (FE-R-03). Dispatch: `BPE` with a byte-level pre-tokenizer → byte-level
* BPE; `BPE` with Metaspace / byte fallback → [MetaspaceBpeTokenizer]; `Unigram` →
* [UnigramTokenizer]; `WordPiece` → [WordPieceTokenizer]. Added tokens are matched atomically
* for every algorithm (#1324).
* for every algorithm (#1324). The evidence → implementation table and the compatibility
* notes are on the *Input front ends* reference page (`docs/modules/ROOT/pages/reference/input-frontends.adoc`).
*/
@JvmStatic
@JvmOverloads
Expand Down
Loading