From d07884c039d7c9ec9d8d98876ced4c72a924ac2d Mon Sep 17 00:00:00 2001 From: michalharakal Date: Fri, 16 Oct 2026 07:57:05 +0200 Subject: [PATCH] docs(skeep): SKEEP-006 status Implemented, link the input-frontends page from fromTokenizerJson --- .../skeep/pages/006-input-frontends-and-model-identity.adoc | 3 ++- docs/modules/skeep/pages/index.adoc | 2 +- .../kotlin/sk/ainet/frontend/tokenizer/TextFrontEnds.kt | 3 ++- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc b/docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc index 667c4f57..ce4bf34c 100644 --- a/docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc +++ b/docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc @@ -1,7 +1,8 @@ = SKEEP-006: Input front ends and model identity :description: SKaiNET proposal for one resolvable input front-end contract (tokenizers first, audio and image features next) keyed on a string-based model identity with an immutable family registry, so that every model type SKaiNET runs gets its preprocessing from the engine instead of re-deriving it downstream. -Status: Draft + +Status: Implemented + +Implementation: https://github.com/SKaiNET-developers/SKaiNET/pull/1332[#1332] (text lane, identity, resolver; engine 0.58.0), https://github.com/SKaiNET-developers/SKaiNET-transformers/pull/485[SKaiNET-transformers #485] (consumer, 0.58.0); the io-core compatibility aliases moved to `DeprecationLevel.ERROR` in 0.60.0 (https://github.com/SKaiNET-developers/SKaiNET/pull/1350[#1350]) and leave in 0.61. The audio (F4, F6) and image (F5) front ends are follow-up proposals. + Audience: SKaiNET maintainers and contributors; SKaiNET-transformers maintainers (the first consumer, and the repository whose parallel tokenizer stack this proposal retires); SKaiNET-cartridge maintainers (the facade binds front ends to cartridge ports) + Created: 2026-10-11 + Tracking issue: https://github.com/SKaiNET-developers/SKaiNET/issues/1331[#1331] (implementation lanes: https://github.com/SKaiNET-developers/SKaiNET/issues/1321[#1321], https://github.com/SKaiNET-developers/SKaiNET-transformers/issues/471[SKaiNET-transformers #471], https://github.com/SKaiNET-developers/SKaiNET-transformers/issues/330[SKaiNET-transformers #330]) + diff --git a/docs/modules/skeep/pages/index.adoc b/docs/modules/skeep/pages/index.adoc index fdadc3e7..80787fde 100644 --- a/docs/modules/skeep/pages/index.adoc +++ b/docs/modules/skeep/pages/index.adoc @@ -132,6 +132,6 @@ Every proposal should include: | Schedules — structured concurrency for the compute layer (algorithm/schedule split) | xref:skeep:006-input-frontends-and-model-identity.adoc[SKEEP-006] -| Draft +| Implemented | Input front ends and model identity — one resolvable preprocessing contract (tokenizers first) keyed on a string-based family registry |=== diff --git a/skainet-frontend/skainet-frontend-text/src/commonMain/kotlin/sk/ainet/frontend/tokenizer/TextFrontEnds.kt b/skainet-frontend/skainet-frontend-text/src/commonMain/kotlin/sk/ainet/frontend/tokenizer/TextFrontEnds.kt index 751648fa..2f395594 100644 --- a/skainet-frontend/skainet-frontend-text/src/commonMain/kotlin/sk/ainet/frontend/tokenizer/TextFrontEnds.kt +++ b/skainet-frontend/skainet-frontend-text/src/commonMain/kotlin/sk/ainet/frontend/tokenizer/TextFrontEnds.kt @@ -127,7 +127,8 @@ public object TextFrontEnds : FrontEndProvider { * `tokenizer_config.json` (FE-R-03). Dispatch: `BPE` with a byte-level pre-tokenizer → byte-level * BPE; `BPE` with Metaspace / byte fallback → [MetaspaceBpeTokenizer]; `Unigram` → * [UnigramTokenizer]; `WordPiece` → [WordPieceTokenizer]. Added tokens are matched atomically - * for every algorithm (#1324). + * for every algorithm (#1324). The evidence → implementation table and the compatibility + * notes are on the *Input front ends* reference page (`docs/modules/ROOT/pages/reference/input-frontends.adoc`). */ @JvmStatic @JvmOverloads