Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

## [Unreleased]

## [0.60.0] - 2026-10-15

Headline: **key-padding lengths and window metadata for attention (#1343).**
`scaledDotProductAttention` takes `keyLengths`, `window` and `rightContext`; padding, the
sliding-window band and `causal` are rules over positions defined once in `AttentionMask` and
shared by the CPU kernel, the recorded graph and the StableHLO export. The recorded node carries
the rules as metadata (`maskKind`, `window`, `rightContext`, the lengths as an operand), the
export lowers them to one `iota`/`compare`/`select` keep mask, and the composite seam of 0.59 is
n-ary so a target that keeps attention fused receives `stablehlo.composite "skainet.sdpa"` with
the rules as attributes.

Version note: SKaiNET-transformers follows on this release with padded batches for the BERT and
Laya encoders. The SKEEP-006 compatibility aliases move to `DeprecationLevel.ERROR` as scheduled
in 0.59 (see *Deprecated*) and are removed in 0.61.

### Added

- **Key-padding lengths and window metadata for `scaledDotProductAttention`** (#1343, part 1): `scaledDotProductAttention(query, key, value, mask, scale, causal, keyLengths: Tensor<DType, *>? = null, window: Int = 0, rightContext: Int = 0)`. Padding (`keyLengths`, a `[batch]` integer tensor), the sliding-window band (`window`/`rightContext`, the `MultiHeadAttention` formula) and `causal` are *rules over positions* defined once in `AttentionMask` (`keyRange`, `attends`, `additive`, `MASKED = -1e30`), shared by the CPU kernel, the recorded graph and the StableHLO lowering. The CPU kernel computes scores only inside each row's attended key range — bit-identical to the dense `-1e30` mask on the attended keys, finite on a fully padded row — and a padded batch equals the unpadded single passes. The dense additive `mask` stays the generic fallback and is added on top of the rules.
Expand All @@ -16,6 +31,10 @@
- `TensorOps.scaledDotProductAttention` gained `keyLengths`, `window` and `rightContext`: implementors of the interface must update the override (callers are source-compatible through the defaults). The generated `GraphDsl` (`skainet-lang-dag`) now types optional tensor operands (`mask`, `bias`, `keyLengths`) as `GraphValue<*>? = null` instead of mandatory positionals.
- StableHLO SDPA export: the causal diagonal now carries the `Sk − Sq` offset the CPU kernel uses (a KV-cache decode step attends to its whole cache; `Sq == Sk` emission is unchanged); a dense mask and `causal = true` now apply together, as on the CPU, instead of the mask silently replacing the rule; positional/padding rules on a dynamic key length return a conversion failure instead of invalid `iota` IR (a dense mask still exports on the dynamic path).

### Deprecated

- **The SKEEP-006 compatibility surface moves to `DeprecationLevel.ERROR`** (scheduled in 0.59: ERROR in 0.60, removed in 0.61): the `sk.ainet.io.tokenizer.*` aliases (`Tokenizer`, `QwenByteLevelBpeTokenizer`, `SentencePieceTokenizer`, `SpecialTokenSplitter`, `TekkenTokenizer`, `ByteToUnicode`), `sk.ainet.io.model.ModelFormat`, `sk.ainet.io.weights.NameMap` / `RoleTableNameMap` / `TransformerNameMaps`, and in `skainet-io-gguf` `ModelArchitecture.fromGguf`, `ModelLoaderRegistry`, `LlamaLoaderDescriptor` and `Gemma3nLoaderDescriptor`. Sources that still name them stop compiling; every `@Deprecated` carries its `ReplaceWith` to the new home (`sk.ainet.frontend.text`, `sk.ainet.frontend.tokenizer`, `sk.ainet.model`). `TokenizerFactory` and `UnsupportedTokenizerException` stay real classes in `sk.ainet.io.tokenizer` for Java callers; `TransformerNameMaps.Gguf.forArchitecture` stays at WARNING.

### Fixed

- `DefaultCpuOps.scaledDotProductAttention` indexed a dense mask with `i % size`, which read batch row 0 for every batch: a `[batch, 1, Sq, Sk]` mask with `batch > 1` was silently wrong. The mask is now broadcast trailing-aligned (`[b | 1, H | nKV | 1, Sq | 1, Sk | 1]`, the StableHLO rule); the shapes that worked before index identically.
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ authors:
orcid: https://orcid.org/0009-0006-7981-4075
- name: SKaiNET-developers contributors
website: https://github.com/SKaiNET-developers
version: "0.59.0"
date-released: "2026-10-14"
version: "0.60.0"
date-released: "2026-10-15"
license: MIT
repository-code: https://github.com/SKaiNET-developers/SKaiNET
url: https://skainet-developers.github.io/SKaiNET/
Expand Down
50 changes: 30 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL):
```kotlin
dependencies {
// Recommended: import the umbrella BOM and drop versions on the engine modules.
implementation(platform("sk.ainet:skainet-bom:0.59.0"))
implementation(platform("sk.ainet:skainet-bom:0.60.0"))

implementation("sk.ainet.core:skainet-lang-core")
implementation("sk.ainet.core:skainet-backend-cpu")
Expand Down Expand Up @@ -308,24 +308,27 @@ val withoutLabel = dataPipeline<RawDataset>()

---

## What's New in 0.59.0

Exact GELU and lowering metadata (#1342):

- **`erf` and the exact GELU** — `gelu(approximate = false)` computes `0.5·x·(1 + erf(x/√2))`, the
form BERT-class encoders and the Laya decision models were trained with, on the CPU scalar kernel
and the JVM Vector API lanes. The default stays the tanh form, byte for byte; the exact backward
rule and `erfBackward` come with it.
- **The form travels as op metadata** — `parameters["approximate"]` sits on the graph node, so the
graph executor, the backward rule and the StableHLO export read the same flag instead of each
picking a default. StableHLO lowers `erf` to `chlo.erf`.
- **Fused ops as `stablehlo.composite`** — a `LoweringSpec` per op (`skainet.gelu`, `skainet.erf`)
emits a composite with a private decomposition function when a target's `OpGranularityPolicy`
keeps the op fused, and the inline decomposition otherwise, both from one builder. An explicit
`GeluApproximationLegalizationPass` records an exact→tanh rewrite on the node instead of
substituting silently, and `@Lowering` adds a *Lowering* column to the operator matrix.
- `TensorOps.gelu` gained a parameter: implementors update the override, callers are
source-compatible. The 0.58 io-core deprecation aliases stay at WARNING (ERROR in 0.60).
## What's New in 0.60.0

Key-padding lengths and window metadata for attention (#1343):

- **`scaledDotProductAttention(keyLengths, window, rightContext)`** — padding (a `[batch]`
lengths tensor), the sliding-window band and `causal` are rules over positions defined once in
`AttentionMask` and shared by the CPU kernel, the recorded graph and the StableHLO export. The
CPU kernel computes scores only inside each row's attended key range, so a padded batch equals
the unpadded single passes; the dense additive mask stays the generic fallback.
- **The rules travel as op metadata** — the recorded node carries `maskKind`, `window` and
`rightContext` with the lengths as an operand, so a compiled target regenerates the cheapest
legal form (`iota`/`compare`/`select` on the export) instead of inheriting a dense
`[batch, 1, Sq, Sk]` mask that scales with the batch.
- **`skainet.sdpa` composite** — the composite seam is n-ary (`HloDecomposition` takes every
operand), and `SdpaLowering.spec` emits `stablehlo.composite "skainet.sdpa"` with `scale`,
`causal`, `window`, `rightContext` and `maskKind` attributes when a target's
`OpGranularityPolicy` keeps attention fused, from the same builder as the inline form.
- The SKEEP-006 compatibility aliases (`sk.ainet.io.tokenizer.*`, `sk.ainet.io.model.ModelFormat`,
`sk.ainet.io.weights.NameMap`, the GGUF `ModelLoaderRegistry`) are errors since this release and
are removed in 0.61. Fixed on the way: a batched dense attention mask on the CPU read batch
row 0 for every batch.

See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release.

Expand All @@ -351,6 +354,13 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:

Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).

### Contributors (0.60.0)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — key-padding lengths
and window metadata for `scaledDotProductAttention` across the CPU kernel, recorder, executor
and StableHLO lowering, the n-ary composite seam with the `skainet.sdpa` composite, the
SKEEP-006 deprecation step, and the release

### Contributors (0.59.0)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — exact GELU and `erf`
Expand Down Expand Up @@ -446,7 +456,7 @@ from it). Ready-to-paste BibTeX:
author = {Harakal, Michal and {SKaiNET-developers contributors}},
title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework},
year = {2026},
note = {Version 0.59.0. MIT License},
note = {Version 0.60.0. MIT License},
howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ asciidoc:
framework_name: SKaiNET
# Current SKaiNET release — bump once per release; referenced as
# {skainet_version} in dependency snippets (blocks need subs="attributes+").
skainet_version: 0.59.0
skainet_version: 0.60.0
ksp_version: 2.2.21-2.0.5
dokka_version: 2.1.0
asciidoctorj_version: 3.0.0
4 changes: 2 additions & 2 deletions docs/modules/ROOT/pages/reference/input-frontends.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ own side files and decorated by its model family.
|===

`skainet-io-core` keeps `sk.ainet.io.tokenizer.*`, `sk.ainet.io.model.ModelFormat` and
`sk.ainet.io.weights.NameMap` as deprecated aliases for one release cycle, and
`TokenizerFactory` as a forwarding object for Java callers.
`sk.ainet.io.weights.NameMap` as deprecated aliases (WARNING in 0.58 and 0.59, ERROR since 0.60,
removed in 0.61), and `TokenizerFactory` as a forwarding object for Java callers.

== Resolving a checkpoint's tokenizer

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Kernel × platform support matrix
:description: Which compute-kernel provider serves each weight format on each KMP target.

Generated from `kernel-support.json` (version `0.59.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
Generated from `kernel-support.json` (version `0.60.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.

Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only).

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= AI-NET Operators Reference

Generated from version `0.59.0` on 2026-10-15
Generated from version `0.60.0` on 2026-10-15

== Operators by Modality

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/reference/ops-status-matrix.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Operator Coverage Matrix
:description: Cross-backend status for every operator function in SKaiNET.

Generated from `operators.json` version `0.59.0` on 2026-10-15.
Generated from `operators.json` version `0.60.0` on 2026-10-15.

Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). The `Lowering` column lists the compile-path forms the function declares with `@Lowering` (`composite` = a `stablehlo.composite` when a target keeps it fused, `decompose` = StableHLO/CHLO primitives, `native` = a 1:1 target op); `—` means undeclared. Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported").

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules:
[source,kotlin]
----
dependencies {
implementation("sk.ainet.core:skainet-lang-core:0.59.0") // tensors, DSL, training
implementation("sk.ainet.core:skainet-backend-cpu:0.59.0") // CPU ops
implementation("sk.ainet.core:skainet-compile-dag:0.59.0") // autograd (training context)
implementation("sk.ainet.core:skainet-data-api:0.59.0") // Dataset / DataBatch
implementation("sk.ainet.core:skainet-data-simple:0.59.0") // embedded Iris
runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.59.0") // NEON kernels (see below)
implementation("sk.ainet.core:skainet-lang-core:0.60.0") // tensors, DSL, training
implementation("sk.ainet.core:skainet-backend-cpu:0.60.0") // CPU ops
implementation("sk.ainet.core:skainet-compile-dag:0.60.0") // autograd (training context)
implementation("sk.ainet.core:skainet-data-api:0.60.0") // Dataset / DataBatch
implementation("sk.ainet.core:skainet-data-simple:0.60.0") // embedded Iris
runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.60.0") // NEON kernels (see below)
}
----

Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
GROUP=sk.ainet.core
VERSION_NAME=0.59.0
VERSION_NAME=0.60.0
POM_DESCRIPTION=SKaiNET

POM_URL=https://github.com/SKaiNET-developers/skainet/
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
package sk.ainet.io.model

/** Moved to `skainet-model-identity` (SKEEP-006); alias removed in 0.60. */
/** Moved to `skainet-model-identity` (SKEEP-006); alias an error since 0.60, removed in 0.61. */
@Deprecated(
"Moved to sk.ainet.model.ModelFormat (SKEEP-006); the alias is removed in 0.60.",
"Moved to sk.ainet.model.ModelFormat (SKEEP-006); the alias is an error since 0.60 and is removed in 0.61.",
ReplaceWith("ModelFormat", "sk.ainet.model.ModelFormat"),
level = DeprecationLevel.ERROR,
)
public typealias ModelFormat = sk.ainet.model.ModelFormat
Original file line number Diff line number Diff line change
@@ -1,31 +1,31 @@
@file:Suppress("DEPRECATION")
@file:Suppress("DEPRECATION", "DEPRECATION_ERROR")

package sk.ainet.io.tokenizer

/*
* SKEEP-006: the tokenizer implementations moved to `skainet-frontend-text`
* (`sk.ainet.frontend.tokenizer`) and the contract to `skainet-frontend-core`
* (`sk.ainet.frontend.text`). These aliases keep 0.57 sources compiling for one release cycle
* (WARNING in 0.58, ERROR in 0.59, removed in 0.60). `TokenizerFactory` stays a real object
* (`sk.ainet.frontend.text`). These aliases kept 0.57 sources compiling for one release cycle
* (WARNING in 0.58 and 0.59, ERROR in 0.60, removed in 0.61). `TokenizerFactory` stays a real object
* because Java callers pin its static methods.
*/

private const val MOVED = "Moved to skainet-frontend-core / skainet-frontend-text (SKEEP-006); the alias is removed in 0.60."
private const val MOVED = "Moved to skainet-frontend-core / skainet-frontend-text (SKEEP-006); the alias is an error since 0.60 and is removed in 0.61."

@Deprecated(MOVED, ReplaceWith("Tokenizer", "sk.ainet.frontend.text.Tokenizer"))
@Deprecated(MOVED, ReplaceWith("Tokenizer", "sk.ainet.frontend.text.Tokenizer"), level = DeprecationLevel.ERROR)
public typealias Tokenizer = sk.ainet.frontend.text.Tokenizer

@Deprecated(MOVED, ReplaceWith("QwenByteLevelBpeTokenizer", "sk.ainet.frontend.tokenizer.QwenByteLevelBpeTokenizer"))
@Deprecated(MOVED, ReplaceWith("QwenByteLevelBpeTokenizer", "sk.ainet.frontend.tokenizer.QwenByteLevelBpeTokenizer"), level = DeprecationLevel.ERROR)
public typealias QwenByteLevelBpeTokenizer = sk.ainet.frontend.tokenizer.QwenByteLevelBpeTokenizer

@Deprecated(MOVED, ReplaceWith("SentencePieceTokenizer", "sk.ainet.frontend.tokenizer.SentencePieceTokenizer"))
@Deprecated(MOVED, ReplaceWith("SentencePieceTokenizer", "sk.ainet.frontend.tokenizer.SentencePieceTokenizer"), level = DeprecationLevel.ERROR)
public typealias SentencePieceTokenizer = sk.ainet.frontend.tokenizer.SentencePieceTokenizer

@Deprecated(MOVED, ReplaceWith("SpecialTokenSplitter", "sk.ainet.frontend.tokenizer.SpecialTokenSplitter"))
@Deprecated(MOVED, ReplaceWith("SpecialTokenSplitter", "sk.ainet.frontend.tokenizer.SpecialTokenSplitter"), level = DeprecationLevel.ERROR)
public typealias SpecialTokenSplitter = sk.ainet.frontend.tokenizer.SpecialTokenSplitter

@Deprecated(MOVED, ReplaceWith("TekkenTokenizer", "sk.ainet.frontend.tokenizer.TekkenTokenizer"))
@Deprecated(MOVED, ReplaceWith("TekkenTokenizer", "sk.ainet.frontend.tokenizer.TekkenTokenizer"), level = DeprecationLevel.ERROR)
public typealias TekkenTokenizer = sk.ainet.frontend.tokenizer.TekkenTokenizer

@Deprecated(MOVED, ReplaceWith("ByteToUnicode", "sk.ainet.frontend.tokenizer.ByteToUnicode"))
@Deprecated(MOVED, ReplaceWith("ByteToUnicode", "sk.ainet.frontend.tokenizer.ByteToUnicode"), level = DeprecationLevel.ERROR)
public typealias ByteToUnicode = sk.ainet.frontend.tokenizer.ByteToUnicode
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
@file:Suppress("DEPRECATION")
@file:Suppress("DEPRECATION", "DEPRECATION_ERROR")

package sk.ainet.io.weights

import sk.ainet.lang.tensor.TensorId

/** Moved to `skainet-model-identity` (SKEEP-006); alias removed in 0.60. */
@Deprecated("Moved to sk.ainet.model.NameMap (SKEEP-006); the alias is removed in 0.60.", ReplaceWith("NameMap", "sk.ainet.model.NameMap"))
/** Moved to `skainet-model-identity` (SKEEP-006); alias an error since 0.60, removed in 0.61. */
@Deprecated("Moved to sk.ainet.model.NameMap (SKEEP-006); the alias is an error since 0.60 and is removed in 0.61.", ReplaceWith("NameMap", "sk.ainet.model.NameMap"), level = DeprecationLevel.ERROR)
public typealias NameMap = sk.ainet.model.NameMap

/** Moved to `skainet-model-identity` (SKEEP-006); alias removed in 0.60. */
@Deprecated("Moved to sk.ainet.model.RoleTableNameMap (SKEEP-006); the alias is removed in 0.60.", ReplaceWith("RoleTableNameMap", "sk.ainet.model.RoleTableNameMap"))
/** Moved to `skainet-model-identity` (SKEEP-006); alias an error since 0.60, removed in 0.61. */
@Deprecated("Moved to sk.ainet.model.RoleTableNameMap (SKEEP-006); the alias is an error since 0.60 and is removed in 0.61.", ReplaceWith("RoleTableNameMap", "sk.ainet.model.RoleTableNameMap"), level = DeprecationLevel.ERROR)
public typealias RoleTableNameMap = sk.ainet.model.RoleTableNameMap

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
package sk.ainet.io.weights

/** Moved to `skainet-model-identity` (SKEEP-006); alias removed in 0.60. */
/** Moved to `skainet-model-identity` (SKEEP-006); alias an error since 0.60, removed in 0.61. */
@Deprecated(
"Moved to sk.ainet.model.TransformerNameMaps (SKEEP-006); the alias is removed in 0.60.",
"Moved to sk.ainet.model.TransformerNameMaps (SKEEP-006); the alias is an error since 0.60 and is removed in 0.61.",
ReplaceWith("TransformerNameMaps", "sk.ainet.model.TransformerNameMaps"),
level = DeprecationLevel.ERROR,
)
public typealias TransformerNameMaps = sk.ainet.model.TransformerNameMaps
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
package sk.ainet.io.tokenizer

import sk.ainet.frontend.tokenizer.UnigramTokenizer
import sk.ainet.frontend.tokenizer.ByteToUnicode
import sk.ainet.frontend.tokenizer.QwenByteLevelBpeTokenizer
import sk.ainet.frontend.tokenizer.SentencePieceTokenizer
import sk.ainet.frontend.tokenizer.SpecialTokenSplitter
import sk.ainet.frontend.tokenizer.WordPieceTokenizer

import kotlin.test.Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
package sk.ainet.io.tokenizer

import sk.ainet.io.JvmRandomAccessSource
import sk.ainet.frontend.text.Tokenizer
import sk.ainet.frontend.tokenizer.QwenByteLevelBpeTokenizer
import sk.ainet.io.gguf.GgufModelMetadata
import sk.ainet.io.gguf.StreamingGGUFReader
import java.io.File
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
package sk.ainet.io.tokenizer

import sk.ainet.io.JvmRandomAccessSource
import sk.ainet.frontend.text.Tokenizer
import sk.ainet.frontend.tokenizer.SentencePieceTokenizer
import sk.ainet.io.gguf.StreamingGGUFReader
import java.io.File
import kotlin.test.Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package sk.ainet.io.gguf

import kotlinx.io.buffered
import sk.ainet.io.model.*
import sk.ainet.model.ModelFormat
import sk.ainet.io.openRandomAccessSource

/**
Expand Down
Loading
Loading