Skip to content

Ground truth: expand Python-side op coverage to match OperationExecutor #985

Open

Description

@michalharakal

Sub-issue of #984.

skainet-test-groundtruth's OperationExecutor already dispatches ~25 op names to
SKaiNET's TensorOps (matmul, transpose, relu, leakyRelu, elu, sigmoid, silu, gelu,
softmax, logSoftmax, maxpool2d, avgpool2d, sum, mean, variance, squeeze, unsqueeze,
conv1d, ...) — but skainet-ground-truth's Python side currently only produces tagged
op_type fixtures for add, subtract, conv2d, flatten. Most of the dispatch
logic is untested by anything real today.

Task

Add new TS-XXX/UC-YYY.py test suites in skainet-ground-truth, one (or a few) per
currently-unfixtured op, following the existing @Executable(description, op_type=..., op_params={...}) pattern (see TS-001 for the op_params convention — every
non-default PyTorch argument needs to be passed via op_params too, or it can't be
reproduced on the Kotlin side, see CONTRACT.md).

Suggested priority order (cheapest to verify first): matmul, relu, sigmoid,
softmax (all pure elementwise/matrix, easy to eyeball), then maxpool2d/avgpool2d
(need real op_params), then conv1d/transpose/reductions.

Each new suite should be run against skainet-test-groundtruth locally
(./gradlew generateGroundTruth jvmTest) before merging, to confirm SKaiNET's
implementation actually matches — this issue is explicitly about testing existing
OperationExecutor coverage, not implementing new SKaiNET ops.

Activity

  1. michalharakal commented on Aug 13, 2026

    @michalharakal
    ContributorAuthor

    Tooling note (not required, just an option worth knowing about when this issue adds a
    lot more individual test cases): today's `GroundTruthIntegrationTest` loops over all
    GGUF fixtures in a test suite inside one `@Test`, aggregating pass/fail into a single
    assertion — CI shows "1 test failed," not which specific case. As coverage grows
    (dozens of ops per #985), that gets harder to read.

    OpenSavvy Prepared's dynamic-test
    pattern — `suite { for (case in cases) { test(case.name) { ... } } }` — would turn each
    fixture into its own individually-reported test instead. Also has a KMP-portable
    `prepared { }` lazy fixture builder that maps cleanly onto "load this GGUF, build
    tensors." Full context on the library in #983.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    codingImplementation task (DARC: C)enhancementNew feature or requestsub-issueSub-issue of a tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions