Skip to content

Add a TBLIS.jl package extension - #290

Open
lkdvos wants to merge 5 commits into
mainfrom
ld/tblis-ext
Open

Add a TBLIS.jl package extension#290
lkdvos wants to merge 5 commits into
mainfrom
ld/tblis-ext

Conversation

@lkdvos

@lkdvos lkdvos commented Aug 2, 2026

Copy link
Copy Markdown
Member

Adds TensorOperationsTBLISExt, providing a TBLISBackend on top of TBLIS.jl v0.3.
Supersedes the standalone TensorOperationsTBLIS.jl.

What it does

TBLISBackend() routes tensoradd!, tensortrace! and tensorcontract! through TBLIS, which contracts strided tensors in place rather than reshaping them into matrices, avoiding the permuted intermediates the BLAS path has to materialize.

It is opt-in: loading TBLIS.jl registers no select_backend method.

using TensorOperations, TBLIS
TBLIS.set_num_threads(16)
@tensor backend = TensorOperations.TBLISBackend() D[a,b,c,d] := A[a,e,c,f] * B[g,d,e] * conj(C[g,f,b])

Design notes

  • Unsupported arguments throw. Mixed or unsupported element types, non-strided arrays and conjugated output views raise an ArgumentError instead of being delegated to another backend, so a contraction that never reaches TBLIS cannot pass for one that did.
  • Conjugation is a runtime flag on the descriptor, not part of a StridedView's type, so the entry points stay type stable without branching over conj variants (@inferred covers all three). A conjugation already carried by the input, as for an Adjoint, combines with the requested one.

Library quirks worked around

  • tblis_tensor_mult ignores the per-tensor conjugation flags, while tblis_tensor_add honours them (verified against tblis 1.3). Resolved by conjugating the output in place when both factors are conjugated, and by materializing the conjugated factor into an allocator temporary otherwise.
  • TBLIS.jl's tblis_tensor constructor only accepts StridedArray, so it cannot take the StridedViews these kernels use, and it cannot set the conjugation flag. The descriptor is initialized through the low-level bindings and every referenced buffer is rooted with GC.@preserve. Widening that constructor upstream would let this drop back to the public API.

Tests

test/tblis.jl, wired into runtests.jl: compared against StridedNative over all four supported element types for add / trace / contract, every conjugation combination, Adjoint inputs, non-contiguous views, outer products, scalar outputs, argument checking, the rejection paths, @tensor/ncon integration, a GC-pressure loop, and threading.
All 20 testsets pass against the registered TBLIS.jl v0.3.0.

β == 0 is exercised with NaN-poisoned outputs, confirming TBLIS ignores C in that case for both add and mult.

Performance

Measured separately (the harness is not part of this PR) on 2× Xeon Gold 6244, with OpenBLAS, TBLIS and Strided all on 16 threads.

TBLIS is not a throughput win on this hardware. It is roughly on par with StridedBLAS for permuted real contractions (1.19×) and slower for other shapes: 0.64× on GEMM, 0.94× on a DMRG-style network, 0.56× on a partial trace.
What it buys is memory: for a permuted contraction StridedBLAS allocates 76.6 MiB of temporaries per call where TBLIS allocates zero.

Warning

As of tblis_jll v1.3, TBLIS has no competitive support for complex element types. Complex contractions run at 0.04–0.11× of BLAS; a bare tblis_tensor_mult on a 1000³ complex matrix product reaches ~25 GFLOP/s against ~500 GFLOP/s for mul!. tblis_jll v1.2.0 behaves identically, so this is not a regression from the v1.3 bump. This is called out in the TBLISBackend docstring and in the backends documentation.

🤖 Generated with Claude Code

@lkdvos
lkdvos marked this pull request as ready for review August 7, 2026 13:30
@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.97959% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
ext/TensorOperationsTBLISExt.jl 98.96% 1 Missing ⚠️
Files with missing lines Coverage Δ
src/backends.jl 77.77% <100.00%> (+6.34%) ⬆️
ext/TensorOperationsTBLISExt.jl 98.96% <98.96%> (ø)

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Adds `TBLISBackend`, which routes `tensoradd!`, `tensortrace!` and
`tensorcontract!` through the TBLIS library via TBLIS.jl. TBLIS contracts
strided tensors in place, so it avoids the permuted intermediates that the
BLAS-based backend has to materialize.

The backend is opt-in: loading TBLIS.jl does not register a `select_backend`
method. Arguments TBLIS cannot express, that is mixed or unsupported element
types, non-strided arrays and conjugated outputs, throw an `ArgumentError`
rather than being handed to another backend, so that a contraction which never
reaches TBLIS cannot pass for one that did.

Conjugation is carried as an ordinary runtime flag on the `TBLISTensor`
descriptor rather than in the type of a `StridedView`, which keeps the entry
points type stable without branching over `conj` variants. A conjugation
already present on the input, as for an `Adjoint`, combines with the requested
one.

Two library quirks are worked around:

* `tblis_tensor_mult` ignores the per-tensor conjugation flags, unlike
  `tblis_tensor_add`. When both factors are conjugated this is resolved by
  conjugating the output in place, and otherwise by materializing the
  conjugated factor into a temporary from the allocator.
* TBLIS.jl only exposes its tensor constructor for `StridedArray` and offers no
  way to set the conjugation flag, so the descriptor is initialized through the
  low-level bindings and every referenced buffer is rooted with `GC.@preserve`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lkdvos and others added 4 commits August 7, 2026 11:03
* Replace the `TBLISTensor` struct with a `tblis_tensor` function returning
  the `Ref{tblis_tensor}` directly. The caller owns the length and stride
  buffers and keeps them rooted, so nothing has to be wrapped to stay alive.
* `isconj` specializes on `StridedView` to fold any conjugation the view
  already carries into the one the caller requested.
* Inline `tblis_add!` and `unsafe_add!` into `tensoradd!` and `tensortrace!`.
  `materialize_conj` now goes through `tensoradd!` instead of a private copy
  of the same call sequence.
* Drop the `try`/`finally` around the conjugated temporaries and free them on
  the success path only, matching how the cuTENSOR extension handles this.
* Guard the threading test so it only asks for as many threads as the machine
  reports, for single-core CI runners.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Note at each `SV` conversion that it is there to support inputs such as
  `Adjoint`, which have no `strides` or `pointer` of their own.
* Explain why the `C` descriptor is always built unconjugated: TBLIS applies
  the flag of `C` when reading the `β * C` term but not when writing the
  result back, so a conjugated output would conjugate half the operation.
  `check_arguments` rejects such a `C` for that reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keeps the section banners and the `tblis_tensor` docstring.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One-line comments for the two library restrictions that are not visible from
the code, and `LazyString` messages for the three argument-check errors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lkdvos
lkdvos requested a review from kshyatt August 7, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant