Add a TBLIS.jl package extension - #290
Open
lkdvos wants to merge 5 commits into
Open
Conversation
lkdvos
marked this pull request as ready for review
August 7, 2026 13:30
Codecov Report❌ Patch coverage is
... and 6 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Adds `TBLISBackend`, which routes `tensoradd!`, `tensortrace!` and `tensorcontract!` through the TBLIS library via TBLIS.jl. TBLIS contracts strided tensors in place, so it avoids the permuted intermediates that the BLAS-based backend has to materialize. The backend is opt-in: loading TBLIS.jl does not register a `select_backend` method. Arguments TBLIS cannot express, that is mixed or unsupported element types, non-strided arrays and conjugated outputs, throw an `ArgumentError` rather than being handed to another backend, so that a contraction which never reaches TBLIS cannot pass for one that did. Conjugation is carried as an ordinary runtime flag on the `TBLISTensor` descriptor rather than in the type of a `StridedView`, which keeps the entry points type stable without branching over `conj` variants. A conjugation already present on the input, as for an `Adjoint`, combines with the requested one. Two library quirks are worked around: * `tblis_tensor_mult` ignores the per-tensor conjugation flags, unlike `tblis_tensor_add`. When both factors are conjugated this is resolved by conjugating the output in place, and otherwise by materializing the conjugated factor into a temporary from the allocator. * TBLIS.jl only exposes its tensor constructor for `StridedArray` and offers no way to set the conjugation flag, so the descriptor is initialized through the low-level bindings and every referenced buffer is rooted with `GC.@preserve`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Replace the `TBLISTensor` struct with a `tblis_tensor` function returning
the `Ref{tblis_tensor}` directly. The caller owns the length and stride
buffers and keeps them rooted, so nothing has to be wrapped to stay alive.
* `isconj` specializes on `StridedView` to fold any conjugation the view
already carries into the one the caller requested.
* Inline `tblis_add!` and `unsafe_add!` into `tensoradd!` and `tensortrace!`.
`materialize_conj` now goes through `tensoradd!` instead of a private copy
of the same call sequence.
* Drop the `try`/`finally` around the conjugated temporaries and free them on
the success path only, matching how the cuTENSOR extension handles this.
* Guard the threading test so it only asks for as many threads as the machine
reports, for single-core CI runners.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Note at each `SV` conversion that it is there to support inputs such as `Adjoint`, which have no `strides` or `pointer` of their own. * Explain why the `C` descriptor is always built unconjugated: TBLIS applies the flag of `C` when reading the `β * C` term but not when writing the result back, so a conjugated output would conjugate half the operation. `check_arguments` rejects such a `C` for that reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keeps the section banners and the `tblis_tensor` docstring. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One-line comments for the two library restrictions that are not visible from the code, and `LazyString` messages for the three argument-check errors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
TensorOperationsTBLISExt, providing aTBLISBackendon top of TBLIS.jl v0.3.Supersedes the standalone TensorOperationsTBLIS.jl.
What it does
TBLISBackend()routestensoradd!,tensortrace!andtensorcontract!through TBLIS, which contracts strided tensors in place rather than reshaping them into matrices, avoiding the permuted intermediates the BLAS path has to materialize.It is opt-in: loading TBLIS.jl registers no
select_backendmethod.Design notes
ArgumentErrorinstead of being delegated to another backend, so a contraction that never reaches TBLIS cannot pass for one that did.StridedView's type, so the entry points stay type stable without branching overconjvariants (@inferredcovers all three). A conjugation already carried by the input, as for anAdjoint, combines with the requested one.Library quirks worked around
tblis_tensor_multignores the per-tensor conjugation flags, whiletblis_tensor_addhonours them (verified against tblis 1.3). Resolved by conjugating the output in place when both factors are conjugated, and by materializing the conjugated factor into an allocator temporary otherwise.tblis_tensorconstructor only acceptsStridedArray, so it cannot take theStridedViews these kernels use, and it cannot set the conjugation flag. The descriptor is initialized through the low-level bindings and every referenced buffer is rooted withGC.@preserve. Widening that constructor upstream would let this drop back to the public API.Tests
test/tblis.jl, wired intoruntests.jl: compared againstStridedNativeover all four supported element types for add / trace / contract, every conjugation combination,Adjointinputs, non-contiguous views, outer products, scalar outputs, argument checking, the rejection paths,@tensor/nconintegration, a GC-pressure loop, and threading.All 20 testsets pass against the registered TBLIS.jl v0.3.0.
β == 0is exercised withNaN-poisoned outputs, confirming TBLIS ignoresCin that case for bothaddandmult.Performance
Measured separately (the harness is not part of this PR) on 2× Xeon Gold 6244, with OpenBLAS, TBLIS and Strided all on 16 threads.
TBLIS is not a throughput win on this hardware. It is roughly on par with
StridedBLASfor permuted real contractions (1.19×) and slower for other shapes: 0.64× on GEMM, 0.94× on a DMRG-style network, 0.56× on a partial trace.What it buys is memory: for a permuted contraction
StridedBLASallocates 76.6 MiB of temporaries per call where TBLIS allocates zero.Warning
As of
tblis_jllv1.3, TBLIS has no competitive support for complex element types. Complex contractions run at 0.04–0.11× of BLAS; a baretblis_tensor_multon a 1000³ complex matrix product reaches ~25 GFLOP/s against ~500 GFLOP/s formul!.tblis_jllv1.2.0 behaves identically, so this is not a regression from the v1.3 bump. This is called out in theTBLISBackenddocstring and in the backends documentation.🤖 Generated with Claude Code