Add support to Turing / sm75 architecture

[nicolasvila]

Before you start

• I have read the Roadmap and this is not already planned there.
• I have read the FAQ.
• I have searched existing issues and found no duplicate.

Applies to

Both

What problem does this solve

Support all RTX 20x0 GPUs, which are compatible with CUDA 13.x
Adding support for SM75 or SM_75, compute_75 – will allow compatibility with

This would add support to the following GPUs:
GTX/RTX Turing – GTX 1660 Ti, RTX 2060, RTX 2070, RTX 2080, Titan RTX, Quadro RTX 4000, Quadro RTX 5000, Quadro RTX 6000, Quadro RTX 8000, Quadro T1000/T2000, Tesla T4

I've been able to run Qwen3.6-35B-A3B on my old RTX 2070 with 8GB ! Thanks to Freetoken, which I patched to support Turing architecture GPUs (2xxx, Tesla T4, etc.) I use crurrently 64k context with 20 tok/sec. I've been able to reach 45tok/sec, with som peaks at 50. Slow but usable in opencode. Tested on python code generation with surprising results. I wish I had 16GB at least but we can do stuff with small VRAM
BTW, I don't use Freetoken Desktop, I created a custom docker image (which is quite hugeBTW) optimized for such low-end GPUs.

My setup is the following :
GPU: RTX 2070 8GB GPU
AMD Ryzen 9 5950X 16-Core Processor
64 GB of DDR4 RAM

Proposed solution

Simply patch attention.py
I made a really (ridiculously) simple patch

diff patch_sm75/attention.py patch_sm75/attention-sm75.py
41c41
< return (128, 64) if fits(128, 64) else (64, 32)

    return (128, 64) if fits(128, 64) else (32, 16)

Here is the benchmark file from my card

benchbw.json

For now I mounted directly the patched file in my Docker image.

services:
freetoken:
volumes:
- ./patch_sm75/attention-sm75.py:/opt/venv/lib/python3.12/site-packages/freetoken/kernel/triton/attention.py:ro

Alternatives considered

No response

[nicolasvila]

Already an PR to add a support of Turring GPUs: #24