Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
432 commits
Select commit Hold shift + click to select a range
f336359
Add more benchmarks to llama-eval (#26174)
pwilkin Oct 16, 2026
2a66703
convert : respect Mistral format for DeepseekV2Model (#30669)
aiqubits Oct 16, 2026
7182ef7
common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client (#21752)
texasich Oct 16, 2026
23f7aab
model : Granite Switch Decision (#30676)
gabe-l-hart Oct 16, 2026
01fc3aa
model: add Beam (Reflection) (#30670)
danielhanchen Oct 16, 2026
2f2763d
opencl: Add MxFP4 MoE bin kernel for Adreno X1 devices (#30691)
shawngu-quic Oct 16, 2026
27c7a76
hexagon: vectorize softplus with HVX (#30435)
huangwei858 Oct 16, 2026
6420cf5
ci: update device matrix for Snapdragon jobs by removing SM8750 (#30620)
zhiyuan8 Oct 16, 2026
ea0c4c2
skills: add slice-model (#30677)
ngxson Oct 17, 2026
f74e2fa
sycl: fix prompt-processing regression after Q4_0/Q8_0 weight reorder…
marcusds Oct 17, 2026
1268eda
model : Granite switch decision 256 options (#30693)
gabe-l-hart Oct 17, 2026
83f9389
ci : added ABI reports as github artifact (#30619)
sandrohanea Oct 17, 2026
62318cd
model : add Kolibri-1 (Aleph-Alpha) support (#30673)
riteme Oct 17, 2026
f2f667b
cuda : add grouped experts top-k fusion (#29181)
CISC Oct 17, 2026
9d1e036
cuda : fix -Wtautological-compare in fattn vec dispatch (#30519)
angt Oct 17, 2026
10b8b75
tests: give the random model generator a per-tensor filler table (#30…
ServeurpersoCom Oct 17, 2026
2744e48
CUDA: faster MMVQ for low-bit types (#30517)
am17an Oct 17, 2026
c0fd786
ui : Discover & Download Models from HF Hub (#29584)
allozaur Oct 17, 2026
9d1878a
ggml-cuda: HIP: use __builtin_amdgcn_perm for q1_0 unpacking (#29927)
thelittlefireman Oct 17, 2026
797de63
hexagon: add support for Q1_0 and Q2_0 (#30122)
tboinovski1 Oct 17, 2026
848bbda
hexagon: add support for hadamard MUL_MAT and GET_ROW for flat and ti…
tboinovski1 Oct 18, 2026
547208d
Restore ESIMD kernels on FFN path (#30239)
malsbat Oct 18, 2026
5aeac37
ggml-cuda: fattn do not check oob if k_VKQ_max = (k_VKQ_max - nbatch_…
thelittlefireman Oct 18, 2026
0878f7c
ggml: support non-contiguous SUM (CPU, CUDA, Vulkan, WebGPU) (#28851)
rssndr Oct 18, 2026
ff5a233
cuda : fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain (#3…
am17an Oct 18, 2026
fe8841a
hexagon: allow multistream FA (#30218)
jhen0409 Oct 19, 2026
31288c9
hexagon: FLASH_ATTN_EXT improvements for longer context and SWA model…
max-krasnyansky Oct 19, 2026
99db719
CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec (#30573)
invaliddayta Oct 19, 2026
7db0d3d
pi : turn .pi/gg into a system prompt addendum (#30762)
ggerganov Oct 19, 2026
27af581
tests : update Metal fusion baseline for beam (#30764)
ggerganov Oct 19, 2026
8f97d82
tools : reject truncated GGUF tensor data in split and export tools (…
ppAlexa Oct 19, 2026
2b57148
ui: Fix markdown tables and lists being cut off on phones (#30687)
allozaur Oct 19, 2026
412a363
qwen3 : bidirectional attention and dense_2 output projection (#30607)
fzowl Oct 19, 2026
4c1004c
cuda: cap MMQ Stream-K blocks at available work (#30553)
joelagnel Oct 19, 2026
42b7003
ggml-cpu: vectorize ggml_v_silu and ggml_v_expf (#30791)
taronaeo Oct 19, 2026
4866db3
CUDA: fix warning for nonexistent extra semicolon (#30774)
JohannesGaessler Oct 19, 2026
fda00f1
tests: add big-endian conversion tests (#30705)
taronaeo Oct 19, 2026
f3fb2fc
cann : refactor `#ifdef` blocks to avoid unreachable code after `retu…
rauletorresc Oct 19, 2026
098ce52
metal : fix moe_reduce fusion race when dst aliases its inputs (#30795)
ggerganov Oct 19, 2026
33d6135
moe-cache : replace MoE LRU cache with a 50/50 SLRU cache (#30637)
eapache Oct 19, 2026
7951f60
vendor : update cpp-httplib to 0.62.0 (#30785)
cabelo Oct 19, 2026
f174553
server : fix swa checkpoint invalidation logic (#30666)
tdakhran Oct 19, 2026
0ce8cd1
jinja : implement `unique` filter (#30819)
aldehir Oct 19, 2026
6fa4f6c
opencl: add get_rows for i32, cpy for i32 and f32 (#30756)
lhez Oct 19, 2026
1b29621
opencl: add cooperative-k dp4a gemm for q4_0, q4_k and q6_k (#30741)
lhez Oct 19, 2026
4c3c581
hybrid dna tokenizer fixes (#30825)
kashif Oct 19, 2026
ab7ed0d
ui : fix attachment text preview clipped when a line is longer than t…
A-Raphie Oct 19, 2026
7adf160
server: add --connect, allow p2p access to server outside LAN (#28277)
ngxson Oct 19, 2026
3ffb629
ci : re-enable type/lint-check for skills (#30713)
CISC Oct 19, 2026
921257f
ci, docs: add WASI CI and docs (#30754)
MendyBerger Oct 19, 2026
f711c8f
mtmd : match the transformers Qwen-VL resize (#30736)
boshjerns Oct 19, 2026
3bb63cb
sycl : fix get_rows on reordered quantized weights (#30430)
skipzx-lab Oct 19, 2026
1d88cb0
ui: hide the native clear button in SearchInput (#30719)
aiqubits Oct 20, 2026
dd002b9
sycl: fix compiler warnings - missing prototypes, dead code, extra se…
mctylr-gh Oct 20, 2026
8466395
hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (…
max-krasnyansky Oct 20, 2026
7879491
tests : make MoE router selection independent of the backend noise (#…
ggerganov Oct 20, 2026
d8d9297
vocab : add \p{M} to lookaheads in Tekken and GPT-4o pre-tokenizer re…
tkran2 Oct 20, 2026
707616f
ci: Fix --strict mode for API check (#30623)
ckastner Oct 20, 2026
704366d
metal : fix missing MoE kernel variant for 9 experts per token (#3061…
Caladrel65 Oct 20, 2026
d806193
server : pass id_slot through the Anthropic /v1/messages conversion (…
pcvelz Oct 20, 2026
13786fb
CUDA: Clear CUDA graphs after legacy memory pool flush (#26771)
bokrosbalint Oct 20, 2026
83c2d01
spec : save the dynamic ngram cache file (#22055)
petersid2022 Oct 20, 2026
522f2b6
ggml-zdnn: introduce flash attention op (#30512)
taronaeo Oct 20, 2026
b684498
HIP/CUDA: Fix reduction in mmf getting tile widths not devisiable by …
IMbackK Oct 20, 2026
119f68d
ggml-cpu : select the NEON fp32->fp16 conversion (#30505)
bernardladenthin Oct 20, 2026
8115b4c
sycl : replace memset due to not always ordered (#30812)
arthw Oct 20, 2026
a01ce9f
opencl: add several gemm variants for small batches and some fixes (#…
lhez Oct 20, 2026
b38960f
ci : remove superfluous webgpu jobs (#30868)
CISC Oct 20, 2026
9df0b73
model : drop the redundant precision overrides in the Kolibri-1 route…
ggerganov Oct 20, 2026
df11702
metal : optimize argmax and pool_1d (#30862)
ggerganov Oct 20, 2026
36140db
mtmd : read LFM2 tiling params from GGUF metadata (#30840)
tdakhran Oct 20, 2026
1ba568c
server: fix missing output_index in Responses API SSE events for comp…
Renzishi Oct 20, 2026
4e20138
server: Router wasn't propagating the client disconnects fully to the…
Me42019 Oct 20, 2026
39da130
ggml-cuda : support ssm_conv kernel size 7 (#30873)
tdakhran Oct 20, 2026
5a9a34d
server : avoid repeated mmproj memory margin addition on sleep/resume…
TANGBUDU Oct 20, 2026
5ba3b9f
mtmd: fix gemma 4 ffn act (#30882)
ngxson Oct 20, 2026
7e3eeee
model : add H2O-Lightning-4B decision model (#30361)
pseudotensor Oct 20, 2026
4a45de4
mtmd : lfm2 tiles and layout match transformers (#30261)
ykhrustalev Oct 20, 2026
b684b49
hexagon: fix VTCM size query on images that need layout out-params (#…
tdakhran Oct 20, 2026
ead9f32
server: Support logprobs with speculative decoding (#27196)
pnb Oct 20, 2026
839a572
qwen: reshape output to 2d instead of 3d (#30823)
am17an Oct 20, 2026
52ffa02
model : support Nemotron-3(.5)-Embed (#30877)
tokinasin Oct 20, 2026
84e66d0
refactor : move LFM2 shortconv and FFN blocks into a shared LFM base …
tdakhran Oct 20, 2026
1242be6
server: add realtime transcription API (#30636)
ngxson Oct 20, 2026
5c0b49a
server : add POST /tts endpoint for TTS models (#26603)
ngxson Oct 20, 2026
9a3b91c
server : fix usage.completion_tokens undercount with n > 1 (#29486)
unbug Oct 20, 2026
339f324
opencl: fix rope program build error with newer Clang (#30893)
bashbaug Oct 21, 2026
493253f
Sycl iq3 crash fix (#30604)
anantshri Oct 21, 2026
8baabd5
llama : scale moe down projection per expert token (#30861)
ggerganov Oct 21, 2026
479a527
ggml-cpu : compute f32 gelu_quick and geglu_quick without the fp16 ta…
boshjerns Oct 21, 2026
96caa80
sycl: collapse duplicate need_check launch bodies in mmq (#30739)
Titaniumtown Oct 21, 2026
74b115c
sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / …
arthw Oct 21, 2026
b531e4c
opencl: add dp4a GEMM variants to vector load activation (#30753)
lhez Oct 21, 2026
0efe8e4
hexagon: neon-vectorized and threaded repack (tiling) (#30901)
max-krasnyansky Oct 21, 2026
ea9e344
metal : fix memory leaks and cleanup logic in GPU capture path (#30890)
nikwen Oct 21, 2026
eb182d8
llama : place d2t by its set_rows use so it can stay on the GPU (#30912)
am17an Oct 21, 2026
824eaec
llama : optimize vocabulary loading (#30679)
mykhailen Oct 21, 2026
f849095
vendor: update BoringSSL to 0.20261020.0 (#30889)
cabelo Oct 21, 2026
261700b
server : forward llama.cpp sampling params on /v1/messages (#30856)
Dgarc359 Oct 21, 2026
0a1ef30
mtmd: add state object for mtmd_gen (#30902)
ngxson Oct 21, 2026
9b56ccf
ggml-opencl : add IM2COL_3D support (#30895)
kurquhar Oct 21, 2026
9febe63
jinja: support hexadecimal string escapes (#30929)
alexey-rsi Oct 21, 2026
0756e83
sycl: q8 fast dequant (#30775)
92es Oct 21, 2026
d454a20
llama: create new fused op for QSA mask (#30640)
am17an Oct 21, 2026
7775ffb
model: Add support for SenseNova U1 model (#28919)
Maphist0 Oct 21, 2026
99fb997
jinja : fix namespace attributes shadowed by dict methods (#30941)
tdakhran Oct 21, 2026
a37cc46
models : add Flint model (FlintForCausalLM) (#30928)
alexey-rsi Oct 21, 2026
c7fc509
opencl: add F32-to-I32 ARGMAX (#30897)
kurquhar Oct 22, 2026
a6b83d4
ui : key draft sidecar badges by model instead of repo (#30919)
allozaur Oct 22, 2026
0682a57
lora : support prism.hadamard folded weights (#30910)
vieduy Oct 22, 2026
84746c5
ui: Add "Show tool call in progress" setting (#30681)
allozaur Oct 22, 2026
4912ac2
dflash : load optional FC input scale (#30945)
hthadicherla Oct 22, 2026
0b1281a
webgpu : add Q2_0 support (#30914)
teddygood Oct 22, 2026
e3797b1
sycl: merge the fp16 and fp32 dequant switches (#30740)
Titaniumtown Oct 22, 2026
dfeea52
kleidiai : add Q8_0 SVE dotprod GEMV and SVE i8mm GEMM kernels (#30808)
chaxu01 Oct 22, 2026
bf039c8
models : Record layer inputs for all granite models (#30957)
gabe-l-hart Oct 22, 2026
e197cef
vocab : use Tekken pre-tokenizer for pixtral rather than the Llama 3 …
tkran2 Oct 22, 2026
d88040c
vocab : add clean_spaces GGUF key (#30940)
tkran2 Oct 22, 2026
7170cad
sycl: fix CONV_2D wrong results for non-contiguous input (#30363)
mctylr-gh Oct 22, 2026
1f4c2ee
ci : be able to reconfigure HTTP port for server pytest workers (#30972)
ggerganov Oct 22, 2026
5f5acfe
ci : disable slow WebGPU tests (#30963)
ggerganov Oct 22, 2026
a5d2227
server : support typed content input for RERANK endpoint (#29625)
timothywang21 Oct 22, 2026
0693dbf
nemotron : support non-standard GQA, BPE tokenizer and domyn-small pr…
martin-cimmino Oct 22, 2026
d772569
vulkan: replace std::aligned_alloc with posix_memalign for termux com…
bill88t Oct 22, 2026
d81d0d8
Multi-column on Intel XMX matrix engines refactored (#29864)
budiony Oct 22, 2026
d9ee007
model : fix qwen4exp layer inputs for DFlash drafters (#30927)
ggerganov Oct 22, 2026
5c19a5a
glm5 : optimize inputs (#30932)
ggerganov Oct 22, 2026
36cbf41
Fix !30793
gaugarg-nv Oct 22, 2026
19ab5a5
clef : include <algorithm> for std::find and std::copy (#30981)
kleinbem Oct 22, 2026
a08ab75
mtmd : add MiniCPM-OCR 1.0 support (#30979)
tc-mb Oct 22, 2026
970b51c
opencl: add depthwise conv2d (#30900)
kurquhar Oct 22, 2026
d814cbb
server: realtime API: send complete after committed audio drains (#30…
ngxson Oct 22, 2026
184f653
server: Substitute rerank placeholder from the original template only…
Pengkun-ZHU Oct 22, 2026
7377bad
sycl: Add graph record and replay (#28725)
malsbat Oct 23, 2026
0a9ef00
sycl: Replace host waits in fattn-mkl with explicit dependencies (#30…
malsbat Oct 23, 2026
643b0cd
hexagon: Op Fusion refactoring, optimizations and new fusions like `B…
max-krasnyansky Oct 23, 2026
f9599d0
vocab : fix Nemotron-3.5-Embed tokenization (#30974)
tokinasin Oct 23, 2026
1ab09d6
Extend probabilistic sampling for DFlash2 drafts (#30663)
praneshgo Oct 23, 2026
f64a498
sycl : pass device into mul_mat_uses_onednn (#31014)
Titaniumtown Oct 23, 2026
f5027fb
CANN: missing `const` qualifiers added for consistency (#30933)
rauletorresc Oct 23, 2026
f00c908
ci : add behavior check workflow (#31036)
CISC Oct 23, 2026
4231b23
ggml : fix time overflow in ggml_time_us (#30969)
frozenblade1224 Oct 23, 2026
e1e3b2c
ui : bump packages to clear npm audit (#31038)
allozaur Oct 23, 2026
36e36e6
hexagon: add CEIL op (#31006)
max-krasnyansky Oct 23, 2026
280141e
CUDA: chunk graphs to amortize graph launch (#30804)
am17an Oct 23, 2026
da2ee1e
vendor : update LibreSSL to 4.4.0 [no ci] (#31049)
angt Oct 23, 2026
5c31142
qwen4exp : build the QSA selection without the kq_mask (#30965)
sanmai Oct 23, 2026
06230cb
allows the usage of 4-10 gpus for stepfun and laguna in tensor paral…
krampenschiesser Oct 23, 2026
9ff0d7e
CUDA: fix build after chunked CUDA graphs (#30804) (#31053)
sreeram-11 Oct 23, 2026
1a24df1
ggm-cuda: Enforce DP4A MMQ tile height multiple of warp size (#30837)
thelittlefireman Oct 23, 2026
9145ddb
hip: fix bf16 WMMA operand vector type for gfx11/gfx12 (#30885)
waltercool Oct 23, 2026
21ec307
common : allow concurrent downloads across processes (#29312)
angt Oct 23, 2026
0ae0e19
hexagon: add SGN op (#30346)
kurquhar Oct 23, 2026
9ae73e6
mtmd : add Soprano TTS support (#28607)
hans00 Oct 23, 2026
edc3538
mtmd : lfm2 resize matches torchvision (#30943)
ykhrustalev Oct 24, 2026
7894e8b
mtmd: move build_inp_temporal_merge to base class (#31065)
ngxson Oct 24, 2026
1ddffe9
HIP: add byte permute fallback when __builtin_amdgcn_perm is unavaila…
aobolensk Oct 24, 2026
8e5a58d
vulkan: ggml src1 precision support (#30656)
0cc4m Oct 24, 2026
6b9ba4f
ci : fix data race between runners sharing the model cache (#31046)
ggerganov Oct 24, 2026
3642238
tests : add speculative rejection draft sampling test (#30213)
danbev Oct 24, 2026
701fd25
ggml : allow MUL_MAT_ID -1 index to skip the computation (#26631)
ngxson Oct 24, 2026
3032209
llama : deny -sm tensor for arches with moe amax rescaling (#31081)
ggerganov Oct 24, 2026
4d32b29
ci : exclude test-speculative-rejection-sampling from macos-latest-ar…
ggerganov Oct 24, 2026
df9beac
tests : fix test-llama-context for large n_batch (#31083)
sanmai Oct 24, 2026
73adf89
minimax-m3 : forward quant scales to the graph (fixes #31063) (#31097)
origin-labs-ai Oct 24, 2026
71255fe
ci : pass per-runner results dir to self-hosted jobs on T4 runners (#…
ggerganov Oct 24, 2026
ea1d7d6
fix(ui): align hover card and context popup styling with dropdown men…
allozaur Oct 24, 2026
e168e04
ui: Models Manager as Route + UI improvements (#30988)
allozaur Oct 24, 2026
9a0fde9
rpc : add bounds check in rdma_recv (#30399)
sohaibharraoui Oct 24, 2026
3f03b17
metal : remove mul_mm_id amax rescaling (#30924)
ggerganov Oct 24, 2026
f3269ef
server: refactor server entrypoint (#31113)
ngxson Oct 24, 2026
78a8ae1
hexagon: KV-head team split for multicore HMX flash attention (#31055)
ebateni Oct 24, 2026
d327bf1
jinja : fix select and selectattr argument handling (#31059)
JoeyPatricio Oct 24, 2026
74bca39
model: DSV4.1 flash (#31086)
am17an Oct 25, 2026
b3fa699
chat : reject JSON nested more than 128 levels before walking it (#30…
ykhrustalev Oct 25, 2026
344be0a
ci : update CPU job assignments (#31136)
ggerganov Oct 25, 2026
b083c38
opencl: add initial q2_k support (#30999)
lhez Oct 25, 2026
9a4580b
opencl: add topk and lightning indexer (#31026)
lhez Oct 25, 2026
f0fb598
hexagon: generic dims collapse/reshape for unary, binary and matmul o…
max-krasnyansky Oct 25, 2026
459a1ba
vulkan : one warp tile per subgroup in the small matmul tiles (#30106)
ianloic Oct 25, 2026
e321bda
vulkan: Add a larger tile size for conv3d (coopmat2 only) (#29920)
jeffbolznv Oct 25, 2026
a55c16e
ggml-cuda: fuse qwen MRoPE + set-rows (#31025)
am17an Oct 25, 2026
c49edf0
CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/u…
anujj Oct 25, 2026
7790a92
tools : reject truncated GGUF tensor data in merge (#31044)
shreya-11733 Oct 25, 2026
fa3645f
llama : respect no_alloc in all memory caches (#31087)
ggerganov Oct 25, 2026
f1168ac
grammar: limit recursion depth to prevent stack overflow (#29495) (#2…
Hrushikesh-ramilla Oct 25, 2026
5c965fd
llama-quant : fix 4D tensor quantization using ggml_nrows() and name-…
anthony-wss Oct 25, 2026
63412d8
llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31…
trunk-trick Oct 25, 2026
1a0b56c
ci : disable test-llama-context with fa off in the webgpu (#30977)
yomaytk Oct 25, 2026
2280f0f
qwen: merge tensors on convert (#31028)
am17an Oct 25, 2026
6050b46
tests : adjust default threads for test-llama-context (#31157)
ggerganov Oct 25, 2026
48c9f08
common, grammar: add recursive depth limit, make it a requirement (#3…
ngxson Oct 25, 2026
d854d98
cuda : use std::string instead of VLA for archName (#30463)
angt Oct 25, 2026
a074215
deepseekv4.1: follow-up review fixes (#31137)
am17an Oct 26, 2026
48c3611
ci : drop the `macos-latest-x64` job (#31175)
ggerganov Oct 26, 2026
224d94c
ci : bump server timeout during startup (#31174)
ggerganov Oct 26, 2026
69b3cc4
ui: Sidebar resizing improvements (#31119)
allozaur Oct 26, 2026
dc9e3be
ui: UX improvements (#31030)
allozaur Oct 26, 2026
9b8446b
metal : stop overflow from happening in large matrix output offsets (…
feni6 Oct 26, 2026
9125a2e
ci : fix race in webgpu memset (#31162)
eapache Oct 26, 2026
fd8f845
tests : refactor test-llama-archs to not link internal libllama symbo…
ggerganov Oct 26, 2026
8b28c23
ggml-cpu : fix unused variables in arm arch (#31180)
CISC Oct 26, 2026
c6fb3b5
mtmd: add mtmd_helper_get_capabilities (#31183)
ngxson Oct 26, 2026
74b104c
common: pin model resolution on real HF repo layouts (#30548)
ServeurpersoCom Oct 26, 2026
5320c87
sycl : bound reorder temp memory to 32 MiB (#31062)
KyleHagy Oct 26, 2026
02269b4
sycl: add set_tensor_2d for faster tensor-split loading (#30358)
Captain-Tripps Oct 26, 2026
38645c2
sycl: template the per-quant concat impls (#31008)
Titaniumtown Oct 26, 2026
5cd62ff
sycl: template the plain MMVQ launch triples (#31013)
Titaniumtown Oct 26, 2026
70c5f26
vulkan: rms norm optimization using subgroup reductions (#29882)
virajwad Oct 26, 2026
4d71b42
ggml-cpu : rewrite concat as one row kernel for all types (#31179)
ServeurpersoCom Oct 26, 2026
4b851fb
models : replace ggml_tensor** out-params with output structs (#31142)
ggerganov Oct 26, 2026
a469940
llama : move no_alloc out of llama_hparams (#31141)
ggerganov Oct 26, 2026
caa8191
hexagon: MUL_MAT_ID row skipping optimizations and improvements (#31169)
max-krasnyansky Oct 26, 2026
63b4b30
opencl: write KV-cache views in set_tensor so a state restore is not …
wanghqc Oct 26, 2026
4244ad7
ci : add windows behavior check (#31191)
CISC Oct 26, 2026
961c376
llama : enable moe amax rescaling for laguna (#31147)
begcdn Oct 26, 2026
7aca0d2
hexagon: add GGML_HEXAGON_VTCM_SIZE (#31221)
ebateni Oct 26, 2026
f5892ca
ui: use shared confirmation dialog for conversation deletion (#31187)
allozaur Oct 27, 2026
bc262f3
ui: Open model details in a drawer without leaving the page (#31176)
allozaur Oct 27, 2026
1a97e38
ui: add conversation compaction (#27401)
ServeurpersoCom Oct 27, 2026
823df7d
ui: Highlight reasoning toggle on hover and align it with model selec…
allozaur Oct 27, 2026
a2b3e12
opencl: initial support for q3_k (#31172)
lhez Oct 27, 2026
d930fe3
opencl: refine mul mat id -1 index (#31170)
lhez Oct 27, 2026
0d5ca60
ui: list org-less models first in the model selector (#31215)
allozaur Oct 27, 2026
7a2e5ab
llama : remove obsolete n_pad from memory instances (#31200)
ggerganov Oct 27, 2026
6666a73
tests : fix multi-seq split replay and partial-state cases for traine…
ggerganov Oct 27, 2026
de5122d
ggml : relax scale to non-contiguous views (#31146)
ggerganov Oct 27, 2026
a8276b7
ci : force python 3.12 on ubuntu 26.04 (#31222)
CISC Oct 27, 2026
fae9ac5
ci : use ccache depend mode for ROCm release builds (#31231)
sreeram-11 Oct 27, 2026
d78f40d
glm5-next : QSA without kq_mask (+ DSA) (#31009)
sanmai Oct 27, 2026
2763785
metal : add GGML_OP_TOP_K_MASK (#31230)
ggerganov Oct 27, 2026
7eed6b7
model: Add MiniCPM-OCR-Jev (#30980)
tc-mb Oct 27, 2026
74a1fd3
common : --model or -hf is required (#31242)
julien-c Oct 27, 2026
85cc432
server: add router device memory margin parameter for dynamic unloadi…
0cc4m Oct 27, 2026
3ab37ae
ui: fix system message row stretching to full prompt height in WebKit…
allozaur Oct 27, 2026
0c8c59e
mtmd-cli: Reduced scope of batch pointer (#31155) (#31165)
SamMalayek Oct 27, 2026
922f53e
vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT (#31239)
ServeurpersoCom Oct 27, 2026
8542e94
tests : speed up tensor data generation and model saving (#31145)
ggerganov Oct 27, 2026
7ecb57f
rpc : catch std::length_error in recv_msg (#31204)
sohaibharraoui Oct 27, 2026
27b7a8a
llama : build the k-pool sparse selection independently of the kq_mas…
am17an Oct 27, 2026
e86f5fd
cmake: WASI doesn't support spawning (#30751)
MendyBerger Oct 27, 2026
19cce69
model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252)
tdakhran Oct 27, 2026
6c44591
snapdragon: switch QDC artifact upload and log download to v2 API (#3…
mengshengwu Oct 27, 2026
eb7818c
common : support deferred tools and tool_reference content parts (#31…
tdakhran Oct 27, 2026
19ff817
model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257)
tdakhran Oct 27, 2026
338fce6
HIP/CUDA: remove pragma unroll from loops in conv2d where itteration …
IMbackK Oct 27, 2026
a919179
tests: fix CI riscv timeout in test-llama-context (#31265)
ServeurpersoCom Oct 27, 2026
1b4554e
server: expose the reasoning effort levels of the chat template (#31271)
ServeurpersoCom Oct 27, 2026
7518f30
Merge upstream ggml-org/llama.cpp master into ra/main.
Siddhesh2377 Oct 27, 2026
0ead4a8
llama: score Liquid d1-2B and d1-omni-2B on the LFM3 graph
Siddhesh2377 Oct 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 3 additions & 2 deletions .devops/cann.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ ENTRYPOINT ["/app/tools.sh"]
# ==============================================================================
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

ENTRYPOINT [ "/app/llama-cli" ]

Expand All @@ -155,8 +155,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

HEALTHCHECK --interval=5m CMD [ "curl", "-f", "http://localhost:8080/health" ]

Expand Down
5 changes: 3 additions & 2 deletions .devops/cpu.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -114,8 +114,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
5 changes: 3 additions & 2 deletions .devops/cuda.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -123,8 +123,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
16 changes: 11 additions & 5 deletions .devops/intel.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ RUN npm ci
COPY tools/ui/ ./
RUN LLAMA_BUILD_NUMBER="$APP_VERSION" npm run build

FROM docker.io/intel/oneapi-toolkit:$ONEAPI_VERSION AS build
FROM docker.io/intel/deep-learning-essentials:$ONEAPI_VERSION AS build

RUN echo "Download oneDNN installation package" && \
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/94c3dbac-0852-45be-a57d-21c204cada3e/intel-onednn-2026.0.2.46_offline.sh -O intel-onednn_offline.sh && \
echo "Install oneDNN components" && \
bash intel-onednn_offline.sh -s -a --silent --eula accept

ARG GGML_SYCL_F16=ON
ARG LEVEL_ZERO_VERSION=1.28.2
Expand All @@ -45,7 +50,7 @@ RUN if [ "${GGML_SYCL_F16}" = "ON" ]; then \
fi && \
echo "Building with dynamic libs" && \
cmake -B build -DGGML_NATIVE=OFF -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DLLAMA_BUILD_TESTS=OFF ${OPT_SYCL_F16} && \
cmake --build build --config Release -j$(nproc)
cmake --build build --config Release -j$((($(nproc)+1)/2))

RUN mkdir -p /app/lib && \
find build -name "*.so*" -exec cp -P {} /app/lib \;
Expand Down Expand Up @@ -102,7 +107,7 @@ RUN mkdir /tmp/neo/ && cd /tmp/neo/ \
&& dpkg --install *.deb

RUN apt-get update \
&& apt-get install -y libgomp1 curl ffmpeg \
&& apt-get install -y libgomp1 curl ffmpeg openssl libssl-dev \
&& apt autoremove -y \
&& apt clean -y \
&& rm -rf /tmp/* /var/tmp/* \
Expand Down Expand Up @@ -141,7 +146,7 @@ ENTRYPOINT ["/app/tools.sh"]
FROM base AS light

COPY --from=build /app/lib/ /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -151,9 +156,10 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/lib/ /app
COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
5 changes: 3 additions & 2 deletions .devops/musa.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -130,8 +130,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
1 change: 1 addition & 0 deletions .devops/openvino.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app/

Expand Down
5 changes: 3 additions & 2 deletions .devops/rocm.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -136,8 +136,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
1 change: 1 addition & 0 deletions .devops/s390x.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ ENTRYPOINT [ "/llama.cpp/bin/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

WORKDIR /llama.cpp/bin

Expand Down
5 changes: 3 additions & 2 deletions .devops/vulkan.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -117,8 +117,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
5 changes: 3 additions & 2 deletions .devops/zendnn.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ ENTRYPOINT ["/app/tools.sh"]
### Light, CLI only
FROM base AS light

COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app
COPY --from=build /app/full/llama /app/full/llama-cli /app/full/llama-completion /app/

WORKDIR /app

Expand All @@ -107,8 +107,9 @@ ENTRYPOINT [ "/app/llama-cli" ]
FROM base AS server

ENV LLAMA_ARG_HOST=0.0.0.0
ENV LLAMA_ARG_PORT=8080

COPY --from=build /app/full/llama /app/full/llama-server /app
COPY --from=build /app/full/llama /app/full/llama-server /app/

WORKDIR /app

Expand Down
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ insert_final_newline = unset
trim_trailing_whitespace = unset
insert_final_newline = unset

[vendor/**.patch]
trim_trailing_whitespace = unset

[tools/ui/**]
indent_style = unset
indent_size = unset
Expand Down
2 changes: 2 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ exclude =
# Do not traverse examples and tools
examples,
tools,
# Do not traverse .venv directories
.venv*,
# Do not include package initializers
__init__.py,
# No need to traverse our git directory
Expand Down
158 changes: 158 additions & 0 deletions .github/workflows/behavior-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
name: Behavior Check

on:
workflow_dispatch: # allows manual triggering
push:
branches:
- master
paths: [
'.github/workflows/behavior-check.yml',
'**/CMakeLists.txt',
'**/.cmake',
'**/*.h',
'**/*.hpp',
'**/*.c',
'**/*.cpp',
]

pull_request:
types: [opened, synchronize, reopened]
paths: [
'.github/workflows/behavior-check.yml',
'**/CMakeLists.txt',
'**/.cmake',
'**/*.h',
'**/*.hpp',
'**/*.c',
'**/*.cpp'
]

cache-mode: none
permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true

env:
GGML_NLOOP: 3
GGML_N_THREADS: 1
GGML_SCHED_DEBUG_REALLOC: 1
LLAMA_ARG_LOG_COLORS: 1
LLAMA_ARG_LOG_PREFIX: 1
LLAMA_ARG_LOG_TIMESTAMPS: 1

jobs:
ubuntu:
runs-on: ubuntu-26.04-arm

steps:
- name: Clone
id: checkout
uses: actions/checkout@v6

- name: ccache
uses: ggml-org/ccache-action@v1.2.24
with:
restore: false
save: false

- name: Build Dependencies
id: build_depends
run: |
sudo apt update
sudo apt install -y build-essential libssl-dev git-lfs libjpeg-dev python3 python3-venv python3-pip
- name: Python Dependencies
id: python_depends
run: |
python3 -m venv .venv
source .venv/bin/activate
pip install jinja2==3.1.6
- name: ccache-buckets-restore
uses: ./.github/actions/ccache-buckets
env:
HF_TOKEN: ${{ secrets.HF_TOKEN_CI }}
with:
key: behavior-check-ubuntu-26.04-arm
folder: llama.cpp
hf_bucket: ggml-org/cache

- name: Build
id: cmake_build
run: |
cmake -B build \
-DGGML_NATIVE=OFF \
-DGGML_BACKEND_DL=OFF \
-DLLAMA_FATAL_WARNINGS=ON
time cmake --build build --config Release -j $(nproc)
- name: ccache-buckets-save
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
uses: ./.github/actions/ccache-buckets
env:
HF_TOKEN: ${{ secrets.HF_TOKEN_CACHE_OUTPUT }}
with:
key: behavior-check-ubuntu-26.04-arm
folder: llama.cpp
evict-old-files: 1d
hf_bucket: ggml-org/cache
save: true

- name: Test
id: cmake_test
run: |
source .venv/bin/activate
cd build
ctest -L behavior -LE 'model|backend' --verbose --timeout 900
windows:
runs-on: windows-2025

cache-mode: write
permissions:
actions: write
contents: read

steps:
- name: Clone
id: checkout
uses: actions/checkout@v6

- name: ccache
uses: ggml-org/ccache-action@v1.2.24
with:
key: behavior-check-windows-2025
evict-old-files: 1d
save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}

- name: Python Dependencies
id: python_depends
shell: bash
run: |
python3 -m venv .venv
source .venv/Scripts/activate
pip install jinja2==3.1.6
- name: Build
id: cmake_build
shell: cmd
run: |
call "C:\Program Files\Microsoft Visual Studio\18\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
cmake -B build -G "Ninja Multi-Config" ^
-DCMAKE_TOOLCHAIN_FILE=cmake/x64-windows-llvm.cmake ^
-DCMAKE_BUILD_TYPE=Release ^
-DGGML_NATIVE=OFF ^
-DGGML_BACKEND_DL=OFF
set /A NINJA_JOBS=%NUMBER_OF_PROCESSORS%-1
cmake --build build --config Release -j %NINJA_JOBS%
- name: Test
id: cmake_test
shell: bash
run: |
source .venv/Scripts/activate
cd build
ctest -C Release -L behavior -LE 'model|backend' --verbose --timeout 900
Loading
Loading