ggml-cpu: unroll the AVX2 q4_K x q8_K sub-block loop

[codspeed-hq]

What

ggml_vec_dot_q4_K_q8_K is the kernel behind every Q4_K matmul. On AVX2 its sub-block loop

for (int j = 0; j < QK_K/64; ++j) { ... }

has a compile-time trip count of 4, but GCC keeps it rolled at -O2. Two sources of pure
overhead follow from that:

1. Every iteration pays a pointer bump, a compare and a branch, and all the q4/q8 loads go
   through a scaled index register instead of an immediate offset.
2. The single serial accumulator (sumj = p16l + p16h; sumi += sumj) forces an extra vmovdqa
   register copy per iteration.

How

The four sub-block pairs are written out explicitly (guarded by a static_assert(QK_K == 256)),
so the q4/q8 offsets become immediates inside the addressing modes, and each nibble half gets
its own int32 accumulator; the two accumulators are added together once per superblock.

The arithmetic is unchanged: the same vpmaddubsw/vpmaddwd products are summed, only the order
of the exact int32 additions differs.

Only the AVX2 path in ggml/src/ggml-cpu/arch/x86/quants.c is touched. Other architectures and
the scalar/AVX paths are untouched.

Correctness

• tests/test-quantize-fns passes (AVX2 build, GGML_NATIVE=OFF -DGGML_AVX2=ON).
• Results are bit-identical to the unmodified build: an FNV-1a hash of the raw ggml_mul_mat
  output bytes was compared between a baseline build and this build over 20 cases
  (K in {256, 2048, 5632}, N in {64...2048}, M in {1, 3, 8, 17, 64}, 2-D and batched 3-D, with
  1/2/8 threads). Every hash matches.

Measured impact

CodSpeed CPU simulation, macro suite, q4_k benchmarks (AVX2 baseline build, same machine):

Benchmark	Base	Head	Change
prompt_layer[q4_k]	801.1 ms	716.9 ms	-10.5%
lm_head[q4_k]	136.6 ms	122.7 ms	-10.2%
decode_deep[q4_k]	621.0 ms	594.5 ms	-4.3%
decode_layer[q4_k]	155.9 ms	158.1 ms	+1.4% (barrier bound, within noise)

The gain lands where the change predicts: on prompt_layer[q4_k] the instruction component
dominates (93% of the measured time) while the cache and memory components are unchanged. No
benchmark regresses meaningfully.

The x86 win is visible in the simulation benchmark job; the walltime macro job runs on aarch64
and is unaffected by this change.

[codspeed-hq]

Merging this PR will improve performance by 12.44%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 27 untouched benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	mul_mat[q4_k]	72.4 ms	64.4 ms	+12.44%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing codspeed-optim-unroll-the-avx2-ggml-vec-dot-q4-k-q8-k-sub-block-s-1785403756979 (44a67ae) with master (46819c9)}