Compare

Loading community benchmarks…

Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 24
Like-for-like pair: 1
Runtimes available: 2

Compare

Side A

Side B

Model

Chip

Quantisation A

Quantisation B

Backend

Prefill size

Decode size

Cooldown

13 runs vs 11 runs · 1 like-for-like pair

Held fixedModel: Qwen3.6 27BPrefill: PP512Decode: TG128

The sides differ in conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the runtime.

Runtime comparison

BaseRT vs llama.cpp

Ratios read side A relative to side B. Geometric mean over 1 like-for-like pair.

Decode: 0.90×-10.3%
Prefill: 0.96×-4.1%

Side A

BaseRT

13 runs

2 contributors

Latest 20 Sep 2026

Decode · TG128: 13.0tok/s median
best 36.8 · worst 7.5
Prefill · PP512: 355tok/s median
best 1,146 · worst 80

Models: Qwen3.6-27B, Qwen3.6-27B-cuda-q4mix, Qwen3.6-27B-cuda-q8, Qwen3.6-27B-Q4, Qwen3.6-27B-Q8
Chips: Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Max, Apple M5 Pro, NVIDIA GB10
Versions: BaseRT 0.2.4, BaseRT 0.2.6
Quantisations: Q4, Q8
Backends: CUDA, Metal
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

Side B

llama.cpp

11 runs

1 contributor

Latest 17 Sep 2026

Decode · TG128: 29.7tok/s median
best 48.0 · worst 11.9
Prefill · PP512: 782tok/s median
best 1,174 · worst 373

Models: Qwen3.6-27B
Chips: AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Versions: llama.cpp b1 (e64c0ea), llama.cpp b10809 (5266f24da)
Quantisations: Q3_K_M, Q4_0, Q4_K_M, Q6_K
Backends: BLAS + Metal, CUDA, ROCm, Vulkan
Cooldown: off
Conditioning: runtime_native_warmup
Decode workload: TG128
Harness schema: computearena-measurements/1

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	Qwen3.6-27B, Qwen3.6-27B-cuda-q4mix, Qwen3.6-27B-cuda-q8, Qwen3.6-27B-Q4, Qwen3.6-27B-Q8	Qwen3.6-27B
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Max, Apple M5 Pro, NVIDIA GB10	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Backend	CUDA, Metal	BLAS + Metal, CUDA, ROCm, Vulkan
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read BaseRT relative to llama.cpp.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3.6-27B Apple M5 Pro	13.3	14.9	0.90×	360	375	0.96×	2 / 4

Side A · BaseRT

Top 13 of 13 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3.6-27B-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	36.8 TG128 @ 1 ctx	281 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3.6-27B BaseRT 0.2.4Q4	Apple M5 Max Metal	31.3 TG128	603 PP512	basecompute	11 Sep 2026	View Benchmark
Qwen3.6-27B-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	30.3 TG128 @ 1 ctx	221 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3.6-27B-Q8 BaseRT 0.2.6Q8	Apple M3 Ultra Metal	22.1 TG128 @ 1 ctx	280 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3.6-27B-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	17.9 TG128 @ 1 ctx	80 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3.6-27B BaseRT 0.2.4Q4	Apple M5 Pro Metal	16.1 TG128	364 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	13.0 TG128 @ 1 ctx	344 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen/Qwen3.6-27B BaseRT 0.2.4Q4	NVIDIA GB10 CUDA	12.4 TG128	1,140 PP512	basecompute	11 Sep 2026	View Benchmark
Qwen3.6-27B-cuda-q4mix BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	12.3 TG128 @ 1 ctx	1,127 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3.6-27B-Q8 BaseRT 0.2.6Q8	Apple M1 Max Metal	11.0 TG128 @ 1 ctx	83 PP512	basecompute	19 Sep 2026	View Benchmark
basecompute/Qwen3.6-27B BaseRT 0.2.4Q8	Apple M5 Pro Metal	10.6 TG128	355 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B-cuda-q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	8.4 TG128 @ 1 ctx	1,146 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3.6-27B-cuda-q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	7.5 TG128 @ 1 ctx	1,141 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 11 of 11 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3.6-27B llama.cpp b1 (e64c0ea)Q4_0	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	48.0 TG128	1,174 PP512	arki05	17 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	36.8 TG128	739 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	33.7 TG128	782 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	32.5 TG128	783 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	29.8 TG128	841 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT ROCm	29.7 TG128	807 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	29.3 TG128	840 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q3_K_M	Apple M5 Pro BLAS + Metal	16.7 TG128	373 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	15.2 TG128	376 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	14.5 TG128	374 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.6-27B llama.cpp b10809 (5266f24da)Q6_K	Apple M5 Pro BLAS + Metal	11.9 TG128	377 PP512	arki05	11 Sep 2026	View Benchmark