Compare

Loading community benchmarks…

Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 20
Like-for-like pair: 1
Runtimes available: 2

Compare

Side A

Side B

Model

Chip

Quantisation A

Quantisation B

Backend

Prefill size

Decode size

Cooldown

16 runs vs 4 runs · 1 like-for-like pair

Held fixedModel: Qwen3.8 27BPrefill: PP512Decode: TG128

The sides differ in conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the runtime.

Runtime comparison

BaseRT vs llama.cpp

Ratios read side A relative to side B. Geometric mean over 1 like-for-like pair.

Decode: 1.38×+37.8%
Prefill: 0.91×-8.7%

Side A

BaseRT

16 runs

4 contributors

Latest 20 Sep 2026

Decode · TG128: 18.9tok/s median
best 38.0 · worst 7.5
Prefill · PP512: 472tok/s median
best 1,152 · worst 82

Models: Qwen3.8-27B, Qwen3.8-27B-Q4, Qwen3.8-27B-Q8
Chips: Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Max, Apple M5 Pro, NVIDIA GB10
Versions: BaseRT 0.2.4, BaseRT 0.2.6
Quantisations: Q4
Backends: CUDA, Metal
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

Side B

llama.cpp

4 runs

3 contributors

Latest 17 Sep 2026

Decode · TG128: 47.1tok/s median
best 95.3 · worst 22.9
Prefill · PP512: 1,021tok/s median
best 2,513 · worst 243

Models: Qwen3.8-27B
Chips: Apple M4 Max, NVIDIA H100 80GB HBM3 × 8, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Versions: llama.cpp b1 (e64c0ea), llama.cpp b10809 (5266f24da), llama.cpp b10902 (df03399b8)
Quantisations: Q4_0, Q4_1, Q4_K_M
Backends: BLAS + Metal, CUDA
Cooldown: off
Conditioning: runtime_native_warmup
Decode workload: TG128
Harness schema: computearena-measurements/1

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	Qwen3.8-27B, Qwen3.8-27B-Q4, Qwen3.8-27B-Q8	Qwen3.8-27B
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Max, Apple M5 Pro, NVIDIA GB10	Apple M4 Max, NVIDIA H100 80GB HBM3 × 8, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Backend	CUDA, Metal	BLAS + Metal, CUDA
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read BaseRT relative to llama.cpp.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3.8-27B Apple M4 Max	31.6	22.9	1.38×	222	243	0.91×	1 / 1

Side A · BaseRT

Top 16 of 16 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3.8-27B-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	38.0 TG128 @ 1 ctx	282 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Max Metal	34.0 TG128	589 PP512	basecompute	10 Sep 2026	View Benchmark
Qwen3.8-27B-Q4 BaseRT 0.2.4Q4	Apple M4 Max Metal	31.6 TG128	222 PP512	fabian	10 Sep 2026	View Benchmark
Qwen3.8-27B-Q8 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	22.1 TG128 @ 1 ctx	280 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Max Metal	21.5 TG128	590 PP512	lukas	15 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Max Metal	21.5 TG128	594 PP512	lukas	15 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Max Metal	20.7 TG128	599 PP512	lukas	15 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Max Metal	19.7 TG128	589 PP512	lukas	10 Sep 2026	View Benchmark
Qwen3.8-27B-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	18.2 TG128 @ 1 ctx	82 PP512	basecompute	19 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Pro Metal	16.5 TG128	327 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.8-27B-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	13.1 TG128 @ 1 ctx	1,131 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3.8-27B-Q8 BaseRT 0.2.6Q4	Apple M1 Max Metal	11.0 TG128 @ 1 ctx	82 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3.8-27B-Q8 BaseRT 0.2.6Q4	Apple M1 Max Metal	11.0 TG128 @ 1 ctx	82 PP512	basecompute	19 Sep 2026	View Benchmark
basecompute/Qwen3.8-27B BaseRT 0.2.4Q4	Apple M5 Pro Metal	10.6 TG128	355 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3.8-27B-Q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	7.6 TG128 @ 1 ctx	1,152 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3.8-27B-Q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	7.5 TG128 @ 1 ctx	1,127 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 4 of 4 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3.8-27B llama.cpp b10902 (df03399b8)Q4_1	NVIDIA H100 80GB HBM3 × 8 CUDA	95.3 TG128	2,513 PP512	sarthak247	11 Sep 2026	View Benchmark
Qwen3.8-27B llama.cpp b1 (e64c0ea)Q4_0	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	48.1 TG128	910 PP512	arki05	17 Sep 2026	View Benchmark
Qwen3.8-27B llama.cpp b1 (e64c0ea)Q4_K_M	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	46.0 TG128	1,132 PP512	arki05	17 Sep 2026	View Benchmark
Qwen3.8-27B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M4 Max BLAS + Metal	22.9 TG128	243 PP512	fabian	11 Sep 2026	View Benchmark