Compare

Loading community benchmarks…

Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 65
Like-for-like pair: 1
Runtimes available: 2

Compare

Side A

Side B

Model

Chip

Quantisation A

Quantisation B

Backend

Prefill size

Decode size

Cooldown

43 runs vs 22 runs · 1 like-for-like pair

Held fixedModel: Qwen3 8BPrefill: PP512Decode: TG128

The sides differ in conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the runtime.

Runtime comparison

llama.cpp vs BaseRT

Ratios read side A relative to side B. Geometric mean over 1 like-for-like pair.

Decode: 0.84×-16.4%
Prefill: 0.81×-19.3%

Side A

llama.cpp

43 runs

1 contributor

Latest 17 Sep 2026

Decode · TG128: 84.0tok/s median
best 150.7 · worst 18.9
Prefill · PP512: 2,563tok/s median
best 3,241 · worst 1,168

Models: Qwen3-8B
Chips: AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Versions: llama.cpp b1 (e64c0ea), llama.cpp b10809 (5266f24da)
Quantisations: BF16, Q2_K, Q3_K_M, Q4_1, Q4_K_M, Q5_K_M, Q6_K, Q8_0
Backends: BLAS + Metal, CUDA, ROCm, Vulkan
Cooldown: off
Conditioning: runtime_native_warmup
Decode workload: TG128
Harness schema: computearena-measurements/1

Side B

BaseRT

22 runs

3 contributors

Latest 20 Sep 2026

Decode · TG128: 59.7tok/s median
best 113.6 · worst 31.6
Prefill · PP512: 1,727tok/s median
best 1,786 · worst 392

Models: Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3-8B-Q4, Qwen3-8B-Q8
Chips: Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Pro, NVIDIA GB10
Versions: BaseRT 0.2.4, BaseRT 0.2.5, BaseRT 0.2.6
Quantisations: Q2, Q3, Q4, Q5, Q6, Q8
Backends: CUDA, Metal
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

What differs besides the runtime

Facet	llama.cpp	BaseRT
Model	Qwen3-8B	Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3-8B-Q4, Qwen3-8B-Q8
Chip	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Pro, NVIDIA GB10
Backend	BLAS + Metal, CUDA, ROCm, Vulkan	CUDA, Metal
Conditioning	runtime_native_warmup	warmup_only
Decode workload	TG128	TG128, TG128 @ 1 ctx
Harness schema	computearena-measurements/1	basert-benchmark-harness/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read llama.cpp relative to BaseRT.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3-8B Apple M5 Pro	49.9	59.7	0.84×	1,416	1,755	0.81×	14 / 13

Side A · llama.cpp

Top 25 of 43 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-8B llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	150.7 TG128	2,528 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	146.2 TG128	2,563 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	130.2 TG128	2,459 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	127.3 TG128	2,498 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b1 (e64c0ea)Q4_K_M	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	124.7 TG128	3,087 PP512	arki05	17 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	118.5 TG128	2,625 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q4_1	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	115.6 TG128	2,902 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	115.3 TG128	2,643 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT ROCm	112.3 TG128	2,798 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q4_1	AMD Radeon RX 7900 XT ROCm	107.3 TG128	2,894 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q5_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	105.7 TG128	2,629 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT ROCm	105.5 TG128	2,834 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q5_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	104.9 TG128	2,640 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	95.6 TG128	2,430 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	93.7 TG128	3,110 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	93.2 TG128	3,076 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT ROCm	91.6 TG128	2,864 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT ROCm	91.6 TG128	2,911 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	88.6 TG128	2,497 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q5_K_M	AMD Radeon RX 7900 XT ROCm	87.5 TG128	2,837 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q5_K_M	AMD Radeon RX 7900 XT ROCm	86.8 TG128	2,869 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT ROCm	84.0 TG128	2,823 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT ROCm	80.1 TG128	2,903 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q8_0	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	77.6 TG128	2,669 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B llama.cpp b10809 (5266f24da)Q8_0	AMD Radeon RX 7900 XT ROCm	74.2 TG128	3,241 PP512	arki05	11 Sep 2026	View Benchmark

Side B · BaseRT

Top 22 of 22 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-8B-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	113.6 TG128 @ 1 ctx	1,357 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-8B-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	99.2 TG128 @ 1 ctx	943 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-8B-base-q2 BaseRT 0.2.4Q2	Apple M5 Pro Metal	81.8 TG128	1,786 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B-Q8 BaseRT 0.2.6Q8	Apple M3 Ultra Metal	72.5 TG128 @ 1 ctx	1,338 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-8B-base-q3 BaseRT 0.2.4Q3	Apple M5 Pro Metal	62.0 TG128	1,748 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q4	Apple M5 Pro Metal	60.7 TG128	1,755 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q4	Apple M5 Pro Metal	60.6 TG128	1,763 PP512	isu	10 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q4	Apple M5 Pro Metal	60.5 TG128	1,763 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	60.2 TG128 @ 1 ctx	507 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.5Q4	Apple M5 Pro Metal	59.7 TG128 @ 1 ctx	1,762 PP512	isu	18 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q4	Apple M5 Pro Metal	59.7 TG128	1,761 PP512	isu	11 Sep 2026	View Benchmark
Qwen3-8B-Q8 BaseRT 0.2.6Q8	Apple M4 Max Metal	59.6 TG128 @ 1 ctx	942 PP512	basecompute	19 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q4	Apple M5 Pro Metal	58.2 TG128	1,758 PP512	isu	11 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q4	Apple M5 Pro Metal	56.9 TG128	1,707 PP512	isu	10 Sep 2026	View Benchmark
Qwen3-8B-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	52.6 TG128 @ 1 ctx	394 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-8B-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	50.3 TG128 @ 1 ctx	1,452 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-8B-base-q5 BaseRT 0.2.4Q5	Apple M5 Pro Metal	48.4 TG128	1,730 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B-base-q6 BaseRT 0.2.4Q6	Apple M5 Pro Metal	42.1 TG128	1,723 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B-Q8 BaseRT 0.2.6Q8	Apple M1 Max Metal	38.1 TG128 @ 1 ctx	500 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q8	Apple M5 Pro Metal	35.2 TG128	1,748 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-8B BaseRT 0.2.4Q8	Apple M5 Pro Metal	34.5 TG128	1,744 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-8B-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	31.6 TG128 @ 1 ctx	392 PP512	basecompute	19 Sep 2026	View Benchmark