Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 28
Like-for-like pair: 1
Runtimes available: 2

Compare

Side A

Side B

Model

Chip

Quantisation A

Quantisation B

Backend

Prefill size

Decode size

Cooldown

14 runs vs 14 runs · 1 like-for-like pair

Held fixedModel: Qwen3 30B A3B Instruct 2507Prefill: PP512Decode: TG128

The sides differ in conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the runtime.

Runtime comparison

BaseRT vs llama.cpp

Ratios read side A relative to side B. Geometric mean over 1 like-for-like pair.

Decode: 1.03×+3.1%
Prefill: 1.97×+97.4%

Side A

BaseRT

14 runs

2 contributors

Latest 19 Oct 2026

Decode · TG128: 95.6tok/s median
best 139.8 · worst 49.4
Prefill · PP512: 4,198tok/s median
best 7,439 · worst 901

Models: Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Instruct-2507-cuda-q4mix, Qwen3-30B-A3B-Instruct-2507-cuda-q8, Qwen3-30B-A3B-Instruct-2507-Q4
Chips: Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Pro, NVIDIA GB10, unknown
Versions: BaseRT 0.2.4, BaseRT 0.2.6, BaseRT 0.3.0
Quantisations: Q4
Backends: CUDA, Metal
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

Side B

llama.cpp

14 runs

1 contributor

Latest 11 Sep 2026

Decode · TG128: 128.5tok/s median
best 198.1 · worst 78.9
Prefill · PP512: 2,430tok/s median
best 2,868 · worst 1,644

Models: Qwen3-30B-A3B-Instruct-2507
Chips: AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro
Versions: llama.cpp b10809 (5266f24da)
Quantisations: Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K
Backends: BLAS + Metal, ROCm, Vulkan
Cooldown: off
Conditioning: runtime_native_warmup
Decode workload: TG128
Harness schema: computearena-measurements/1

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Instruct-2507-cuda-q4mix, Qwen3-30B-A3B-Instruct-2507-cuda-q8, Qwen3-30B-A3B-Instruct-2507-Q4	Qwen3-30B-A3B-Instruct-2507
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Pro, NVIDIA GB10, unknown	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro
Backend	CUDA, Metal	BLAS + Metal, ROCm, Vulkan
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read BaseRT relative to llama.cpp.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3-30B-A3B-Instruct-2507 Apple M5 Pro	98.1	95.1	1.03×	3,493	1,769	1.97×	2 / 6

Side A · BaseRT

Top 14 of 14 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	139.8 TG128 @ 1 ctx	1,789 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	Apple M3 Ultra Metal	139.2 TG128 @ 1 ctx	2,315 PP512	basecompute	19 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	137.1 TG128 @ 1 ctx	2,287 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3-30B-A3B-Instruct-2507 BaseRT 0.2.4Q4	Apple M5 Pro Metal	104.0 TG128	3,680 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	unknown CUDA	103.7 TG128 @ 1 ctx	4,780 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q4mix BaseRT 0.3.0Q4	NVIDIA GB10 CUDA	99.1 TG128 @ 1 ctx	7,437 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q4mix BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	99.0 TG128 @ 1 ctx	7,439 PP512	basecompute	19 Sep 2026	View Benchmark
basecompute/Qwen3-30B-A3B-Instruct-2507 BaseRT 0.2.4Q4	Apple M5 Pro Metal	92.2 TG128	3,305 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	87.7 TG128 @ 1 ctx	901 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	Apple M1 Max Metal	87.5 TG128 @ 1 ctx	902 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	80.7 TG128 @ 1 ctx	4,715 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	61.2 TG128 @ 1 ctx	7,253 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRT 0.3.0Q4	unknown CUDA	49.5 TG128 @ 1 ctx	7,382 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	49.4 TG128 @ 1 ctx	7,315 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 14 of 14 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	198.1 TG128	2,837 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	186.7 TG128	2,868 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	184.5 TG128	2,474 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	181.3 TG128	2,827 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT ROCm	137.4 TG128	2,386 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT ROCm	128.8 TG128	2,596 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	128.6 TG128	2,661 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	128.5 TG128	2,820 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q2_K	Apple M5 Pro BLAS + Metal	102.7 TG128	1,893 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q3_K_M	Apple M5 Pro BLAS + Metal	97.3 TG128	1,708 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	95.2 TG128	1,845 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	95.0 TG128	1,831 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q5_K_M	Apple M5 Pro BLAS + Metal	84.7 TG128	1,644 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q6_K	Apple M5 Pro BLAS + Metal	78.9 TG128	1,646 PP512	arki05	11 Sep 2026	View Benchmark

Compare

Loading community benchmarks…

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Instruct-2507-cuda-q4mix, Qwen3-30B-A3B-Instruct-2507-cuda-q8, Qwen3-30B-A3B-Instruct-2507-Q4	Qwen3-30B-A3B-Instruct-2507
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M5 Pro, NVIDIA GB10, unknown	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro
Backend	CUDA, Metal	BLAS + Metal, ROCm, Vulkan
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Configuration

Decode A

Decode B

Ratio

Prefill A

Prefill B

Ratio

Runs A / B

Qwen3-30B-A3B-Instruct-2507

Apple M5 Pro

98.1

95.1

1.03×

3,493

1,769

1.97×

2 / 6

Side A · BaseRT

Top 14 of 14 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	139.8 TG128 @ 1 ctx	1,789 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	Apple M3 Ultra Metal	139.2 TG128 @ 1 ctx	2,315 PP512	basecompute	19 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	137.1 TG128 @ 1 ctx	2,287 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Qwen3-30B-A3B-Instruct-2507 BaseRT 0.2.4Q4	Apple M5 Pro Metal	104.0 TG128	3,680 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	unknown CUDA	103.7 TG128 @ 1 ctx	4,780 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q4mix BaseRT 0.3.0Q4	NVIDIA GB10 CUDA	99.1 TG128 @ 1 ctx	7,437 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q4mix BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	99.0 TG128 @ 1 ctx	7,439 PP512	basecompute	19 Sep 2026	View Benchmark
basecompute/Qwen3-30B-A3B-Instruct-2507 BaseRT 0.2.4Q4	Apple M5 Pro Metal	92.2 TG128	3,305 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	87.7 TG128 @ 1 ctx	901 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	Apple M1 Max Metal	87.5 TG128 @ 1 ctx	902 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	80.7 TG128 @ 1 ctx	4,715 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	61.2 TG128 @ 1 ctx	7,253 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRT 0.3.0Q4	unknown CUDA	49.5 TG128 @ 1 ctx	7,382 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	49.4 TG128 @ 1 ctx	7,315 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 14 of 14 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	198.1 TG128	2,837 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	186.7 TG128	2,868 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	184.5 TG128	2,474 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	181.3 TG128	2,827 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q2_K	AMD Radeon RX 7900 XT ROCm	137.4 TG128	2,386 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q3_K_M	AMD Radeon RX 7900 XT ROCm	128.8 TG128	2,596 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	128.6 TG128	2,661 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	128.5 TG128	2,820 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q2_K	Apple M5 Pro BLAS + Metal	102.7 TG128	1,893 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q3_K_M	Apple M5 Pro BLAS + Metal	97.3 TG128	1,708 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	95.2 TG128	1,845 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	95.0 TG128	1,831 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q5_K_M	Apple M5 Pro BLAS + Metal	84.7 TG128	1,644 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507 llama.cpp b10809 (5266f24da)Q6_K	Apple M5 Pro BLAS + Metal	78.9 TG128	1,646 PP512	arki05	11 Sep 2026	View Benchmark