Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 170
Like-for-like pairs: 17
Chips available: 17

Compare

Side A

Side B

Model

Runtime

Runtime version

Quantisation

Backend

Prefill size

Decode size

Cooldown

41 runs vs 129 runs · 17 like-for-like pairs

Held fixedPrefill: PP512Decode: TG128

These figures mix different models. Pick a model for a like-for-like comparison.
A side mixes quantisations. Pick a quantisation to compare the same weights.
The sides differ in cooldown, conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the chip.

Chip comparison

unknown vs Apple M5 Pro

Ratios read side A relative to side B. Geometric mean over 17 like-for-like pairs.

Decode: 0.76×-24.4%
Prefill: 2.99×+199%

Side A

unknown

41 runs

1 contributor

Latest 14 Oct 2026

Decode · TG128: 74.8tok/s median
best 450.2 · worst 7.5
Prefill · PP512: 4,356tok/s median
best 51,746 · worst 385

Models: gemma-4-E2B-it-cuda-q4mix, gemma-4-E2B-it-Q4, gpt-oss-120b-MXFP4, gpt-oss-120b-Q4, gpt-oss-20b-Q4, Llama-3.2-1B-Instruct-cuda-q4mix, Llama-3.2-1B-Instruct-Q4, Llama-3.2-1B-Instruct-Q8, Llama-3.2-3B-Instruct-cuda-q4mix, Llama-3.2-3B-Instruct-cuda-q8, Mistral-7B-Instruct-v0.3-Q8, NVIDIA-Nemotron-3-Nano-30B-A3B-Q4, Qwen3-0.6B-cuda-q4, Qwen3-0.6B-cuda-q4mix, Qwen3-30B-A3B-Instruct-2507-cuda-q8, Qwen3-30B-A3B-Instruct-2507-Q4, Qwen3-4B-Instruct-2507-Q4, Qwen3-4B-Thinking-2507-Q8, Qwen3-8B-Q4, Qwen3.5-2B-Base-Q4, Qwen3.5-2B-cuda-q4mix, Qwen3.5-2B-cuda-q8, Qwen3.5-2B-Q4, Qwen3.5-35B-A3B-cuda-q4mix, Qwen3.5-35B-A3B-cuda-q8, Qwen3.5-35B-A3B-Q4, Qwen3.6-27B-cuda-q4mix, Qwen3.6-27B-cuda-q8, Qwen3.6-27B-Q4, Qwen3.6-35B-A3B-cuda-q4mix, Qwen3.6-35B-A3B-cuda-q8, Qwen3.6-35B-A3B-Q4, Qwen3.8-27B-Q4, Qwen3.8-27B-Q8
Runtimes: BaseRT
Versions: BaseRT 0.3.0
Quantisations: mxfp4, Q4, Q8
Backends: CUDA
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

Side B

Apple M5 Pro

129 runs

2 contributors

Latest 18 Sep 2026

Decode · TG128: 72.8tok/s median
best 2,068.0 · worst 10.6
Prefill · PP512: 1,852tok/s median
best 190,107 · worst 327

Models: gemma-3-1b-it, gemma-4-26B-A4B-it, gemma-4-E2B-it, gemma-4-E4B-it, Gemma-4-E4B-It, gpt-oss-20b, Gpt-Oss-20B, Llama 3.2 3B Instruct, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Muse-Glimmer-30B, NVIDIA-Nemotron-3-Nano-30B-A3B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-4B, Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-35B-A3B, Qwen3.6-27B, Qwen3.6-35B-A3B, Qwen3.8-27B, Tinystories Lay8 Hs512 Hd8 33M, tinystories-lay8-hs512-hd8-33M
Runtimes: BaseRTllama.cpp
Versions: BaseRT 0.2.4, BaseRT 0.2.5, llama.cpp b10809 (5266f24da), llama.cpp b9960 (a935fbffe)
Quantisations: BF16, F16, mxfp4, passthrough_gguf, Q2, Q2_K, Q3, Q3_K_M, Q4, Q4_0, Q4_1, Q4_K_M, Q5, Q5_K_M, Q6, Q6_K, Q8, Q8_0
Backends: BLAS + Metal, Metal
Cooldown: off, on
Conditioning: idle_reset_then_warm, runtime_native_warmup, warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1, computearena-measurements/1

What differs besides the chip

Facet	unknown	Apple M5 Pro
Model	gemma-4-E2B-it-cuda-q4mix, gemma-4-E2B-it-Q4, gpt-oss-120b-MXFP4, gpt-oss-120b-Q4, gpt-oss-20b-Q4, Llama-3.2-1B-Instruct-cuda-q4mix, Llama-3.2-1B-Instruct-Q4, Llama-3.2-1B-Instruct-Q8, Llama-3.2-3B-Instruct-cuda-q4mix, Llama-3.2-3B-Instruct-cuda-q8, Mistral-7B-Instruct-v0.3-Q8, NVIDIA-Nemotron-3-Nano-30B-A3B-Q4, Qwen3-0.6B-cuda-q4, Qwen3-0.6B-cuda-q4mix, Qwen3-30B-A3B-Instruct-2507-cuda-q8, Qwen3-30B-A3B-Instruct-2507-Q4, Qwen3-4B-Instruct-2507-Q4, Qwen3-4B-Thinking-2507-Q8, Qwen3-8B-Q4, Qwen3.5-2B-Base-Q4, Qwen3.5-2B-cuda-q4mix, Qwen3.5-2B-cuda-q8, Qwen3.5-2B-Q4, Qwen3.5-35B-A3B-cuda-q4mix, Qwen3.5-35B-A3B-cuda-q8, Qwen3.5-35B-A3B-Q4, Qwen3.6-27B-cuda-q4mix, Qwen3.6-27B-cuda-q8, Qwen3.6-27B-Q4, Qwen3.6-35B-A3B-cuda-q4mix, Qwen3.6-35B-A3B-cuda-q8, Qwen3.6-35B-A3B-Q4, Qwen3.8-27B-Q4, Qwen3.8-27B-Q8	gemma-3-1b-it, gemma-4-26B-A4B-it, gemma-4-E2B-it, gemma-4-E4B-it, Gemma-4-E4B-It, gpt-oss-20b, Gpt-Oss-20B, Llama 3.2 3B Instruct, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Muse-Glimmer-30B, NVIDIA-Nemotron-3-Nano-30B-A3B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-4B, Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-35B-A3B, Qwen3.6-27B, Qwen3.6-35B-A3B, Qwen3.8-27B, Tinystories Lay8 Hs512 Hd8 33M, tinystories-lay8-hs512-hd8-33M
Runtime	BaseRT	BaseRT, llama.cpp
Runtime version	BaseRT 0.3.0	BaseRT 0.2.4, BaseRT 0.2.5, llama.cpp b10809 (5266f24da), llama.cpp b9960 (a935fbffe)
Quantisation	mxfp4, Q4, Q8	BF16, F16, mxfp4, passthrough_gguf, Q2, Q2_K, Q3, Q3_K_M, Q4, Q4_0, Q4_1, Q4_K_M, Q5, Q5_K_M, Q6, Q6_K, Q8, Q8_0
Backend	CUDA	BLAS + Metal, Metal
Cooldown	off	off, on
Conditioning	warmup_only	idle_reset_then_warm, runtime_native_warmup, warmup_only
Decode workload	TG128 @ 1 ctx	TG128, TG128 @ 1 ctx
Harness schema	basert-benchmark-harness/1	basert-benchmark-harness/1, computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read unknown relative to Apple M5 Pro.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3-8B BaseRTQ4	49.7	59.7	0.83×	1,467	1,761	0.83×	1 / 7
Llama-3.2-3B-Instruct-cuda-q8 BaseRTQ4	87.0	94.6	0.92×	15,404	4,297	3.58×	2 / 5
Llama-3.2-1B-Instruct BaseRTQ4	191.9	206.3	0.93×	37,392	11,673	3.20×	3 / 3
Qwen3-0.6B-cuda-q4 BaseRTQ4	427.1	512.0	0.83×	51,561	20,603	2.50×	2 / 3
Qwen3.6-27B-cuda-q4mix BaseRTQ4	11.9	16.1	0.74×	1,706	364	4.68×	4 / 1
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRTQ4	76.6	98.1	0.78×	6,081	3,493	1.74×	2 / 2
Qwen3.5-2B-cuda-q4mix BaseRTQ4	148.7	217.7	0.68×	17,984	2,641	6.81×	3 / 1
gemma-4-E2B-it-cuda-q4mix BaseRTQ4	104.0	144.6	0.72×	14,648	13,240	1.11×	2 / 1
gpt-oss-20b BaseRTQ4	74.8	110.5	0.68×	4,356	1,156	3.77×	1 / 2
Mistral-7B-Instruct-v0.3 BaseRTQ4	30.2	51.7	0.58×	7,662	1,789	4.28×	1 / 2
Qwen3-4B-Instruct-2507 BaseRTQ4	72.2	89.9	0.80×	11,179	3,345	3.34×	1 / 2
Qwen3.5-35B-A3B-cuda-q4mix BaseRTQ4	85.3	113.4	0.75×	4,164	1,304	3.19×	2 / 1
Qwen3.6-35B-A3B-cuda-q4mix BaseRTQ4	63.8	113.2	0.56×	3,550	1,275	2.78×	2 / 1
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 BaseRTQ4	77.7	100.1	0.78×	3,911	2,499	1.56×	1 / 1
Qwen3-4B-Thinking-2507 BaseRTQ8	53.3	64.7	0.82×	11,490	3,414	3.37×	1 / 1
Qwen3.5-2B-cuda-q8 BaseRTQ8	110.5	130.7	0.84×	17,866	2,638	6.77×	1 / 1
Qwen3.6-27B-cuda-q8 BaseRTQ8	7.5	10.6	0.71×	1,700	355	4.79×	1 / 1

Side A · unknown

Top 25 of 41 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-0.6B-cuda-q4 BaseRT 0.3.0Q4	unknown CUDA	450.2 TG128 @ 1 ctx	51,746 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-0.6B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	404.0 TG128 @ 1 ctx	51,376 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-1B-Instruct-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	254.6 TG128 @ 1 ctx	37,864 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-1B-Instruct-Q4 BaseRT 0.3.0Q4	unknown CUDA	191.9 TG128 @ 1 ctx	34,471 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-2B-Q4 BaseRT 0.3.0Q4	unknown CUDA	183.2 TG128 @ 1 ctx	6,988 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.5-2B-Base-Q4 BaseRT 0.3.0Q4	unknown CUDA	182.5 TG128 @ 1 ctx	6,875 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.5-2B-Q4 BaseRT 0.3.0Q4	unknown CUDA	179.0 TG128 @ 1 ctx	6,868 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-1B-Instruct-Q8 BaseRT 0.3.0Q4	unknown CUDA	168.2 TG128 @ 1 ctx	37,392 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-2B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	148.7 TG128 @ 1 ctx	18,297 PP512	basecompute	14 Oct 2026	View Benchmark
gemma-4-E2B-it-Q4 BaseRT 0.3.0Q4	unknown CUDA	131.9 TG128 @ 1 ctx	11,398 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-2B-cuda-q8 BaseRT 0.3.0Q8	unknown CUDA	110.5 TG128 @ 1 ctx	17,866 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.5-2B-cuda-q8 BaseRT 0.3.0Q4	unknown CUDA	109.1 TG128 @ 1 ctx	17,984 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-3B-Instruct-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	107.0 TG128 @ 1 ctx	15,281 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-35B-A3B-Q4 BaseRT 0.3.0Q4	unknown CUDA	104.8 TG128 @ 1 ctx	2,937 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.6-35B-A3B-Q4 BaseRT 0.3.0Q4	unknown CUDA	104.8 TG128 @ 1 ctx	2,765 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	unknown CUDA	103.7 TG128 @ 1 ctx	4,780 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-35B-A3B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	92.6 TG128 @ 1 ctx	3,995 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-35B-A3B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	78.0 TG128 @ 1 ctx	4,334 PP512	basecompute	14 Oct 2026	View Benchmark
NVIDIA-Nemotron-3-Nano-30B-A3B-Q4 BaseRT 0.3.0Q4	unknown CUDA	77.7 TG128 @ 1 ctx	3,911 PP512	basecompute	9 Oct 2026	View Benchmark
gemma-4-E2B-it-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	76.0 TG128 @ 1 ctx	17,897 PP512	basecompute	9 Oct 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.3.0Q4	unknown CUDA	74.8 TG128 @ 1 ctx	4,356 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.6-35B-A3B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	73.1 TG128 @ 1 ctx	3,919 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3-4B-Instruct-2507-Q4 BaseRT 0.3.0Q4	unknown CUDA	72.2 TG128 @ 1 ctx	11,179 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-3B-Instruct-cuda-q8 BaseRT 0.3.0Q4	unknown CUDA	67.1 TG128 @ 1 ctx	15,528 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.6-35B-A3B-cuda-q8 BaseRT 0.3.0Q8	unknown CUDA	54.7 TG128 @ 1 ctx	3,156 PP512	basecompute	14 Oct 2026	View Benchmark

Side B · Apple M5 Pro

Top 25 of 129 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
ivnle/tinystories-lay8-hs512-hd8-33M BaseRT 0.2.4Q4	Apple M5 Pro Metal	2,068.0 TG128	190,107 PP512	arki05	17 Sep 2026	View Benchmark
Tinystories Lay8 Hs512 Hd8 33M llama.cpp b10809 (5266f24da)Q4_0	Apple M5 Pro BLAS + Metal	1,443.3 TG128	124,181 PP512	arki05	17 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q4	Apple M5 Pro Metal	516.8 TG128	20,778 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q4	Apple M5 Pro Metal	512.0 TG128	20,603 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.5Q4	Apple M5 Pro Metal	481.5 TG128 @ 1 ctx	19,595 PP512	isu	18 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	358.3 TG128	14,509 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	358.1 TG128	14,344 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	356.2 TG128	14,552 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	350.2 TG128	20,537 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	349.8 TG128	20,366 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	311.0 TG128	18,474 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	293.1 TG128	15,066 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q8_0	Apple M5 Pro BLAS + Metal	283.0 TG128	14,942 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	270.4 TG128	13,717 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q4	Apple M5 Pro Metal	235.4 TG128	8,040 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	229.1 TG128	11,673 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B-Base BaseRT 0.2.4Q4	Apple M5 Pro Metal	218.2 TG128	2,645 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B BaseRT 0.2.4Q4	Apple M5 Pro Metal	217.7 TG128	2,641 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q4	Apple M5 Pro Metal	210.3 TG128	7,268 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q8	Apple M5 Pro Metal	210.3 TG128	15,061 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	206.3 TG128	10,528 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	205.3 TG128	12,067 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-4-E2B-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	144.6 TG128	13,240 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q8	Apple M5 Pro Metal	139.7 TG128	8,043 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B-Base BaseRT 0.2.4Q8	Apple M5 Pro Metal	131.4 TG128	2,642 PP512	arki05	11 Sep 2026	View Benchmark

Compare

Loading community benchmarks…

What differs besides the chip

Facet	unknown	Apple M5 Pro
Model	gemma-4-E2B-it-cuda-q4mix, gemma-4-E2B-it-Q4, gpt-oss-120b-MXFP4, gpt-oss-120b-Q4, gpt-oss-20b-Q4, Llama-3.2-1B-Instruct-cuda-q4mix, Llama-3.2-1B-Instruct-Q4, Llama-3.2-1B-Instruct-Q8, Llama-3.2-3B-Instruct-cuda-q4mix, Llama-3.2-3B-Instruct-cuda-q8, Mistral-7B-Instruct-v0.3-Q8, NVIDIA-Nemotron-3-Nano-30B-A3B-Q4, Qwen3-0.6B-cuda-q4, Qwen3-0.6B-cuda-q4mix, Qwen3-30B-A3B-Instruct-2507-cuda-q8, Qwen3-30B-A3B-Instruct-2507-Q4, Qwen3-4B-Instruct-2507-Q4, Qwen3-4B-Thinking-2507-Q8, Qwen3-8B-Q4, Qwen3.5-2B-Base-Q4, Qwen3.5-2B-cuda-q4mix, Qwen3.5-2B-cuda-q8, Qwen3.5-2B-Q4, Qwen3.5-35B-A3B-cuda-q4mix, Qwen3.5-35B-A3B-cuda-q8, Qwen3.5-35B-A3B-Q4, Qwen3.6-27B-cuda-q4mix, Qwen3.6-27B-cuda-q8, Qwen3.6-27B-Q4, Qwen3.6-35B-A3B-cuda-q4mix, Qwen3.6-35B-A3B-cuda-q8, Qwen3.6-35B-A3B-Q4, Qwen3.8-27B-Q4, Qwen3.8-27B-Q8	gemma-3-1b-it, gemma-4-26B-A4B-it, gemma-4-E2B-it, gemma-4-E4B-it, Gemma-4-E4B-It, gpt-oss-20b, Gpt-Oss-20B, Llama 3.2 3B Instruct, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Muse-Glimmer-30B, NVIDIA-Nemotron-3-Nano-30B-A3B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-4B, Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-35B-A3B, Qwen3.6-27B, Qwen3.6-35B-A3B, Qwen3.8-27B, Tinystories Lay8 Hs512 Hd8 33M, tinystories-lay8-hs512-hd8-33M
Runtime	BaseRT	BaseRT, llama.cpp
Runtime version	BaseRT 0.3.0	BaseRT 0.2.4, BaseRT 0.2.5, llama.cpp b10809 (5266f24da), llama.cpp b9960 (a935fbffe)
Quantisation	mxfp4, Q4, Q8	BF16, F16, mxfp4, passthrough_gguf, Q2, Q2_K, Q3, Q3_K_M, Q4, Q4_0, Q4_1, Q4_K_M, Q5, Q5_K_M, Q6, Q6_K, Q8, Q8_0
Backend	CUDA	BLAS + Metal, Metal
Cooldown	off	off, on
Conditioning	warmup_only	idle_reset_then_warm, runtime_native_warmup, warmup_only
Decode workload	TG128 @ 1 ctx	TG128, TG128 @ 1 ctx
Harness schema	basert-benchmark-harness/1	basert-benchmark-harness/1, computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read unknown relative to Apple M5 Pro.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3-8B BaseRTQ4	49.7	59.7	0.83×	1,467	1,761	0.83×	1 / 7
Llama-3.2-3B-Instruct-cuda-q8 BaseRTQ4	87.0	94.6	0.92×	15,404	4,297	3.58×	2 / 5
Llama-3.2-1B-Instruct BaseRTQ4	191.9	206.3	0.93×	37,392	11,673	3.20×	3 / 3
Qwen3-0.6B-cuda-q4 BaseRTQ4	427.1	512.0	0.83×	51,561	20,603	2.50×	2 / 3
Qwen3.6-27B-cuda-q4mix BaseRTQ4	11.9	16.1	0.74×	1,706	364	4.68×	4 / 1
Qwen3-30B-A3B-Instruct-2507-cuda-q8 BaseRTQ4	76.6	98.1	0.78×	6,081	3,493	1.74×	2 / 2
Qwen3.5-2B-cuda-q4mix BaseRTQ4	148.7	217.7	0.68×	17,984	2,641	6.81×	3 / 1
gemma-4-E2B-it-cuda-q4mix BaseRTQ4	104.0	144.6	0.72×	14,648	13,240	1.11×	2 / 1
gpt-oss-20b BaseRTQ4	74.8	110.5	0.68×	4,356	1,156	3.77×	1 / 2
Mistral-7B-Instruct-v0.3 BaseRTQ4	30.2	51.7	0.58×	7,662	1,789	4.28×	1 / 2
Qwen3-4B-Instruct-2507 BaseRTQ4	72.2	89.9	0.80×	11,179	3,345	3.34×	1 / 2
Qwen3.5-35B-A3B-cuda-q4mix BaseRTQ4	85.3	113.4	0.75×	4,164	1,304	3.19×	2 / 1
Qwen3.6-35B-A3B-cuda-q4mix BaseRTQ4	63.8	113.2	0.56×	3,550	1,275	2.78×	2 / 1
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 BaseRTQ4	77.7	100.1	0.78×	3,911	2,499	1.56×	1 / 1
Qwen3-4B-Thinking-2507 BaseRTQ8	53.3	64.7	0.82×	11,490	3,414	3.37×	1 / 1
Qwen3.5-2B-cuda-q8 BaseRTQ8	110.5	130.7	0.84×	17,866	2,638	6.77×	1 / 1
Qwen3.6-27B-cuda-q8 BaseRTQ8	7.5	10.6	0.71×	1,700	355	4.79×	1 / 1

Side A · unknown

Top 25 of 41 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-0.6B-cuda-q4 BaseRT 0.3.0Q4	unknown CUDA	450.2 TG128 @ 1 ctx	51,746 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3-0.6B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	404.0 TG128 @ 1 ctx	51,376 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-1B-Instruct-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	254.6 TG128 @ 1 ctx	37,864 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-1B-Instruct-Q4 BaseRT 0.3.0Q4	unknown CUDA	191.9 TG128 @ 1 ctx	34,471 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-2B-Q4 BaseRT 0.3.0Q4	unknown CUDA	183.2 TG128 @ 1 ctx	6,988 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.5-2B-Base-Q4 BaseRT 0.3.0Q4	unknown CUDA	182.5 TG128 @ 1 ctx	6,875 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.5-2B-Q4 BaseRT 0.3.0Q4	unknown CUDA	179.0 TG128 @ 1 ctx	6,868 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-1B-Instruct-Q8 BaseRT 0.3.0Q4	unknown CUDA	168.2 TG128 @ 1 ctx	37,392 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-2B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	148.7 TG128 @ 1 ctx	18,297 PP512	basecompute	14 Oct 2026	View Benchmark
gemma-4-E2B-it-Q4 BaseRT 0.3.0Q4	unknown CUDA	131.9 TG128 @ 1 ctx	11,398 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-2B-cuda-q8 BaseRT 0.3.0Q8	unknown CUDA	110.5 TG128 @ 1 ctx	17,866 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.5-2B-cuda-q8 BaseRT 0.3.0Q4	unknown CUDA	109.1 TG128 @ 1 ctx	17,984 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-3B-Instruct-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	107.0 TG128 @ 1 ctx	15,281 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-35B-A3B-Q4 BaseRT 0.3.0Q4	unknown CUDA	104.8 TG128 @ 1 ctx	2,937 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3.6-35B-A3B-Q4 BaseRT 0.3.0Q4	unknown CUDA	104.8 TG128 @ 1 ctx	2,765 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3-30B-A3B-Instruct-2507-Q4 BaseRT 0.3.0Q4	unknown CUDA	103.7 TG128 @ 1 ctx	4,780 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-35B-A3B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	92.6 TG128 @ 1 ctx	3,995 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.5-35B-A3B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	78.0 TG128 @ 1 ctx	4,334 PP512	basecompute	14 Oct 2026	View Benchmark
NVIDIA-Nemotron-3-Nano-30B-A3B-Q4 BaseRT 0.3.0Q4	unknown CUDA	77.7 TG128 @ 1 ctx	3,911 PP512	basecompute	9 Oct 2026	View Benchmark
gemma-4-E2B-it-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	76.0 TG128 @ 1 ctx	17,897 PP512	basecompute	9 Oct 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.3.0Q4	unknown CUDA	74.8 TG128 @ 1 ctx	4,356 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.6-35B-A3B-cuda-q4mix BaseRT 0.3.0Q4	unknown CUDA	73.1 TG128 @ 1 ctx	3,919 PP512	basecompute	14 Oct 2026	View Benchmark
Qwen3-4B-Instruct-2507-Q4 BaseRT 0.3.0Q4	unknown CUDA	72.2 TG128 @ 1 ctx	11,179 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.2-3B-Instruct-cuda-q8 BaseRT 0.3.0Q4	unknown CUDA	67.1 TG128 @ 1 ctx	15,528 PP512	basecompute	9 Oct 2026	View Benchmark
Qwen3.6-35B-A3B-cuda-q8 BaseRT 0.3.0Q8	unknown CUDA	54.7 TG128 @ 1 ctx	3,156 PP512	basecompute	14 Oct 2026	View Benchmark

Side B · Apple M5 Pro

Top 25 of 129 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
ivnle/tinystories-lay8-hs512-hd8-33M BaseRT 0.2.4Q4	Apple M5 Pro Metal	2,068.0 TG128	190,107 PP512	arki05	17 Sep 2026	View Benchmark
Tinystories Lay8 Hs512 Hd8 33M llama.cpp b10809 (5266f24da)Q4_0	Apple M5 Pro BLAS + Metal	1,443.3 TG128	124,181 PP512	arki05	17 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q4	Apple M5 Pro Metal	516.8 TG128	20,778 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q4	Apple M5 Pro Metal	512.0 TG128	20,603 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.5Q4	Apple M5 Pro Metal	481.5 TG128 @ 1 ctx	19,595 PP512	isu	18 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	358.3 TG128	14,509 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	358.1 TG128	14,344 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	356.2 TG128	14,552 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	350.2 TG128	20,537 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	349.8 TG128	20,366 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	311.0 TG128	18,474 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	293.1 TG128	15,066 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q8_0	Apple M5 Pro BLAS + Metal	283.0 TG128	14,942 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	270.4 TG128	13,717 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q4	Apple M5 Pro Metal	235.4 TG128	8,040 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	229.1 TG128	11,673 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B-Base BaseRT 0.2.4Q4	Apple M5 Pro Metal	218.2 TG128	2,645 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B BaseRT 0.2.4Q4	Apple M5 Pro Metal	217.7 TG128	2,641 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q4	Apple M5 Pro Metal	210.3 TG128	7,268 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q8	Apple M5 Pro Metal	210.3 TG128	15,061 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	206.3 TG128	10,528 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	205.3 TG128	12,067 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-4-E2B-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	144.6 TG128	13,240 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q8	Apple M5 Pro Metal	139.7 TG128	8,043 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B-Base BaseRT 0.2.4Q8	Apple M5 Pro Metal	131.4 TG128	2,642 PP512	arki05	11 Sep 2026	View Benchmark