Compare

Loading community benchmarks…

Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 163
Like-for-like pairs: 29
Chips available: 16

Compare

Side A

Side B

Model

Runtime

Runtime version

Quantisation

Backend

Prefill size

Decode size

Cooldown

34 runs vs 129 runs · 29 like-for-like pairs

Held fixedPrefill: PP512Decode: TG128

These figures mix different models. Pick a model for a like-for-like comparison.
A side mixes quantisations. Pick a quantisation to compare the same weights.
The sides differ in conditioning. Ratios may reflect benchmark protocol rather than the chip.

Chip comparison

Apple M4 Pro vs Apple M5 Pro

Ratios read side A relative to side B. Geometric mean over 29 like-for-like pairs.

Decode: 0.83×-17.1%
Prefill: 0.28×-72.2%

Side A

Apple M4 Pro

34 runs

2 contributors

Latest 20 Sep 2026

Decode · TG128: 78.8tok/s median
best 284.7 · worst 31.6
Prefill · PP512: 943tok/s median
best 4,787 · worst 392

Models: gemma-3-1b-it-Q4, gemma-3-1b-it-Q8, Gemma-4 26B-A4B IT (smart Q4_0, QAT-lossless), gemma-4-E2B-it-Q4, gemma-4-E2B-it-Q8, gemma-4-E4B-it-Q4, gemma-4-E4B-it-Q8, gpt-oss-20b-MXFP4, gpt-oss-20b-Q4, gpt-oss-20b-Q8, Llama-3.1-8B-Instruct-Q4, Llama-3.1-8B-Instruct-Q8, Llama-3.2-1B-Instruct-Q4, Llama-3.2-1B-Instruct-Q8, Llama-3.2-3B-Instruct-Q4, Llama-3.2-3B-Instruct-Q8, Mistral-7B-Instruct-v0.3-Q4, Mistral-7B-Instruct-v0.3-Q8, Qwen3-0.6B-Q4, Qwen3-0.6B-Q8, Qwen3-1.7B-Q4, Qwen3-1.7B-Q8, Qwen3-4B-Instruct-2507-Q4, Qwen3-4B-Instruct-2507-Q8, Qwen3-4B-Q4, Qwen3-4B-Q8, Qwen3-4B-Thinking-2507-Q4, Qwen3-4B-Thinking-2507-Q8, Qwen3-8B-Q4, Qwen3-8B-Q8, Qwen3.5-2B-Base-Q4, Qwen3.5-2B-Base-Q8, Qwen3.5-2B-Q4, Qwen3.5-2B-Q8
Runtimes: BaseRTllama.cpp
Versions: BaseRT 0.2.6, llama.cpp b9650 (d8a3f523c)
Quantisations: mxfp4, Q4, Q4_0, Q8
Backends: BLAS + Metal, Metal
Cooldown: off, on
Conditioning: idle_reset_before_workload_process, warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1, computearena-measurements/1

Side B

Apple M5 Pro

129 runs

2 contributors

Latest 18 Sep 2026

Decode · TG128: 72.8tok/s median
best 2,068.0 · worst 10.6
Prefill · PP512: 1,852tok/s median
best 190,107 · worst 327

Models: gemma-3-1b-it, gemma-4-26B-A4B-it, gemma-4-E2B-it, gemma-4-E4B-it, Gemma-4-E4B-It, gpt-oss-20b, Gpt-Oss-20B, Llama 3.2 3B Instruct, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Muse-Glimmer-30B, NVIDIA-Nemotron-3-Nano-30B-A3B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-4B, Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-35B-A3B, Qwen3.6-27B, Qwen3.6-35B-A3B, Qwen3.8-27B, Tinystories Lay8 Hs512 Hd8 33M, tinystories-lay8-hs512-hd8-33M
Runtimes: BaseRTllama.cpp
Versions: BaseRT 0.2.4, BaseRT 0.2.5, llama.cpp b10809 (5266f24da), llama.cpp b9960 (a935fbffe)
Quantisations: BF16, F16, mxfp4, passthrough_gguf, Q2, Q2_K, Q3, Q3_K_M, Q4, Q4_0, Q4_1, Q4_K_M, Q5, Q5_K_M, Q6, Q6_K, Q8, Q8_0
Backends: BLAS + Metal, Metal
Cooldown: off, on
Conditioning: idle_reset_then_warm, runtime_native_warmup, warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1, computearena-measurements/1

What differs besides the chip

Facet	Apple M4 Pro	Apple M5 Pro
Model	gemma-3-1b-it-Q4, gemma-3-1b-it-Q8, Gemma-4 26B-A4B IT (smart Q4_0, QAT-lossless), gemma-4-E2B-it-Q4, gemma-4-E2B-it-Q8, gemma-4-E4B-it-Q4, gemma-4-E4B-it-Q8, gpt-oss-20b-MXFP4, gpt-oss-20b-Q4, gpt-oss-20b-Q8, Llama-3.1-8B-Instruct-Q4, Llama-3.1-8B-Instruct-Q8, Llama-3.2-1B-Instruct-Q4, Llama-3.2-1B-Instruct-Q8, Llama-3.2-3B-Instruct-Q4, Llama-3.2-3B-Instruct-Q8, Mistral-7B-Instruct-v0.3-Q4, Mistral-7B-Instruct-v0.3-Q8, Qwen3-0.6B-Q4, Qwen3-0.6B-Q8, Qwen3-1.7B-Q4, Qwen3-1.7B-Q8, Qwen3-4B-Instruct-2507-Q4, Qwen3-4B-Instruct-2507-Q8, Qwen3-4B-Q4, Qwen3-4B-Q8, Qwen3-4B-Thinking-2507-Q4, Qwen3-4B-Thinking-2507-Q8, Qwen3-8B-Q4, Qwen3-8B-Q8, Qwen3.5-2B-Base-Q4, Qwen3.5-2B-Base-Q8, Qwen3.5-2B-Q4, Qwen3.5-2B-Q8	gemma-3-1b-it, gemma-4-26B-A4B-it, gemma-4-E2B-it, gemma-4-E4B-it, Gemma-4-E4B-It, gpt-oss-20b, Gpt-Oss-20B, Llama 3.2 3B Instruct, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Muse-Glimmer-30B, NVIDIA-Nemotron-3-Nano-30B-A3B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-30B-A3B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-4B, Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507, Qwen3-8B, Qwen3-8B-base-q2, Qwen3-8B-base-q3, Qwen3-8B-base-q5, Qwen3-8B-base-q6, Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-35B-A3B, Qwen3.6-27B, Qwen3.6-35B-A3B, Qwen3.8-27B, Tinystories Lay8 Hs512 Hd8 33M, tinystories-lay8-hs512-hd8-33M
Runtime version	BaseRT 0.2.6, llama.cpp b9650 (d8a3f523c)	BaseRT 0.2.4, BaseRT 0.2.5, llama.cpp b10809 (5266f24da), llama.cpp b9960 (a935fbffe)
Quantisation	mxfp4, Q4, Q4_0, Q8	BF16, F16, mxfp4, passthrough_gguf, Q2, Q2_K, Q3, Q3_K_M, Q4, Q4_0, Q4_1, Q4_K_M, Q5, Q5_K_M, Q6, Q6_K, Q8, Q8_0
Conditioning	idle_reset_before_workload_process, warmup_only	idle_reset_then_warm, runtime_native_warmup, warmup_only

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read Apple M4 Pro relative to Apple M5 Pro.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Qwen3-8B BaseRTQ4	52.6	59.7	0.88×	394	1,761	0.22×	1 / 7
Llama-3.2-3B-Instruct BaseRTQ4	83.1	94.6	0.88×	943	4,297	0.22×	2 / 5
Llama-3.1-8B-Instruct BaseRTQ4	44.1	48.8	0.90×	393	1,784	0.22×	2 / 4
Llama-3.2-1B-Instruct BaseRTQ4	191.9	206.3	0.93×	2,688	11,673	0.23×	2 / 3
Mistral-7B-Instruct-v0.3 BaseRTQ4	46.4	51.7	0.90×	393	1,789	0.22×	2 / 2
Qwen3-0.6B BaseRTQ4	208.6	512.0	0.41×	3,571	20,603	0.17×	1 / 3
Qwen3-0.6B BaseRTQ8	284.7	349.8	0.81×	4,787	20,366	0.24×	1 / 3
gemma-3-1b-it BaseRTQ4	224.9	281.7	0.80×	3,430	14,392	0.24×	1 / 2
gemma-4-E4B-it BaseRTQ4	69.1	82.3	0.84×	1,094	4,646	0.24×	1 / 2
gemma-4-E4B-it BaseRTQ8	44.5	51.2	0.87×	1,081	4,476	0.24×	1 / 2
gpt-oss-20b BaseRTQ4	79.3	110.5	0.72×	732	1,156	0.63×	1 / 2
Qwen3-1.7B BaseRTQ4	193.1	222.9	0.87×	1,828	7,654	0.24×	1 / 2
Qwen3-1.7B BaseRTQ8	125.3	132.4	0.95×	1,822	7,655	0.24×	1 / 2
Qwen3-4B BaseRTQ4	90.3	105.6	0.85×	734	3,378	0.22×	1 / 2
Qwen3-4B-Instruct-2507 BaseRTQ4	77.9	89.9	0.87×	727	3,345	0.22×	1 / 2
Qwen3-4B-Instruct-2507 BaseRTQ8	56.2	63.5	0.88×	731	3,402	0.21×	1 / 2
Qwen3-8B BaseRTQ8	31.6	34.9	0.91×	392	1,746	0.22×	1 / 2
gemma-3-1b-it BaseRTQ8	178.8	210.3	0.85×	3,337	15,061	0.22×	1 / 1
gemma-4-E2B-it BaseRTQ4	126.0	144.6	0.87×	3,722	13,240	0.28×	1 / 1
gemma-4-E2B-it BaseRTQ8	83.9	96.0	0.87×	3,628	12,646	0.29×	1 / 1
gpt-oss-20b BaseRTQ8	73.7	100.6	0.73×	731	1,065	0.69×	1 / 1
gpt-oss-20b BaseRTmxfp4	72.7	103.6	0.70×	734	1,013	0.72×	1 / 1
Qwen3-4B BaseRTQ8	56.3	64.7	0.87×	726	3,412	0.21×	1 / 1
Qwen3-4B-Thinking-2507 BaseRTQ4	78.4	91.6	0.86×	732	3,363	0.22×	1 / 1
Qwen3-4B-Thinking-2507 BaseRTQ8	56.3	64.7	0.87×	731	3,414	0.21×	1 / 1
Qwen3.5-2B BaseRTQ4	179.1	217.7	0.82×	1,150	2,641	0.44×	1 / 1
Qwen3.5-2B BaseRTQ8	113.5	130.7	0.87×	1,138	2,638	0.43×	1 / 1
Qwen3.5-2B-Base BaseRTQ4	176.8	218.2	0.81×	1,150	2,645	0.43×	1 / 1
Qwen3.5-2B-Base BaseRTQ8	113.9	131.4	0.87×	1,149	2,642	0.43×	1 / 1

Side A · Apple M4 Pro

Top 25 of 34 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Qwen3-0.6B-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	284.7 TG128 @ 1 ctx	4,787 PP512	basecompute	19 Sep 2026	View Benchmark
gemma-3-1b-it-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	224.9 TG128 @ 1 ctx	3,430 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-0.6B-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	208.6 TG128 @ 1 ctx	3,571 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.2-1B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	204.6 TG128 @ 1 ctx	2,685 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-1.7B-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	193.1 TG128 @ 1 ctx	1,828 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.2-1B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M4 Pro Metal	179.3 TG128 @ 1 ctx	2,692 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3.5-2B-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	179.1 TG128 @ 1 ctx	1,150 PP512	basecompute	20 Sep 2026	View Benchmark
gemma-3-1b-it-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	178.8 TG128 @ 1 ctx	3,337 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3.5-2B-Base-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	176.8 TG128 @ 1 ctx	1,150 PP512	basecompute	20 Sep 2026	View Benchmark
gemma-4-E2B-it-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	126.0 TG128 @ 1 ctx	3,722 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-1.7B-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	125.3 TG128 @ 1 ctx	1,822 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3.5-2B-Base-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	113.9 TG128 @ 1 ctx	1,149 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3.5-2B-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	113.5 TG128 @ 1 ctx	1,138 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.2-3B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	95.0 TG128 @ 1 ctx	943 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-4B-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	90.3 TG128 @ 1 ctx	734 PP512	basecompute	20 Sep 2026	View Benchmark
gemma-4-E2B-it-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	83.9 TG128 @ 1 ctx	3,628 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	79.3 TG128 @ 1 ctx	732 PP512	basecompute	19 Sep 2026	View Benchmark
Qwen3-4B-Thinking-2507-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	78.4 TG128 @ 1 ctx	732 PP512	basecompute	20 Sep 2026	View Benchmark
Qwen3-4B-Instruct-2507-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	77.9 TG128 @ 1 ctx	727 PP512	basecompute	20 Sep 2026	View Benchmark
Gemma-4 26B-A4B IT (smart Q4_0, QAT-lossless) llama.cpp b9650 (d8a3f523c)Q4_0	Apple M4 Pro BLAS + Metal	74.7 TG128	757 PP512	invocation	15 Sep 2026	View Benchmark
gpt-oss-20b-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	73.7 TG128 @ 1 ctx	731 PP512	basecompute	19 Sep 2026	View Benchmark
gpt-oss-20b-MXFP4 BaseRT 0.2.6mxfp4	Apple M4 Pro Metal	72.7 TG128 @ 1 ctx	734 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.2-3B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M4 Pro Metal	71.2 TG128 @ 1 ctx	943 PP512	basecompute	20 Sep 2026	View Benchmark
gemma-4-E4B-it-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	69.1 TG128 @ 1 ctx	1,094 PP512	basecompute	20 Sep 2026	View Benchmark
Mistral-7B-Instruct-v0.3-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	59.0 TG128 @ 1 ctx	394 PP512	basecompute	20 Sep 2026	View Benchmark

Side B · Apple M5 Pro

Top 25 of 129 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
ivnle/tinystories-lay8-hs512-hd8-33M BaseRT 0.2.4Q4	Apple M5 Pro Metal	2,068.0 TG128	190,107 PP512	arki05	17 Sep 2026	View Benchmark
Tinystories Lay8 Hs512 Hd8 33M llama.cpp b10809 (5266f24da)Q4_0	Apple M5 Pro BLAS + Metal	1,443.3 TG128	124,181 PP512	arki05	17 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q4	Apple M5 Pro Metal	516.8 TG128	20,778 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q4	Apple M5 Pro Metal	512.0 TG128	20,603 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.5Q4	Apple M5 Pro Metal	481.5 TG128 @ 1 ctx	19,595 PP512	isu	18 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	358.3 TG128	14,509 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	358.1 TG128	14,344 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	356.2 TG128	14,552 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	350.2 TG128	20,537 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	349.8 TG128	20,366 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-0.6B BaseRT 0.2.4Q8	Apple M5 Pro Metal	311.0 TG128	18,474 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	293.1 TG128	15,066 PP512	arki05	11 Sep 2026	View Benchmark
Qwen3-0.6B llama.cpp b10809 (5266f24da)Q8_0	Apple M5 Pro BLAS + Metal	283.0 TG128	14,942 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	270.4 TG128	13,717 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q4	Apple M5 Pro Metal	235.4 TG128	8,040 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	229.1 TG128	11,673 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B-Base BaseRT 0.2.4Q4	Apple M5 Pro Metal	218.2 TG128	2,645 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B BaseRT 0.2.4Q4	Apple M5 Pro Metal	217.7 TG128	2,641 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q4	Apple M5 Pro Metal	210.3 TG128	7,268 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-3-1b-it BaseRT 0.2.4Q8	Apple M5 Pro Metal	210.3 TG128	15,061 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	206.3 TG128	10,528 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.2-1B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	205.3 TG128	12,067 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gemma-4-E2B-it BaseRT 0.2.4Q4	Apple M5 Pro Metal	144.6 TG128	13,240 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3-1.7B BaseRT 0.2.4Q8	Apple M5 Pro Metal	139.7 TG128	8,043 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Qwen3.5-2B-Base BaseRT 0.2.4Q8	Apple M5 Pro Metal	131.4 TG128	2,642 PP512	arki05	11 Sep 2026	View Benchmark