Compare

Loading community benchmarks…

Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 27
Like-for-like pair: 1
Runtimes available: 2

Compare

Side A

Side B

Model

Chip

Quantisation A

Quantisation B

Backend

Prefill size

Decode size

Cooldown

20 runs vs 7 runs · 1 like-for-like pair

Held fixedModel: GPT OSS 20BPrefill: PP512Decode: TG128

The sides differ in conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the runtime.

Runtime comparison

BaseRT vs llama.cpp

Ratios read side A relative to side B. Geometric mean over 1 like-for-like pair.

Decode: 1.31×+31.2%
Prefill: 0.60×-40.0%

Side A

BaseRT

20 runs

3 contributors

Latest 20 Sep 2026

Decode · TG128: 102.1tok/s median
best 185.1 · worst 51.4
Prefill · PP512: 1,428tok/s median
best 4,424 · worst 731

Models: gpt-oss-20b, gpt-oss-20b-MXFP4, gpt-oss-20b-Q4, gpt-oss-20b-Q8
Chips: Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Max, Apple M5 Pro, NVIDIA GB10
Versions: BaseRT 0.2.4, BaseRT 0.2.6
Quantisations: mxfp4, Q4, Q8
Backends: CUDA, Metal
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

Side B

llama.cpp

7 runs

1 contributor

Latest 17 Sep 2026

Decode · TG128: 146.8tok/s median
best 195.6 · worst 66.6
Prefill · PP512: 3,262tok/s median
best 3,604 · worst 1,849

Models: Gpt-Oss-20B
Chips: AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Versions: llama.cpp b1 (e64c0ea), llama.cpp b10809 (5266f24da)
Quantisations: F16, Q4_0, Q4_K_M
Backends: BLAS + Metal, CUDA, ROCm, Vulkan
Cooldown: off
Conditioning: runtime_native_warmup
Decode workload: TG128
Harness schema: computearena-measurements/1

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	gpt-oss-20b, gpt-oss-20b-MXFP4, gpt-oss-20b-Q4, gpt-oss-20b-Q8	Gpt-Oss-20B
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Max, Apple M5 Pro, NVIDIA GB10	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Backend	CUDA, Metal	BLAS + Metal, CUDA, ROCm, Vulkan
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read BaseRT relative to llama.cpp.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
gpt-oss-20b Apple M5 Pro	106.9	81.5	1.31×	1,110	1,851	0.60×	4 / 2

Side A · BaseRT

Top 20 of 20 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
gpt-oss-20b-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	185.1 TG128 @ 1 ctx	2,554 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-Q8 BaseRT 0.2.6Q8	Apple M3 Ultra Metal	175.8 TG128 @ 1 ctx	2,627 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/gpt-oss-20b BaseRT 0.2.4Q4	Apple M5 Max Metal	169.0 TG128	2,179 PP512	lukas	11 Sep 2026	View Benchmark
gpt-oss-20b-MXFP4 BaseRT 0.2.6mxfp4	Apple M3 Ultra Metal	163.9 TG128 @ 1 ctx	2,590 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	160.5 TG128 @ 1 ctx	1,709 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-Q8 BaseRT 0.2.6Q8	Apple M4 Max Metal	146.9 TG128 @ 1 ctx	1,700 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-MXFP4 BaseRT 0.2.6mxfp4	Apple M4 Max Metal	146.7 TG128 @ 1 ctx	1,715 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/gpt-oss-20b BaseRT 0.2.4Q4	Apple M5 Pro Metal	110.9 TG128	1,155 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gpt-oss-20b BaseRT 0.2.4Q4	Apple M5 Pro Metal	110.1 TG128	1,156 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gpt-oss-20b BaseRT 0.2.4mxfp4	Apple M5 Pro Metal	103.6 TG128	1,013 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/gpt-oss-20b BaseRT 0.2.4Q8	Apple M5 Pro Metal	100.6 TG128	1,065 PP512	arki05	11 Sep 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	82.3 TG128 @ 1 ctx	881 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	79.3 TG128 @ 1 ctx	732 PP512	basecompute	19 Sep 2026	View Benchmark
gpt-oss-20b-Q8 BaseRT 0.2.6Q8	Apple M1 Max Metal	76.7 TG128 @ 1 ctx	880 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-MXFP4 BaseRT 0.2.6mxfp4	Apple M1 Max Metal	76.6 TG128 @ 1 ctx	890 PP512	basecompute	20 Sep 2026	View Benchmark
gpt-oss-20b-Q8 BaseRT 0.2.6Q8	Apple M4 Pro Metal	73.7 TG128 @ 1 ctx	731 PP512	basecompute	19 Sep 2026	View Benchmark
gpt-oss-20b-MXFP4 BaseRT 0.2.6mxfp4	Apple M4 Pro Metal	72.7 TG128 @ 1 ctx	734 PP512	basecompute	19 Sep 2026	View Benchmark
gpt-oss-20b-MXFP4 BaseRT 0.2.6mxfp4	NVIDIA GB10 CUDA	60.2 TG128 @ 1 ctx	4,424 PP512	basecompute	19 Sep 2026	View Benchmark
gpt-oss-20b-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	57.3 TG128 @ 1 ctx	4,292 PP512	basecompute	19 Sep 2026	View Benchmark
gpt-oss-20b-Q8 BaseRT 0.2.6Q8	NVIDIA GB10 CUDA	51.4 TG128 @ 1 ctx	4,421 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 7 of 7 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Gpt-Oss-20B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	195.6 TG128	3,292 PP512	arki05	11 Sep 2026	View Benchmark
Gpt-Oss-20B llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	160.1 TG128	3,502 PP512	arki05	11 Sep 2026	View Benchmark
Gpt-Oss-20B llama.cpp b1 (e64c0ea)Q4_0	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	157.9 TG128	3,038 PP512	arki05	17 Sep 2026	View Benchmark
Gpt-Oss-20B llama.cpp b10809 (5266f24da)F16	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	146.8 TG128	3,262 PP512	arki05	11 Sep 2026	View Benchmark
Gpt-Oss-20B llama.cpp b10809 (5266f24da)F16	AMD Radeon RX 7900 XT ROCm	128.8 TG128	3,604 PP512	arki05	11 Sep 2026	View Benchmark
Gpt-Oss-20B llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	96.3 TG128	1,849 PP512	arki05	11 Sep 2026	View Benchmark
Gpt-Oss-20B llama.cpp b10809 (5266f24da)F16	Apple M5 Pro BLAS + Metal	66.6 TG128	1,852 PP512	arki05	11 Sep 2026	View Benchmark