Community benchmarks

Compare

Put two chips, runtimes, or runtime versions side by side. Everything else is held fixed or called out, so a ratio is worth exactly what the matched runs say.

Add your benchmark

Runs compared: 40
Like-for-like pair: 1
Runtimes available: 2

Compare

Side A

Side B

Model

Chip

Quantisation A

Quantisation B

Backend

Prefill size

Decode size

Cooldown

24 runs vs 16 runs · 1 like-for-like pair

Held fixedModel: Llama 3.1 8B InstructPrefill: PP512Decode: TG128

The sides differ in conditioning, decode workload, harness schema. Ratios may reflect benchmark protocol rather than the runtime.

Runtime comparison

BaseRT vs llama.cpp

Ratios read side A relative to side B. Geometric mean over 1 like-for-like pair.

Decode: 0.88×-12.1%
Prefill: 1.25×+25.3%

Side A

BaseRT

24 runs

2 contributors

Latest 19 Oct 2026

Decode · TG128: 58.2tok/s median
best 125.7 · worst 25.4
Prefill · PP512: 1,173tok/s median
best 7,567 · worst 392

Models: Llama-3.1-8B-Instruct, Llama-3.1-8B-Instruct-Q4, Llama-3.1-8B-Instruct-Q8
Chips: Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Pro, NVIDIA GB10
Versions: BaseRT 0.2.4, BaseRT 0.2.6, BaseRT 0.3.0
Quantisations: Q4
Backends: CUDA, Metal
Cooldown: off
Conditioning: warmup_only
Decode workload: TG128, TG128 @ 1 ctx
Harness schema: basert-benchmark-harness/1

Side B

llama.cpp

16 runs

1 contributor

Latest 17 Sep 2026

Decode · TG128: 92.3tok/s median
best 136.1 · worst 29.6
Prefill · PP512: 2,749tok/s median
best 3,390 · worst 1,421

Models: Llama-3.1-8B-Instruct, Meta Llama 3.1 8B Instruct
Chips: AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Versions: llama.cpp b1 (e64c0ea), llama.cpp b10809 (5266f24da)
Quantisations: Q4_0, Q4_K_M, Q6_K, Q8_0
Backends: BLAS + Metal, CUDA, ROCm, Vulkan
Cooldown: off
Conditioning: runtime_native_warmup
Decode workload: TG128
Harness schema: computearena-measurements/1

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	Llama-3.1-8B-Instruct, Llama-3.1-8B-Instruct-Q4, Llama-3.1-8B-Instruct-Q8	Llama-3.1-8B-Instruct, Meta Llama 3.1 8B Instruct
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Pro, NVIDIA GB10	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Backend	CUDA, Metal	BLAS + Metal, CUDA, ROCm, Vulkan
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Like-for-like pairs

Each row is a configuration present on both sides. Values are per-cell medians; ratios read BaseRT relative to llama.cpp.

Configuration	Decode A	Decode B	Ratio	Prefill A	Prefill B	Ratio	Runs A / B
Llama-3.1-8B-Instruct Apple M5 Pro	48.8	55.5	0.88×	1,784	1,424	1.25×	4 / 5

Side A · BaseRT

Top 24 of 24 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M3 Ultra Metal	125.7 TG128 @ 1 ctx	1,430 PP512	basecompute	19 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	124.4 TG128 @ 1 ctx	1,438 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M4 Max Metal	106.5 TG128 @ 1 ctx	954 PP512	basecompute	17 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	105.5 TG128 @ 1 ctx	946 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M3 Ultra Metal	78.1 TG128 @ 1 ctx	1,423 PP512	basecompute	19 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	75.3 TG128 @ 1 ctx	1,392 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	63.7 TG128 @ 1 ctx	509 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M1 Max Metal	63.5 TG128 @ 1 ctx	511 PP512	basecompute	9 Oct 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	63.2 TG128	1,795 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	62.0 TG128	1,795 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M4 Max Metal	60.8 TG128 @ 1 ctx	947 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M4 Max Metal	60.3 TG128 @ 1 ctx	948 PP512	basecompute	17 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	56.1 TG128 @ 1 ctx	394 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M4 Pro Metal	55.7 TG128 @ 1 ctx	394 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	53.1 TG128 @ 1 ctx	7,567 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	NVIDIA GB10 CUDA	51.6 TG128 @ 1 ctx	7,333 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M1 Max Metal	38.7 TG128 @ 1 ctx	508 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M1 Max Metal	38.6 TG128 @ 1 ctx	509 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	35.6 TG128	1,774 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	35.5 TG128	1,773 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M4 Pro Metal	32.1 TG128 @ 1 ctx	392 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M4 Pro Metal	31.9 TG128 @ 1 ctx	392 PP512	basecompute	11 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	NVIDIA GB10 CUDA	25.5 TG128 @ 1 ctx	7,170 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	25.4 TG128 @ 1 ctx	7,299 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 16 of 16 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Meta Llama 3.1 8B Instruct llama.cpp b1 (e64c0ea)Q4_K_M	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	136.1 TG128	3,390 PP512	arki05	17 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_0	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	126.7 TG128	2,959 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	122.7 TG128	2,743 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	119.8 TG128	2,756 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_0	AMD Radeon RX 7900 XT ROCm	117.4 TG128	3,233 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	97.7 TG128	2,512 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	97.5 TG128	3,150 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	97.1 TG128	3,172 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT ROCm	87.4 TG128	2,851 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q8_0	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	68.6 TG128	2,286 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q8_0	AMD Radeon RX 7900 XT ROCm	66.2 TG128	3,292 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_0	Apple M5 Pro BLAS + Metal	58.7 TG128	1,538 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	55.6 TG128	1,424 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	55.5 TG128	1,423 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q6_K	Apple M5 Pro BLAS + Metal	43.4 TG128	1,421 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q8_0	Apple M5 Pro BLAS + Metal	29.6 TG128	1,465 PP512	arki05	11 Sep 2026	View Benchmark

Compare

Loading community benchmarks…

What differs besides the runtime

Facet	BaseRT	llama.cpp
Model	Llama-3.1-8B-Instruct, Llama-3.1-8B-Instruct-Q4, Llama-3.1-8B-Instruct-Q8	Llama-3.1-8B-Instruct, Meta Llama 3.1 8B Instruct
Chip	Apple M1 Max, Apple M3 Ultra, Apple M4 Max, Apple M4 Pro, Apple M5 Pro, NVIDIA GB10	AMD Radeon RX 7900 XT, AMD Radeon RX 7900 XT (RADV NAVI31), Apple M5 Pro, Tesla T10/Tesla T10/Tesla T10/Tesla T10
Backend	CUDA, Metal	BLAS + Metal, CUDA, ROCm, Vulkan
Conditioning	warmup_only	runtime_native_warmup
Decode workload	TG128, TG128 @ 1 ctx	TG128
Harness schema	basert-benchmark-harness/1	computearena-measurements/1

Configuration

Decode A

Decode B

Ratio

Prefill A

Prefill B

Ratio

Runs A / B

Llama-3.1-8B-Instruct

Apple M5 Pro

48.8

55.5

0.88×

1,784

1,424

1.25×

4 / 5

Side A · BaseRT

Top 24 of 24 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M3 Ultra Metal	125.7 TG128 @ 1 ctx	1,430 PP512	basecompute	19 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	124.4 TG128 @ 1 ctx	1,438 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M4 Max Metal	106.5 TG128 @ 1 ctx	954 PP512	basecompute	17 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M4 Max Metal	105.5 TG128 @ 1 ctx	946 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M3 Ultra Metal	78.1 TG128 @ 1 ctx	1,423 PP512	basecompute	19 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M3 Ultra Metal	75.3 TG128 @ 1 ctx	1,392 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M1 Max Metal	63.7 TG128 @ 1 ctx	509 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M1 Max Metal	63.5 TG128 @ 1 ctx	511 PP512	basecompute	9 Oct 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	63.2 TG128	1,795 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	62.0 TG128	1,795 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M4 Max Metal	60.8 TG128 @ 1 ctx	947 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M4 Max Metal	60.3 TG128 @ 1 ctx	948 PP512	basecompute	17 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	Apple M4 Pro Metal	56.1 TG128 @ 1 ctx	394 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	Apple M4 Pro Metal	55.7 TG128 @ 1 ctx	394 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	53.1 TG128 @ 1 ctx	7,567 PP512	basecompute	19 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q4 BaseRT 0.3.0Q4	NVIDIA GB10 CUDA	51.6 TG128 @ 1 ctx	7,333 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M1 Max Metal	38.7 TG128 @ 1 ctx	508 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M1 Max Metal	38.6 TG128 @ 1 ctx	509 PP512	basecompute	20 Sep 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	35.6 TG128	1,774 PP512	arki05	11 Sep 2026	View Benchmark
basecompute/Llama-3.1-8B-Instruct BaseRT 0.2.4Q4	Apple M5 Pro Metal	35.5 TG128	1,773 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	Apple M4 Pro Metal	32.1 TG128 @ 1 ctx	392 PP512	basecompute	20 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	Apple M4 Pro Metal	31.9 TG128 @ 1 ctx	392 PP512	basecompute	11 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.3.0Q4	NVIDIA GB10 CUDA	25.5 TG128 @ 1 ctx	7,170 PP512	basecompute	9 Oct 2026	View Benchmark
Llama-3.1-8B-Instruct-Q8 BaseRT 0.2.6Q4	NVIDIA GB10 CUDA	25.4 TG128 @ 1 ctx	7,299 PP512	basecompute	19 Sep 2026	View Benchmark

Side B · llama.cpp

Top 16 of 16 runs by decode throughput.

Model / format	Chip / backend	Decode	Prefill	Contributor	Date	Report
Meta Llama 3.1 8B Instruct llama.cpp b1 (e64c0ea)Q4_K_M	Tesla T10/Tesla T10/Tesla T10/Tesla T10 CUDA	136.1 TG128	3,390 PP512	arki05	17 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_0	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	126.7 TG128	2,959 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	122.7 TG128	2,743 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	119.8 TG128	2,756 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_0	AMD Radeon RX 7900 XT ROCm	117.4 TG128	3,233 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	97.7 TG128	2,512 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	97.5 TG128	3,150 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	AMD Radeon RX 7900 XT ROCm	97.1 TG128	3,172 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q6_K	AMD Radeon RX 7900 XT ROCm	87.4 TG128	2,851 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q8_0	AMD Radeon RX 7900 XT (RADV NAVI31) Vulkan	68.6 TG128	2,286 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q8_0	AMD Radeon RX 7900 XT ROCm	66.2 TG128	3,292 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_0	Apple M5 Pro BLAS + Metal	58.7 TG128	1,538 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	55.6 TG128	1,424 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q4_K_M	Apple M5 Pro BLAS + Metal	55.5 TG128	1,423 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q6_K	Apple M5 Pro BLAS + Metal	43.4 TG128	1,421 PP512	arki05	11 Sep 2026	View Benchmark
Llama-3.1-8B-Instruct llama.cpp b10809 (5266f24da)Q8_0	Apple M5 Pro BLAS + Metal	29.6 TG128	1,465 PP512	arki05	11 Sep 2026	View Benchmark