Qwen3.8 27B Q4_K_M

Qwen3.8 27B · 27.3B dense · llama.cpp · GGUF Q4_K_M · 17.44 GB · bartowski/Qwen3.8-27B-GGUF @125a02a

Chat generation

30.6 t/s

default sampling ± 0.05

Chat generation, greedy

30.5 t/s

± 0.03

Time to first token

225 ms

Peak VRAM

18.3 GB

serving the chat benchmark

GPU power while generating

249 W

Energy efficiency

0.12 tok/J

chat benchmark

Settings

Hosted chat config. Its llama-bench numbers match 32k-q8kv (llama-bench takes no -c); it is benchmarked so the hosted config has a published run.

llama-server -m Qwen3.8-27B-Q4_K_M.gguf --host 127.0.0.1 --port 8080 -a qwen3.8-27b-q4_k_m-gguf/64k-q8kv --fit off -c 65536 -np 1 --cache-ram 0 -ngl 999 -fa on -ctk q8_0 -ctv q8_0 -b 2048 -ub 512 -t 6 -ncmoe 0 -lm mmap --metrics --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 --presence-penalty 0

Setting	Value
`ctx`	65536
`gpu_layers`	999
`flash_attn`	on
`cache_type_k`	q8_0
`cache_type_v`	q8_0
`batch`	2048
`ubatch`	512
`threads`	6
`n_cpu_moe`	0
`parallel`	1
`load_mode`	mmap
`cache_ram_mib`	0
`spec`	—
`reasoning_format`	—
`reasoning_budget`	—
`extra_args`	["--temp","1.0","--top-p","0.95","--top-k","20","--min-p","0","--presence-penalty","0"]

Highlighted rows differ between this model's configs.

Chat benchmark

Eight real chat prompts sent through the server's HTTP API, one at a time, up to 1,024 tokens each, thinking off. Every engine runs the same protocol, so these numbers compare across engines. How it's measured

Sampling	Decode t/s	± sd	TPOT ms	TTFT ms	Prefill t/s	End-to-end t/s	Output tokens	GPU W	Tok/J
default sampling · 8 prompts	30.6	0.05	32.7	225	450	30.4	947	249	0.12
greedy · 8 prompts	30.5	0.03	32.8	225	449	30.3	948	250	0.12

Load time 18 s · Peak VRAM 18.3 GB · Peak RAM 23.9 GB · Run of 2026-09-17 17:21

Chat generation, every Qwen3.8 27B config

Same prompts and settings on every engine. Click a bar to open that config.

Speed by context depth

llama.cpp's own benchmark, llama-bench, with no server in the loop.

Generation vs context depth

All configs of this model; the selected one is solid.

Prompt processing vs context depth

Test	Depth	t/s	± sd	Reps	GPU W	Tok/J
pp512 llama-bench	0	949	73.6	5	151	6.29
pp512 llama-bench	4k	1,091	25.1	5	227	4.81
pp512 llama-bench	16k	987	15.4	5	240	4.11
tg128 llama-bench	0	31.8	0.24	5	246	0.13
tg128 llama-bench	4k	30.9	0.21	5	245	0.13
tg128 llama-bench	16k	28.2	0.12	5	244	0.12

GPU power during the run

VRAM during the run

Quality

Not measured yet: perplexity and KL-divergence against a reference quant.

Capability

Not measured yet. The quick tier (LiveCodeBench, BFCL, GPQA Diamond, AIME 2025) runs for every config; SWE-bench Verified and Terminal-Bench run for the configs worth a night each.

Run history

Date	Kind	Engine build	Driver	Duration	Flags
2026-09-17 17:21	chat benchmark	`llama.cpp@91f6a6cf3`	595.99.02	12.8 min
2026-09-15 18:47	speed (llama-bench)	`llama.cpp@91f6a6cf3`	595.99.02	2.8 min