Qwen3.8 27B W4A16 AutoRound

Qwen3.8 27B · 27.3B dense · vLLM · SAFETENSORS W4A16 · 15.81 GB · dbirks/Qwen3.8-27B-W4A16-AutoRound @1f05c44

Qwen's Qwen3.8 27B, quantized to 4-bit weights by dbirks (AutoRound). The syv-ai/qwen38-27b-rtx3090 scripts requantize its lm_head, embeddings and multi-token-prediction head and add syv-ai's fast-variant shards; it serves on syv-ai's patched vLLM 0.28.0 with their DFlash2 W4A16 draft model.

Chat generation

146 t/s

default sampling ± 46.8

Chat generation, greedy

152 t/s

± 50.9

Time to first token

137 ms

Peak VRAM

23.5 GB

preallocated at startup

GPU power while generating

249 W

Energy efficiency

0.57 tok/J

chat benchmark

Settings

Hosted chat config: syv-ai's single-user profile SPEC=dflash2 PREFIX_CACHE=1 CTX=fast (64k context, bf16 KV cache in a 5.2 GiB pool, 8 request slots).

CUDA_HOME=/usr/local/cuda HOST=127.0.0.1 PORT=8080 MODEL=models/Qwen3.8-27B-W4A16-AutoRound-fast SPEC=dflash2 CTX=fast MAX_LEN=65536 PREFIX_CACHE=1 KV_MEM=5583457484 MAX_SEQS=8 GPU_UTIL=0.93 DFLASH_TOKENS=7 LOOKUP=1 TOOLS=1 VISION=0 DRAFT=models/Qwen3.8-27B-DFlash2-W4A16 'EXTRA_ARGS=--served-model-name qwen3.8-27b-w4a16-autoround-fast/64k-dflash2 --load-format auto' single-user/start_qwen.sh

Setting	Value
`launcher`	single-user/start_qwen.sh
`launcher_commit`	bae2023ffc98753d337d2d2041784a277599a4c4
`ctx`	65536
`spec`	dflash2
`ctx_profile`	fast
`prefix_cache`	true
`kv_mem`	5583457484
`max_seqs`	8
`gpu_util`	0.93
`dflash_tokens`	7
`lookup`	true
`tools`	true
`vision`	false
`cudagraph_mode`	—
`draft`	models/Qwen3.8-27B-DFlash2-W4A16
`load_format`	auto
`env`	{}
`extra_args`	[]

Chat benchmark

Eight real chat prompts sent through the server's HTTP API, one at a time, up to 1,024 tokens each, thinking off. Every engine runs the same protocol, so these numbers compare across engines. How it's measured

Sampling	Decode t/s	± sd	TPOT ms	TTFT ms	Prefill t/s	End-to-end t/s	Output tokens	GPU W	Tok/J	Accepted / step
default sampling · 8 prompts	146	46.8	6.86	137	741	141	938	249	0.57	3.71
greedy · 8 prompts	152	50.9	6.58	140	723	147	951	250	0.59	3.84

Load time 62 s · Peak VRAM 23.5 GB (preallocated: vLLM reserves its KV-cache pool at startup) · Peak RAM 23.9 GB · Run of 2026-09-17 17:16

Chat generation, every Qwen3.8 27B config

Same prompts and settings on every engine. Click a bar to open that config.

GPU power during the run

VRAM during the run

Quality

Not measured yet: perplexity and KL-divergence against a reference quant.

Capability

Quick-tier score

71.5 ± 3.0

not ranked: the deep tier has not run

Coding

60.0 % solved

50% of the score

Agents & tools

87.5 % solved

30% of the score

Reasoning

82.2 % solved

20% of the score

Thinking allowance

42.2k tokens

8.0k-token prompt, 5-minute task

Benchmark	Tier	Solved	Tasks	Allowance	Length stops	s / task	Harness	Run
LiveCodeBench Coding · pass@1 · 100 problems	quick	60.0 ± 4.9	100	43,628	32	213	livecodebench 28fef95ea8c9f7a547c8329f2cd3d32b92c1fa24	2026-09-17 23:05
SWE-bench Verified Coding · resolved · 30 issues	deep	—	—	—	—	—	—	—
BFCL Agents & tools · accuracy · 400 cases	quick	87.5 ± 1.7	400	43,091	0	8	bfcl-eval 2026.3.23 native tool calls	2026-09-17 23:05
Terminal-Bench Agents & tools · resolved · 30 tasks	deep	—	—	—	—	—	—	—
GPQA Diamond Reasoning · accuracy · 198 questions	quick	81.8 ± 2.7	198	43,698	23	123	lab 0.1.0+80d380e	2026-09-17 23:05
AIME 2025 Reasoning · mean accuracy · 30 problems × 4 attempts	quick	82.5 ± 6.0	30 × 4	43,718	18	595	lab 0.1.0+80d380e	2026-09-17 23:05

Run history

Date	Kind	Engine build	Driver	Duration	Flags
2026-09-17 23:05	evals (quick)	`vllm@bae2023ff`	595.99.02	1,112.9 min
2026-09-17 17:16	chat benchmark	`vllm@bae2023ff`	595.99.02	3.8 min