Model benchmarks

Every config ranked for coding work on one RTX 3090. The score weights coding 50%, agents & tools 30% and reasoning 20%, and combines them as a weighted geometric mean, so being weak in any one area pulls it down. Answers must finish within a thinking allowance taken from each config’s own measured speed, so faster configs get more room to reason. How it's measured

Coding work, every config

Rows are ranked by score. Each row has three lines, top to bottom: coding, agents & tools, reasoning. Hover or tap a mark for details, or tab through the rows. The table below lists every number.

CodingLiveCodeBenchSWE-bench VerifiedAgents & toolsBFCLTerminal-BenchReasoningGPQA DiamondAIME 2025Score ± 1 standard errorGeneration, deepest context measured · t/s

Capability benchmarks

Percent of tasks solved. The score carries ± one standard error, and rows sharing a rank are too close to call.

#	Model · config	Score	Coding · 50%		Agents & tools · 30%		Reasoning · 20%		Speed
#	Model · config	Score	LiveCodeBench	SWE-bench Verified	BFCL	Terminal-Bench	GPQA Diamond	AIME 2025	Generation	Thinking allowance	Fixed / night
Not ranked yet · quick tier only
–	Qwen3.8 27B W4A16 AutoRoundquick tier only 64k ctx, DFlash2 speculative decoding	quick 71.5	60.0	—	87.5	—	81.8	82.5	146 @0	42.2k tokens	—
–	Gemma 3 4B IT Q4_K_Mquick tier only 32k ctx, q8_0 KV	quick 26.0	18.0	—	53.3	—	32.3	12.5	156 @16k	24.8k tokens (context limit)	—

4 configs have speed numbers but no capability evals yet; they are on the Speed tab.

How the score works

Quick tier, every config: LiveCodeBench (100 problems), BFCL (400 cases), GPQA Diamond (198 questions) and AIME 2025 (30 problems × 4 attempts). Limit: 5 minutes per task.
Deep tier, only for configs worth a night each: SWE-bench Verified (30 issues, solved by mini-SWE-agent) and Terminal-Bench (30 tasks). Limit: 20 minutes per task. Configs without it aren't ranked, because quick-tier scores aren't comparable with full ones.
Thinking allowance = (time limit − time to read the prompt) × generation speed at that depth, capped by the config's context. The table shows it for an 8k-token prompt in a 5-minute task. An answer that doesn't finish inside it counts as wrong.
Too close to call: scores within one combined standard error of each other share a rank, shown as "=2".
Issues fixed per night = 8 hours ÷ average time per SWE-bench issue × fix rate.