← Gemma 3 4B IT Q4_K_M · 32k ctx, q8_0 KV

Run a2caf10e

Kind	speed (llama-bench)
Started	2026-09-15 18:42 UTC
Duration	37 s
Engine	`llama.cpp@91f6a6cf3` (build 10883)
Driver	595.99.02
Throttled	no
Command	`lab bench gemma-3-4b-it-q4_k_m-gguf/32k-q8kv --wait`
Lab version	`0.1.0+16df55c`

Metrics

Metric	Method	Prompt	Gen	Depth	Value	± sd	Unit
`gpu_w_avg`	llama-bench	—	128	0	210	—	W
`gpu_w_avg`	llama-bench	—	128	4k	214	—	W
`gpu_w_avg`	llama-bench	512	—	4k	196	—	W
`gpu_w_avg`	llama-bench	—	128	16k	222	—	W
`gpu_w_avg`	llama-bench	512	—	16k	211	—	W
`pp_tps`	llama-bench	512	—	0	7,855	574	t/s
`pp_tps`	llama-bench	512	—	4k	7,369	426	t/s
`pp_tps`	llama-bench	512	—	16k	6,598	332	t/s
`proc_rss_peak_mb`	llama-bench	—	—	16k	2,761	—	MiB
`ram_peak_mb`	llama-bench	—	—	16k	16,252	—	MiB
`tg_tps`	llama-bench	—	128	0	166	0.44	t/s
`tg_tps`	llama-bench	—	128	4k	161	0.66	t/s
`tg_tps`	llama-bench	—	128	16k	156	0.60	t/s
`tokens_per_joule`	llama-bench	—	128	0	0.79	—	tok/J
`tokens_per_joule`	llama-bench	—	128	4k	0.75	—	tok/J
`tokens_per_joule`	llama-bench	512	—	4k	37.7	—	tok/J
`tokens_per_joule`	llama-bench	—	128	16k	0.70	—	tok/J
`tokens_per_joule`	llama-bench	512	—	16k	31.2	—	tok/J
`vram_peak_mb`	llama-bench	—	—	16k	3,474	—	MiB

Raw

{
  "bench": {
    "n_prompt": 512,
    "n_gen": 128,
    "depths": [
      0,
      4096,
      16384
    ],
    "reps": 5,
    "delay_s": 3,
    "http_prompt_lengths": [
      512,
      4096
    ]
  },
  "telemetry_summary": {
    "vram_baseline_mb": 0.9375,
    "vram_peak_mb": 3475.125,
    "ram_baseline_mb": 14929.046875,
    "ram_peak_mb": 16252.12890625,
    "proc_rss_peak_mb": 2760.8046875,
    "temp_max_c": 52,
    "power_max_w": 249.645,
    "clock_max_mhz": 1620,
    "throttled": false,
    "power_capped_frac": 0.43956043956043955,
    "phases": {
      "llama-bench": {
        "duration_s": 36.45752109802561,
        "energy_j": 6060.605,
        "power_avg_w": 166.2374406560576
      }
    }
  }
}