Task LLM generation · model-size class LLM ≤4B · batch 1 · 64 platforms
✓ Like-for-like: same model-size class and batch.
| Platform | Result | Model · precision · stack | Grade | Power | Price | Fit perf/power/budget | Source |
|---|---|---|---|---|---|---|---|
| Intel 13900K (Nvidia 5090) (as tested by Jeff Geerling) | 271 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel Core Ultra 7 368H | 250 tok/s 2nd-token throughput, input 32 tokens b=1 | minicpm4-0.5b INT4-MIXED · OpenVINO 2026.4.1 GenAI | D | unknown no power published | unknown | – / unknown / – | OpenVINO official LLM benchmarks (llm_models_7-368H.csv) |
| Ryzen 9 7900X (Nvidia 4090) (as tested by Jeff Geerling) | 237 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 13900K (Nvidia 4090) (as tested by Jeff Geerling) | 216 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Ryzen 9 7950X (Nvidia 4080) (as tested by Jeff Geerling) | 204 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Ryzen 9 7950X (Nvidia 4070 Ti Super) (as tested by Jeff Geerling) | 199 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Ryzen 9 5950X (Nvidia 4070) (as tested by Jeff Geerling) | 161 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Dell XPS 13 (Intel Core 5 320) (as tested by Jeff Geerling) | 23.88 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 14.00 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel Core 5 330 | 137 tok/s 2nd-token throughput, input 32 tokens b=1 | minicpm4-0.5b INT4-MIXED · OpenVINO 2026.4.1 GenAI | D | unknown no power published | unknown | – / unknown / – | OpenVINO official LLM benchmarks (llm_models_5-330.csv) |
| Intel Core Ultra 7 258V (Lunar Lake) | 135 tok/s 2nd-token throughput, input 32 tokens b=1 | minicpm4-0.5b INT4-MIXED · OpenVINO 2026.4.1 GenAI | D | unknown no power published | unknown | – / unknown / – | OpenVINO official LLM benchmarks (llm_models_7-258V.csv) |
| Ryzen 9 9950X (AMD 7900 XT) (as tested by Jeff Geerling) | 131 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Apple M4 (10-core GPU, 120 GB/s) | 41.31 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 30.10 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| M1 Ultra (64 GPU Core) 128GB (as tested by Jeff Geerling) | 128 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel Core Ultra 7 255H | 110 tok/s 2nd-token throughput, input 32 tokens b=1 | minicpm4-0.5b INT4-MIXED · OpenVINO 2026.4.1 GenAI | D | unknown no power published | unknown | – / unknown / – | OpenVINO official LLM benchmarks (llm_models_7-255H.csv) |
| M1 Ultra (48 GPU Core) 64GB (as tested by Jeff Geerling) | 109 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| NVIDIA Jetson Thor T4000 | 95.00 tok/s generation_throughput b=1 | Qwen3-VL-2B NVFP4 · vLLM | C | unknown no power published | unknown | – / unknown / – | NVIDIA Jetson AI Lab benchmarks (benchmarks.json) |
| M1 Max Mac Studio (10 core - 64GB) (as tested by Jeff Geerling) | 59.38 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | D | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Framework 13 (Ryzen AI 5 340) (as tested by Jeff Geerling) | 23.81 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 51.10 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Radxa Dragon Q8B (as tested by Jeff Geerling) | 12.30 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 24.30 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| MetaComputing AI PC Framework 13 Mainboard (as tested by Jeff Geerling) | 13.10 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 29.00 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 500+ - 16GB (as tested by Jeff Geerling) | 5.55 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 13.00 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Raspberry Pi 5 (CPU only) | 4.88 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 11.90 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Minisforum MS-R1 (as tested by Jeff Geerling) | 12.12 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 35.00 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| GMKtec G3 Plus (Intel N150) - 16GB (as tested by Jeff Geerling) | 9.06 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 26.40 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 500+ - 16GB (Intel Arc A310 ECO) (as tested by Jeff Geerling) | 13.36 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 50.00 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| NVIDIA Jetson Orin NX 16GB | 25.00 tok/s generation_throughput b=1 | Cosmos Reasoning 2 2B W4A16 · vLLM | C | unknown no power published | unknown | – / unknown / – | NVIDIA Jetson AI Lab benchmarks (benchmarks.json) |
| AmpereOne A192-32X (512GB) (as tested by Jeff Geerling) | 23.52 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | unknown no power published | unknown | – / unknown / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 400 - 4GB (as tested by Jeff Geerling) | 1.60 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 6.00 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| NVIDIA Jetson Orin Nano 8GB | 18.00 tok/s generation_throughput b=1 | Nemotron3 Nano 4B Q4_K_M GGUF · llama.cpp | C | unknown no power published | unknown | – / unknown / – | NVIDIA Jetson AI Lab benchmarks (benchmarks.json) |
| HiFive Premier P550 (4-core RISC-V) (as tested by Jeff Geerling) | 0.240 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 13.50 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| DC-ROMA Mainboard II (8-core RISC-V) (as tested by Jeff Geerling) | 0.310 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 30.60 W reported in result source — measured | unknown | – / yes / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (Nvidia RTX 4090) (as tested by Jeff Geerling) | 335 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 262 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi CM5 - 16GB (Nvidia RTX 4090) (as tested by Jeff Geerling) | 236 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 290 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (Nvidia RTX 3080 Ti) (as tested by Jeff Geerling) | 263 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 495 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (Nvidia RTX 4070 Ti) (as tested by Jeff Geerling) | 196 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 267 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi CM5 - 16GB (Nvidia RTX 3080 Ti) (as tested by Jeff Geerling) | 209 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 394 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (Nvidia RTX 2080 Ti) (as tested by Jeff Geerling) | 180 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 258 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (AMD Radeon AI Pro R9700) (as tested by Jeff Geerling) | 196 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 334 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi CM5 - 16GB (Nvidia RTX 4070 Ti) (as tested by Jeff Geerling) | 166 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 228 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| NVIDIA Jetson AGX Thor | 122 tok/s generation_throughput b=1 | Qwen3-VL-2B NVFP4 · vLLM | C | 130 W platform spec envelope (not measured with this result) | $3499 | – / no / – | NVIDIA Jetson AI Lab benchmarks (benchmarks.json) |
| Pi CM5 - 16GB (Nvidia RTX A4000) (as tested by Jeff Geerling) | 134 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 163 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Mac Studio (M3 Ultra 512GB) (as tested by Jeff Geerling) | 155 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 223 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi CM5 - 16GB (Nvidia RTX 2080 Ti) (as tested by Jeff Geerling) | 155 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 258 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (Nvidia RTX A4000) (as tested by Jeff Geerling) | 149 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 269 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Dell Pro Max with GB10 | 97.93 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 129 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Intel 265K Custom PC (Nvidia RTX 3060) (as tested by Jeff Geerling) | 123 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 214 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| NVIDIA Jetson AGX Orin 64GB | 60.00 tok/s generation_throughput b=1 | Qwen3-VL-2B W4A16 · vLLM | C | 65.00 W platform spec envelope (not measured with this result) | $1999 | – / no / – | NVIDIA Jetson AI Lab benchmarks (benchmarks.json) |
| Pi CM5 - 16GB (Nvidia RTX 3060) (as tested by Jeff Geerling) | 113 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 192 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| AMD Ryzen AI Max+ 395 / PRO 395 (Strix Halo, Radeon 8060S) | 88.14 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 133 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Minisforum MS-R1 (Nvidia RTX A2000) (as tested by Jeff Geerling) | 71.36 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 94.30 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 500+ - 16GB (AMD RX 7900 XT) (as tested by Jeff Geerling) | 109 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 315 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| System76 Thelio Astra (Nvidia A4000) (as tested by Jeff Geerling) | 90.92 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 244 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| System76 Thelio Astra (AMD Pro W7700) (as tested by Jeff Geerling) | 89.31 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 261 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 500+ - 16GB (AMD RX 9070 XT) (as tested by Jeff Geerling) | 89.63 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 304 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 5 - 8GB (AMD RX 6700 XT) 12GB (as tested by Jeff Geerling) | 49.01 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 94.00 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi CM5 - 16GB (AMD AI Pro R9700) (as tested by Jeff Geerling) | 77.54 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 281 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 5 - 8GB (AMD Pro W7700) (as tested by Jeff Geerling) | 56.14 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 145 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 5 - 8GB (AMD RX 6500 XT) (as tested by Jeff Geerling) | 39.82 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 88.00 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 500+ - 16GB (Intel Arc B580) (as tested by Jeff Geerling) | 47.38 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 146 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 5 - 8GB (AMD RX 7600) (as tested by Jeff Geerling) | 48.47 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 156 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Pi 500+ - 16GB (Intel Arc Pro B50) (as tested by Jeff Geerling) | 29.80 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · llama.cpp (Vulkan) | C | 78.50 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| HiFive Premier P550 (AMD RX 580) (as tested by Jeff Geerling) | 36.23 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 150 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| System76 Thelio Astra (Nvidia A400) (as tested by Jeff Geerling) | 35.51 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 167 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
| Dell Optiplex 780 (C2Q Q8400) (as tested by Jeff Geerling) | 1.09 tok/s generation_throughput b=1 | Llama 3.2:3b as in linked issue (Ollama default tags are 4-bit) · Ollama / llama.cpp (see linked issue) | C | 146 W reported in result source — measured | unknown | – / no / – | Jeff Geerling ai-benchmarks README (Findings tables) |
Confidence: A audited MLPerf closed, B documented third-party, C community, D vendor-published, E estimate. Unknown is never assumed. Other like-for-like groups: LLM ≤4B b1 (64); LLM 5–14B b1 (53); LLM 15–40B b1 (22); LLM >40B b1 (11); LLM 5–14B b8 (4); LLM ≤4B b8 (4); LLM 15–40B b8 (3); LLM >40B b8 (1); LLM >40B bunknown (4); LLM 15–40B bunknown (1); unknown b1 (8); unknown b8 (4)