LLM prefill benchmarks on edge AI hardware
41 published rows. Each table compares only results with the same model-size class and batch size. Search this workload · JSON.
Fastest in the best-covered like-for-like group (LLM 5–14B, batch 1): Apple M5 Ultra (80-core GPU, 1228 GB/s) at 4945 tok/s C 2026-10-01 (page last edited). 3 groups, 22 platforms.
Show data — 3 like-for-like tables
LLM 5–14B · batch 1 (19 platforms)
✓ Like-for-like: model-size class LLM 5–14B, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Apple M5 Ultra (80-core GPU, 1228 GB/s) | 4945 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit c1d0e7a) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| NVIDIA DGX Spark | 3322 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit 657e011) · LLM 5–14B | C | 153 W spec platform spec envelope (not measured with this result) | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M5 Max (40-core GPU, 614 GB/s) | 3220 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit c1d0e7a) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| AMD Ryzen AI Max+ (Strix Halo) iGPU | 1704 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit fb4b273) · LLM 5–14B | C | 120 W spec platform spec envelope (not measured with this result) | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M5 Pro (20-core GPU, 307 GB/s) | 1621 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit c1d0e7a) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| Intel Core Ultra 300 Series iGPU | 1380 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 481c65f) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M5 Pro (16-core GPU, 307 GB/s) | 1340 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit 60081bb) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| Apple M3 Ultra iGPU | 1117 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 2d451c8) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M4 Max (40-core GPU, 546 GB/s) | 892 tok/s pp512 · b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| Intel Core Ultra 200 Series iGPU | 867 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit 4a89937) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M4 Max iGPU | 725 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 1ece0cb6) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M4 Max (32-core GPU, 410 GB/s) | 719 tok/s pp512 · b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| AMD Ryzen AI 300 Series iGPU | 533 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit N/A) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| Apple M4 Pro (20-core GPU, 273 GB/s) | 450 tok/s pp512 · b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| Apple M4 Pro (16-core GPU, 273 GB/s) | 367 tok/s pp512 · b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| Apple M4 (10-core GPU, 120 GB/s) | 224 tok/s pp512 · b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #4167 scoreboard (via GitHub API) 2026-10-01 (page last edited) |
| Intel Core Ultra 100 Series iGPU | 186 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 1d72c84) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| AMD Ryzen AI 400 Series iGPU | 150 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit cb26014) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
| MediaTek Dimensity 9400 iGPU | 38.36 tok/s pp512 · b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit b9ab0a4) · LLM 5–14B | C | unknown power n/a no power published | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) 2026-10-10 (page last edited) |
LLM >40B · batch unknown (4 platforms)
✓ Like-for-like: model-size class LLM >40B, batch unknown (size or batch not stated by source — unverified). Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| AMD Ryzen AI Max+ 395 / PRO 395 (Strix Halo, Radeon 8060S) | 676 tok/s pp512 · b=unknown | Qwen3-Next 80B-A3B UD-Q4_K_XL · llama.cpp Vulkan RADV b10687 · LLM >40B | C | 120 W spec platform spec envelope (not measured with this result) | StrixHaloGuide evidence (llama-bench, 20 repeats) 2026-10-02 (page last edited) |
| NVIDIA DGX Spark | 283 tok/s pp · b=unknown | 70B-class LLM (Q4) ~Q4 · as reviewed · LLM >40B | C | 153 W measured reported in result source — measured | vminstall.com review 2026-03-20 (page last edited) |
| NVIDIA Jetson AGX Thor | 130 tok/s pp · b=unknown | 70B-class LLM (Q4) ~Q4 · as reviewed · LLM >40B | C | 100 W measured reported in result source — measured | vminstall.com review 2026-03-20 (page last edited) |
| Apple Mac mini M4 Pro 64GB | 42.00 tok/s pp · b=unknown | 70B-class LLM (Q4) ~Q4 · as reviewed · LLM >40B | C | 80.00 W measured reported in result source — measured | vminstall.com review 2026-03-20 (page last edited) |
LLM 15–40B · batch unknown (1 platforms)
✓ Like-for-like: model-size class LLM 15–40B, batch unknown (size or batch not stated by source — unverified). Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| AMD Ryzen AI Max+ 395 / PRO 395 (Strix Halo, Radeon 8060S) | 1264 tok/s pp512 · b=unknown | Qwen3-Coder 30B-A3B UD-Q4_K_XL · llama.cpp Vulkan RADV b10687 · LLM 15–40B | C | 120 W spec platform spec envelope (not measured with this result) | StrixHaloGuide evidence (llama-bench, 20 repeats) 2026-10-02 (page last edited) |
How to read these numbers
Every number is labelled. Confidence: A MLPerf closed, audited · B documented third-party method · C community / review · D vendor-published, not independently verified · E estimate. Power basis: measured with the result · spec vendor spec/envelope · power n/a not published.
Machine-readable: /find.json (see llms.txt); full tables are in “Show data”. Confidence grades: A audited MLPerf closed · B documented third-party · C community · D vendor-published · E estimate. Unknown values are shown as unknown, never estimated. Generated 2026-10-16 from public sources.