Find edge AI hardware
Task LLM prefill · model-size class LLM 5–14B · batch 1 · 19 platforms
Not like-for-like across groups: same model-size class and batch, but 2 different model/precision combinations. Compare within a group; each group is sorted by result.
Llama 2 7B · Q4_0 (14)
| Platform | Result | Model · precision · stack | Grade · date | Power | Price | Source |
|---|---|---|---|---|---|---|
| Apple M5 Ultra (80-core GPU, 1228 GB/s) | 4945 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit c1d0e7a) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| NVIDIA DGX Spark | 3322 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit 657e011) | C 2026-10-10 (page last edited) | 153 W platform spec envelope (not measured with this result) | $6950 | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| Apple M5 Max (40-core GPU, 614 GB/s) | 3220 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit c1d0e7a) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| AMD Ryzen AI Max+ (Strix Halo) iGPU | 1704 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit fb4b273) | C 2026-10-10 (page last edited) | 120 W platform spec envelope (not measured with this result) | $3999 | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| Apple M5 Pro (20-core GPU, 307 GB/s) | 1621 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit c1d0e7a) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| Intel Core Ultra 300 Series iGPU | 1380 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 481c65f) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| Apple M5 Pro (16-core GPU, 307 GB/s) | 1340 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Metal (commit 60081bb) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| Apple M3 Ultra iGPU | 1117 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 2d451c8) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| Intel Core Ultra 200 Series iGPU | 867 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit 4a89937) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| Apple M4 Max iGPU | 725 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 1ece0cb6) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| AMD Ryzen AI 300 Series iGPU | 533 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit N/A) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| Intel Core Ultra 100 Series iGPU | 186 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit 1d72c84) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| AMD Ryzen AI 400 Series iGPU | 150 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA on, commit cb26014) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| MediaTek Dimensity 9400 iGPU | 38.36 tok/s pp512 b=1 | Llama 2 7B Q4_0 · llama.cpp Vulkan (FA off, commit b9ab0a4) | C 2026-10-10 (page last edited) | unknown no power published | unknown | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
Llama 2 7B · Q8_0 (5)
| Platform | Result | Model · precision · stack | Grade · date | Power | Price | Source |
|---|---|---|---|---|---|---|
| Apple M4 Max (40-core GPU, 546 GB/s) | 892 tok/s pp512 b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| Apple M4 Max (32-core GPU, 410 GB/s) | 719 tok/s pp512 b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| Apple M4 Pro (20-core GPU, 273 GB/s) | 450 tok/s pp512 b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| Apple M4 Pro (16-core GPU, 273 GB/s) | 367 tok/s pp512 b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| Apple M4 (10-core GPU, 120 GB/s) | 224 tok/s pp512 b=1 | Llama 2 7B Q8_0 · llama.cpp Metal (commit 8e672ef) | C 2026-10-01 (page last edited) | unknown no power published | unknown | llama.cpp discussion #4167 scoreboard (via GitHub API) |
Confidence: A audited MLPerf closed, B documented third-party, C community, D vendor-published, E estimate. Unknown is never assumed. Other like-for-like groups: LLM 5–14B, batch 1 (19); LLM >40B, batch not stated (4); LLM 15–40B, batch not stated (1)