Which hardware runs Llama 2 7B at 20 tokens/s or more?
Apple M5 Ultra (80-core GPU, 1228 GB/s): 179 tok/s (Llama 2 7B, Q4_0; grade C; 2026-10-01). Next: Apple M5 Max (40-core GPU, 614 GB/s) 120 tok/s; AMD Ryzen AI Max+ (Strix Halo) iGPU 54.38 tok/s. Like-for-like group: LLM 5–14B, batch 1.
| # | Platform | Result | Perf/W | Price | Grade · date | Source |
|---|---|---|---|---|---|---|
| 1 | Apple M5 Ultra (80-core GPU, 1228 GB/s) | 179 tok/s Llama 2 7B · Q4_0 | unknown | unknown | C 2026-10-01 | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| 2 | Apple M5 Max (40-core GPU, 614 GB/s) | 120 tok/s Llama 2 7B · Q4_0 | unknown | unknown | C 2026-10-01 | llama.cpp discussion #4167 scoreboard (via GitHub API) |
| 3 | AMD Ryzen AI Max+ (Strix Halo) iGPU | 54.38 tok/s Llama 2 7B · Q4_0 | 0.453 spec | $3999 | C 2026-10-10 | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| 4 | Apple M3 Ultra iGPU | 116 tok/s Llama 2 7B · Q4_0 | unknown | unknown | C 2026-10-10 | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
| 5 | NVIDIA DGX Spark | 60.39 tok/s Llama 2 7B · Q4_0 | 0.395 spec | $4699 | C 2026-10-10 | llama.cpp discussion #10879 Vulkan scoreboard (via GitHub API) |
Filters: task=LLM generation, model=llama 2 7b, fps=20. Ranking: boards that meet every known target, then headroom and perf/W; unknown power or price is not penalised. Open in /find · JSON
Sources
- https://github.com/ggml-org/llama.cpp/discussions/4167
- https://github.com/ggml-org/llama.cpp/discussions/10879
Generated 2026-10-16 from HWBench data only. All answers · Submit a result
Machine-readable: /find.json (see llms.txt); full tables are in “Show data”. Confidence grades: A audited MLPerf closed · B documented third-party · C community · D vendor-published · E estimate. Unknown values are shown as unknown, never estimated. Generated 2026-10-16 from public sources.