Methodology

Methodology (phase 1)

Scope: every number is a published fact from a public source, retrieved 2026-10-15. Nothing was measured by us. Each row stores: hardware, compute unit, software stack, model, precision, batch, metric, value, unit, ≈FPS, power (W) and where it was measured, perf/W, confidence, source URL and date.

Confidence grades

• A: MLPerf Inference v6.0 closed division (peer-reviewed submission rules). Source: mlcommons/inference_results_v6.0 (Apache-2.0), summary_results.json.
• B: third party with a documented, repeatable method: Ultralytics official benchmark tables (Jetson, Raspberry Pi, DGX Spark), and MLPerf open division.
• C: community scoreboards and independent reviews: llama.cpp discussions #4167 / #10879 (read through the GitHub API), vminstall, OpenNav, StrixHaloGuide. Only figures and links are stored.
• D: vendor-published: NVIDIA Jetson Benchmarks, Hailo Model Zoo (MIT), Qualcomm AI Hub Models perf.yaml (BSD-3; we keep the fastest runtime per device and precision).
• E: estimate. Applies to any perf/W that uses spec or "typical" power rather than power measured for that run (flagged in perf_per_w_basis).

Normalisation

• ≈FPS = 1000 / latency (ms) for single-stream or batch-1 results, or the published FPS / samples/s. Batch-8 or offline throughput is labelled as such and is not the same as real-time latency.
• Perf/W = ≈FPS (or tok/s) ÷ power. The measurement point (wall / system / module / chip typical / spec max) is always shown. Hailo figures use chip-typical power and exclude the host. Orin Nano Super figures use its 25 W maximum mode, so that perf/W is a floor.
• LLM rows separate prefill (pp) from generation (tg). Llama 2 7B Q4_0 scoreboards are kept for cross-device comparability, not as a current-model guide.

Excluded on purpose

Geekbench (its EULA bans automated access), TokenMark (not scraped), and long text copied from reviews. Before adding any new source, check its terms.

Known limitations

• Models, input sizes, precisions and runtimes differ across vendors. Treat cross-vendor rows as indicative.
• Hailo throughput was measured on an x86 host over PCIe Gen3 x4. A Raspberry Pi 5 AI HAT uses Gen3 x1 and may be slower.
• MLPerf v6.0 edge has no power submissions. The NVIDIA Jetson MLPerf table (with system power) comes from an older, unstated round.

Machine-readable: /find.json (see llms.txt); full tables are in “Show data”. Confidence grades: A audited MLPerf closed · B documented third-party · C community · D vendor-published · E estimate. Unknown values are shown as unknown, never estimated. Generated 2026-10-16 from public sources.