Nvidia A400 (as tested by Jeff Geerling): edge AI benchmarks
- Independent test Jeff Geerling
Key results
How to read power: results are compared only within the same model. By efficiency ranks only results whose source publishes a power figure; By speed ranks every result. Measured = measured for this workload. Vendor-measured (typical/avg) = the vendor’s own measurement at typical or average load, not an independent test and not a maximum, so fps/W looks better than at max TDP. Chip, Module or Board TDP = the vendor’s published maximum, not measured. Typical power, not max TDP = only a typical figure is published, so fps/W looks better than at full load. Scope says what the watts cover: Accelerator only (host not included), Module, or Whole system. Max of range = top of a published range, so fps/W is a floor (≥). “— not published” = the source gives no power; we never estimate one.
| Workload | Model · precision | Result | Power | Perf/W | Rank on this model | Grade · date | Source |
|---|---|---|---|---|---|---|---|
| LLM generation | llama2:13bOllama default tag (4-bit unless tag says otherwise) · batch 1 | 7.93 tok/s | 223 WWhole system Measured | 0.0356● Measured · Whole system | not in the default fps/W ranking #3 of 6 by speed | C 2024-10-22 (page last edited) | Jeff Geerling sbc-reviews (AI / LLM inference tables)ai-benchmarks README @ 2f56f19 |
| llama3.2:3bOllama default tag (4-bit unless tag says otherwise) · batch 1 | 35.51 tok/s | 167 WWhole system Measured | 0.213● Measured · Whole system | not in the default fps/W ranking #15 of 35 by speed | C 2024-10-22 (page last edited) | Jeff Geerling sbc-reviews (AI / LLM inference tables)ai-benchmarks README @ 2f56f19 | |
| llama3.2:8bOllama default tag (4-bit unless tag says otherwise) · batch 1 | 2.79 tok/s | 190 WWhole system Measured | 0.0147● Measured · Whole system | not in the default fps/W ranking #1 of 2 by speed | C 2024-10-22 (page last edited) | Jeff Geerling sbc-reviews (AI / LLM inference tables)ai-benchmarks README @ 2f56f19 |
Ranks compare this board only with boards running the same model. Up to three models per workload; open a rank to see the full same-model table.
Show data — per-workload tables
object detection
No published object detection results for Nvidia A400 (as tested by Jeff Geerling) yet.
We only show cited public figures — nothing is estimated to fill gaps. Requests tell us what to source or test next.
instance segmentation
No published instance segmentation results for Nvidia A400 (as tested by Jeff Geerling) yet.
We only show cited public figures — nothing is estimated to fill gaps. Requests tell us what to source or test next.
pose estimation
No published pose estimation results for Nvidia A400 (as tested by Jeff Geerling) yet.
We only show cited public figures — nothing is estimated to fill gaps. Requests tell us what to source or test next.
image classification
No published image classification results for Nvidia A400 (as tested by Jeff Geerling) yet.
We only show cited public figures — nothing is estimated to fill gaps. Requests tell us what to source or test next.
semantic segmentation
No published semantic segmentation results for Nvidia A400 (as tested by Jeff Geerling) yet.
We only show cited public figures — nothing is estimated to fill gaps. Requests tell us what to source or test next.
speech-to-text
No published speech-to-text results for Nvidia A400 (as tested by Jeff Geerling) yet.
We only show cited public figures — nothing is estimated to fill gaps. Requests tell us what to source or test next.
LLM generation
| Like-for-like group | Best result | Model · precision · stack | Grade | Power | Source · date | Rank in group |
|---|---|---|---|---|---|---|
| LLM ≤4B · b=1 | 35.51 tok/s | llama3.2:3b Ollama default tag (4-bit unless tag says otherwise) · Ollama (geerlingguy/ollama-benchmark) | C | 167 W ● Measured, Whole system | Jeff Geerling sbc-reviews (AI / LLM inference tables) Older data (2024) | 53 of 81 |
| LLM 5–14B · b=1 | 7.93 tok/s | llama2:13b Ollama default tag (4-bit unless tag says otherwise) · Ollama (geerlingguy/ollama-benchmark) | C | 223 W ● Measured, Whole system | Jeff Geerling sbc-reviews (AI / LLM inference tables) Older data (2024) | 54 of 66 |
Grades: A audited MLPerf · B third-party · C community · D vendor. How we rank · JSON