vision-language models (VLM) benchmarks on edge AI hardware
78 published rows. Each table compares only results with the same model-size class and batch size. Search this workload · JSON.
Fastest in the best-covered like-for-like group (LLM ≤4B, batch 1): NVIDIA Jetson AGX Thor at 71.70 tok/s D on or before 2025-11-08. 5 groups, 3 platforms.
Show data — 5 like-for-like tables
LLM ≤4B · batch 1 (3 platforms)
✓ Like-for-like: model-size class LLM ≤4B, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA Jetson AGX Thor | 71.70 tok/s tok/s gen, concurrency 1 · b=1 | Qwen2.5-VL 3B n/s on page · NVIDIA (vLLM/TensorRT-LLM per page) · LLM ≤4B | D | 130 W spec platform spec envelope (not measured with this result) | NVIDIA Jetson Benchmarks on or before 2025-11-08 |
| Axelera Metis Edge 130p PCIe card (16GB) | 7.17 tok/s generation_throughput · b=1 | lfm2_5_vl_3b not stated · Voyager SDK model_axscript.llm · LLM ≤4B | C | unknown power n/a no power published | Axelera Voyager SDK model zoo (docs/reference/models/model-zoo.md) 2026-10-15 (page last edited) |
| Axelera Metis Embedded 113m M.2 card | 7.16 tok/s generation_throughput · b=1 | lfm2_5_vl_3b not stated · Voyager SDK model_axscript.llm · LLM ≤4B | C | unknown power n/a no power published | Axelera Voyager SDK model zoo (docs/reference/models/model-zoo.md) 2026-10-15 (page last edited) |
LLM ≤4B · batch 8 (1 platforms)
✓ Like-for-like: model-size class LLM ≤4B, batch 8. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA Jetson AGX Thor | 357 tok/s tok/s gen, concurrency 8 · b=8 | Qwen2.5-VL 3B n/s on page · NVIDIA (vLLM/TensorRT-LLM per page) · LLM ≤4B | D | 130 W spec platform spec envelope (not measured with this result) | NVIDIA Jetson Benchmarks on or before 2025-11-08 |
LLM 5–14B · batch 1 (1 platforms)
✓ Like-for-like: model-size class LLM 5–14B, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA Jetson AGX Thor | 45.00 tok/s tok/s gen, concurrency 1 · b=1 | Qwen2.5-VL 7B n/s on page · NVIDIA (vLLM/TensorRT-LLM per page) · LLM 5–14B | D | 130 W spec platform spec envelope (not measured with this result) | NVIDIA Jetson Benchmarks on or before 2025-11-08 |
LLM 5–14B · batch 8 (1 platforms)
✓ Like-for-like: model-size class LLM 5–14B, batch 8. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA Jetson AGX Thor | 252 tok/s tok/s gen, concurrency 8 · b=8 | Qwen2.5-VL 7B n/s on page · NVIDIA (vLLM/TensorRT-LLM per page) · LLM 5–14B | D | 130 W spec platform spec envelope (not measured with this result) | NVIDIA Jetson Benchmarks on or before 2025-11-08 |
unknown · batch 1 (2 platforms)
✓ Like-for-like: model-size class unknown, batch 1 (size or batch not stated by source — unverified). Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Axelera Metis Edge 130p PCIe card (16GB) | 42.30 tok/s generation_throughput · b=1 | smolvlm2_256m_video_instruct not stated · Voyager SDK model_axscript.llm · unknown | C | unknown power n/a no power published | Axelera Voyager SDK model zoo (docs/reference/models/model-zoo.md) 2026-10-15 (page last edited) |
| Axelera Metis Embedded 113m M.2 card | 39.90 tok/s generation_throughput · b=1 | smolvlm2_256m_video_instruct not stated · Voyager SDK model_axscript.llm · unknown | C | unknown power n/a no power published | Axelera Voyager SDK model zoo (docs/reference/models/model-zoo.md) 2026-10-15 (page last edited) |
How to read these numbers
Every number is labelled. Confidence: A MLPerf closed, audited · B documented third-party method · C community / review · D vendor-published, not independently verified · E estimate. Power basis: measured with the result · spec vendor spec/envelope · power n/a not published.
Machine-readable: /find.json (see llms.txt); full tables are in “Show data”. Confidence grades: A audited MLPerf closed · B documented third-party · C community · D vendor-published · E estimate. Unknown values are shown as unknown, never estimated. Generated 2026-10-16 from public sources.