recommendation benchmarks on edge AI hardware
14 published rows. Each table compares only results with the same model-size class and batch size. Search this workload · JSON.
Fastest in the best-covered like-for-like group (unknown, batch unknown): NVIDIA GB200 NVL72 (4x NVIDIA GB200) at 104637 FPS A 2026-04-01 (published). 1 groups, 7 platforms.
Show data — 1 like-for-like tables
unknown · batch unknown (7 platforms)
✓ Like-for-like: model-size class unknown, batch unknown (size or batch not stated by source — unverified). Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA GB200 NVL72 (4x NVIDIA GB200) | 104637 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 fp16 · TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0, closed division, data-centre suite 2026-04-01 (published) |
| 8xMI355X_2xEPYC_9575F (8x AMD Instinct MI355X) | 16363 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 fp16,bf16,fp8 · PyTorch 2.10.0+rocm7.2.3.git1a270074, Triton 3.6.0, ROCm 7.2.3 · unknown | B | unknown power n/a no power published | MLPerf Inference v6.1, open division, data-centre suite 2026-09-16 (published) |
| smci355-ccs-aus-m13-13 (8x AMD Instinct MI355X 288GB HBM3e) | 13028 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 fp16,bf16,fp8 · PyTorch 2.10.0+git8514f05, ROCm 7.2.2 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1, closed division, data-centre suite 2026-09-16 (published) |
| 8xMI350X_2xEPYC_9575F (8x AMD Instinct MI350X 288GB HBM3e) | 10891 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 fp16,bf16,fp8 · PyTorch 2.10.0+rocm7.2.3, Triton 3.6.0, ROCm 7.2.3 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1, closed division, data-centre suite 2026-09-16 (published) |
| Camarero PDI200A2HG-810 (8x NVIDIA B200-SXM-180GB) | 10737 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 fp16 · TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0, closed division, data-centre suite 2026-04-01 (published) |
| NVIDIA DGX B200 (8x NVIDIA B200-SXM-180GB) | 10447 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0, closed division, data-centre suite 2026-04-01 (published) |
| XE7745_8xMI350P_2xEPYC_9455 (8x AMD Instinct MI350P) | 4838 FPS throughput_offline · b=unknown | MLPerf dlrm-v3 int8,bf16 · vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1, closed division, data-centre suite 2026-09-16 (published) |
How to read these numbers
Every number is labelled. Confidence: A MLPerf closed, audited · B documented third-party method · C community / review · D vendor-published, not independently verified · E estimate. Power basis: measured with the result · spec vendor spec/envelope · power n/a not published.
Machine-readable: /find.json (see llms.txt); full tables are in “Show data”. Confidence grades: A audited MLPerf closed · B documented third-party · C community · D vendor-published · E estimate. Unknown values are shown as unknown, never estimated. Generated 2026-10-16 from public sources.