speech-to-text benchmarks on edge AI hardware
Each table compares only results with the same model-size class and batch size. Search this workload · JSON.
Fastest in the best-covered like-for-like group (Whisper base, batch 1): Qualcomm Snapdragon X2 Elite (CRD) at 408 FPS D 2026 (year only). 8 groups, 57 platforms.
Show data — 8 like-for-like tables
Whisper base · batch 1 (5 platforms)
✓ Like-for-like: model-size class Whisper base, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Qualcomm Snapdragon X2 Elite (CRD) | 408 FPS inference_time · b=1 | whisper_base / decoder float · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper base | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Snapdragon X Elite (CRD) | 272 FPS inference_time · b=1 | whisper_base / decoder float · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper base | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Dragonwing IQ-9075 EVK | 196 FPS inference_time · b=1 | whisper_base / decoder float · Qualcomm AI Hub qnn_context_binary (QAIRT 2.50.0.260828221209) · Whisper base | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Hailo-8 (M.2/PCIe, x86 host) | 51.40 FPS throughput · b=1 | whisper_base_5s_encoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper base | D | 2.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8) undated |
| Hailo-8L (M.2/PCIe, x86 host) | 41.80 FPS throughput · b=1 | whisper_base_5s_no_kqs_decoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper base | D | 1.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8L) undated |
Whisper tiny · batch 1 (5 platforms)
✓ Like-for-like: model-size class Whisper tiny, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Qualcomm Snapdragon X2 Elite (CRD) | 705 FPS inference_time · b=1 | whisper_tiny / decoder float · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper tiny | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Snapdragon X Elite (CRD) | 454 FPS inference_time · b=1 | whisper_tiny / decoder float · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper tiny | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Dragonwing IQ-9075 EVK | 327 FPS inference_time · b=1 | whisper_tiny / decoder float · Qualcomm AI Hub voice_ai (QAIRT 2.50.0.260828221209) · Whisper tiny | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Hailo-8 (M.2/PCIe, x86 host) | 79.70 FPS throughput · b=1 | whisper_tiny_10s_no_kqs_decoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper tiny | D | 2.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8) undated |
| Hailo-8L (M.2/PCIe, x86 host) | 55.20 FPS throughput · b=1 | whisper_tiny_10s_encoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper tiny | D | 1.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8L) undated |
Whisper small · batch 1 (4 platforms)
✓ Like-for-like: model-size class Whisper small, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Qualcomm Snapdragon X2 Elite (CRD) | 270 FPS inference_time · b=1 | whisper_small_quantized / decoder w8a16 · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper small | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Snapdragon X Elite (CRD) | 134 FPS inference_time · b=1 | whisper_small_quantized / decoder w8a16 · Qualcomm AI Hub qnn_context_binary (QAIRT 2.50.0.260828221209) · Whisper small | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Dragonwing IQ-9075 EVK | 116 FPS inference_time · b=1 | whisper_small_quantized / decoder w8a16 · Qualcomm AI Hub qnn_context_binary (QAIRT 2.50.0.260828221209) · Whisper small | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Dragonwing RB3 Gen 2 (QCS6490) | 42.65 FPS inference_time · b=1 | whisper_small_quantized / decoder w8a16 · Qualcomm AI Hub voice_ai (QAIRT 2.50.0.260828221209) · Whisper small | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
Whisper large · batch 1 (3 platforms)
✓ Like-for-like: model-size class Whisper large, batch 1. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Qualcomm Snapdragon X2 Elite (CRD) | 197 FPS inference_time · b=1 | whisper_large_v3_turbo / decoder float · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper large | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Snapdragon X Elite (CRD) | 117 FPS inference_time · b=1 | whisper_large_v3_turbo / decoder float · Qualcomm AI Hub precompiled_qnn_onnx (QAIRT 2.50.0.260828221209) · Whisper large | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
| Qualcomm Dragonwing IQ-9075 EVK | 95.34 FPS inference_time · b=1 | whisper_large_v3_turbo / decoder float · Qualcomm AI Hub qnn_context_binary (QAIRT 2.50.0.260828221209) · Whisper large | D | unknown power n/a no power published | Qualcomm AI Hub Models perf.yaml (BSD-3) 2026 (year only) |
Whisper base · batch 8 (2 platforms)
✓ Like-for-like: model-size class Whisper base, batch 8. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Hailo-8 (M.2/PCIe, x86 host) | 254 FPS throughput · b=8 | whisper_base_5s_no_kqs_decoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper base | D | 2.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8) undated |
| Hailo-8L (M.2/PCIe, x86 host) | 138 FPS throughput · b=8 | whisper_base_5s_no_kqs_decoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper base | D | 1.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8L) undated |
Whisper tiny · batch 8 (2 platforms)
✓ Like-for-like: model-size class Whisper tiny, batch 8. Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| Hailo-8 (M.2/PCIe, x86 host) | 310 FPS throughput · b=8 | whisper_tiny_10s_no_kqs_decoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper tiny | D | 2.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8) undated |
| Hailo-8L (M.2/PCIe, x86 host) | 136 FPS throughput · b=8 | whisper_tiny_10s_encoder INT8 (Hailo quantized) · Hailo DFC v2.20.0 / HailoRT (model zoo) · Whisper tiny | D | 1.50 W spec reported in result source — spec | Hailo Model Zoo public models (Hailo-8L) undated |
RNN-T · batch unknown (3 platforms)
✓ Like-for-like: model-size class RNN-T, batch unknown (size or batch not stated by source — unverified). Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA Jetson AGX Orin 64GB | 1170 FPS throughput_offline · b=unknown | MLPerf RNN-T INT8/FP16 (MLPerf closed) · TensorRT (MLPerf Inference, older round) · RNN-T | D | 65.00 W spec platform spec envelope (not measured with this result) | NVIDIA Jetson Benchmarks (MLPerf v3.1 results table) 2026-10 |
| NVIDIA Jetson Orin NX 16GB | 432 FPS throughput_offline · b=unknown | MLPerf RNN-T INT8/FP16 (MLPerf closed) · TensorRT (MLPerf Inference, older round) · RNN-T | D | unknown power n/a no power published | NVIDIA Jetson Benchmarks (MLPerf v3.1 results table) 2026-10 |
| NVIDIA Jetson Orin NX 16GB (MaxQ) | 328 FPS throughput_offline · b=unknown | MLPerf RNN-T INT8/FP16 (MLPerf closed) · TensorRT (MLPerf Inference, older round) · RNN-T | D | 17.25 W measured reported in result source — measured | NVIDIA Jetson Benchmarks (MLPerf v3.1 results table) 2026-10 |
unknown · batch unknown (48 platforms)
✓ Like-for-like: model-size class unknown, batch unknown (size or batch not stated by source — unverified). Model, precision and stack may still differ.
| Platform | Result | Model · precision · stack | Grade | Power | Source |
|---|---|---|---|---|---|
| NVIDIA DGX B300 (8x NVIDIA B300-SXM-270GB) | 53267 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID B300-SXM-270GBx8_TRT undated |
| Cisco UCS C880A M8 (8x NVIDIA B300-SXM-270GB) | 52222 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID C880A_M8_B300_270GBx8_TRT undated |
| NVIDIA DGX B200 (8x NVIDIA B200-SXM-180GB) | 49327 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0040 undated |
| G894-SD3-AAX7 (8x NVIDIA B300-SXM-270GB) | 48117 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0039 undated |
| Lenovo ThinkSystem SR680a V4 (8x NVIDIA B300-SXM-270GB) | 47269 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0066 undated |
| HPE ProLiant Compute XD685 (8x NVIDIA B200-SXM-180GB) | 46034 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.11, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0048 undated |
| Dell PowerEdge XE9780L (8x NVIDIA B300-SXM-270GB) | 45659 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0025 undated |
| QuantaGrid D75H-10U (8x NVIDIA B300-SXM-270GB) | 45408 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID D75H-10U_B300-270GBx8_TRT undated |
| Camarero PDI200A2HG-810 (8x NVIDIA B200-SXM-180GB) | 45083 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0085 undated |
| SYS-822GS-NBRT (8x NVIDIA B200-SXM-180GB) | 42390 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0099 undated |
| IBM Cloud qx3d-160x1792x8h200,NVIDIA 8xH200-SXM-141GB,vLLM 0.15.1 (8x NVIDIA H200-SXM-141GB) | 36396 FPS throughput_offline · b=unknown | MLPerf whisper bfloat16 · vLLM 0.15.1, PyTorch 2.9.1+cu128, CUDA 12.9.86 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0093 undated |
| Dell PowerEdge XE7740 (8x NVIDIA H200-NVL-141GB) | 31969 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.13, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0021 undated |
| Cisco UCS C845A M8 (8x NVIDIA H200-NVL-141GB) | 30195 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.11, CUDA 12.9 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0008 undated |
| NVIDIA GB200 NVL72 (4x NVIDIA GB200) | 22051 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0041 undated |
| HPE ProLiant Compute DL380a Gen12 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 19278 FPS throughput_offline · b=unknown | MLPerf whisper fp4 · TensorRT 10.16.1.11, CUDA 13.2 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID HPE_PROLIANT_DL380A_RTX_PRO_6000_PCIE_96GBx8 undated |
| Cisco UCS C845A M8 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 18992 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID C845A_M8_RTXPro6000_96GBx8_TRT undated |
| HPE ProLiant Compute DL380a Gen12 (10x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 18889 FPS throughput_offline · b=unknown | MLPerf whisper fp4 · TensorRT 10.11.0.33, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0044 undated |
| Dell PowerEdge XE7745 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 18434 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0023 undated |
| Lenovo ThinkSystem SR675i V3 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 18401 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0065 undated |
| ESC8000A_E13P_RTX_PRO_6000_PCIE_96GBx8 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 18338 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0004 undated |
| Dell PowerEdge XE7740 (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 18232 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0022 undated |
| QuantaGrid D75E-4U (8x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 17838 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0088 undated |
| Cisco UCS X210 M8 - X580P PCIe Node (4x NVIDIA H200-NVL-141GB) | 15339 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.11, CUDA 12.9 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0013 undated |
| Cisco UCS X215c M8 - X580P PCIe Node (4x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 9237 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID X215M8_X580P_RTXPro6000_96GBx4_TRT undated |
| HPE ProLiant DL385 Gen11 (4x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 9228 FPS throughput_offline · b=unknown | MLPerf whisper fp4 · TensorRT 10.14.1.48, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0054 undated |
| Cisco UCS C845A M8 (8x NVIDIA RTX PRO 4500 Blackwell) | 7778 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID C845A_M8_RTXPro4500_32GBx8_TRT undated |
| B300-SXM-270GBx1_TRT (1x NVIDIA B300-SXM-270GB) | 7059 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID B300-SXM-270GBx1_TRT undated |
| 1-node-4x-BMG-B70 (4x Intel(R) Arc Pro(R) B70) | 5597 FPS throughput_offline · b=unknown | MLPerf whisper INT4 · PyTorch vLLM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0101 undated |
| 1-node-4x-BMG-B70 (4x Intel(R) Arc(TM) Pro B70) | 5540 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-4x-BMG-B70 undated |
| Cisco UCS C240 M8 - (5x NVIDIA RTX PRO 4500 Blackwell) | 4809 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID C240M8_RTXPro4500_32GBx5_TRT undated |
| HPE ProLiant Compute DL380 Gen12 (5x NVIDIA RTX PRO 4500 Blackwell Server Edition) | 4719 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.11.0.33, CUDA 13.2 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, edge) ID HPE_PROLIANT_DL380_RTX_PRO_4500_PCIE_32GBx5_TRT undated |
| HPE ProLiant DL385 Gen11 (2x NVIDIA RTX PRO 6000 Blackwell Server Edition) | 4713 FPS throughput_offline · b=unknown | MLPerf whisper fp4 · TensorRT 10.14.1.48, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0052 undated |
| HPE ProLiant Compute DL380 Gen12 (5x NVIDIA RTX PRO 4500 Blackwell) | 4513 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.11.0.33, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, edge) ID 6.0-0056 undated |
| 1-node-4x-BMG-B60 (4x Intel(R) Arc Pro(R) B60) | 3996 FPS throughput_offline · b=unknown | MLPerf whisper INT4 · PyTorch vLLM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0059 undated |
| Cisco UCS X210 M8 - X580P PCIe Node (4x NVIDIA RTX PRO 4500 Blackwell) | 3878 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.14, CUDA 13.1 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID X210M8_X580P_RTXPro4500_32GBx4_TRT undated |
| HPE ProLiant Compute DL345 Gen12 (4x NVIDIA RTX PRO 4500 Blackwell Server Edition) | 3686 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.15.1.29, CUDA 13.2 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, edge) ID HPE_PROLIANT_DL345_GEN12_RTX_PRO_4500_PCIE_32GBx4_TRT undated |
| IBM Cloud gx3-48x240x2l40s instance,2xL40S-PCIE-48GB,vLLM 0.15.1 (2x NVIDIA L40S) | 3647 FPS throughput_offline · b=unknown | MLPerf whisper bfloat16 · vLLM 0.15.1, RHEL 9.6 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0090 undated |
| Cisco system (Intel Xeon 6788P, CPU only) CPU-only | 2971 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID X410M8_1_node_4S_GNR_86C undated |
| Supermicro system (Intel Xeon 6788P, CPU only) CPU-only | 2881 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-4S-GNR_86C undated |
| Intel system (Intel Xeon 6980P, CPU only) CPU-only | 2319 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-2S-GNR_128C undated |
| Dell system (Intel Xeon 6980P, CPU only) CPU-only | 2276 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-2S-GNR_128C undated |
| 1-node-4x-BMG-B50 (4x Intel(R) Arc Pro(R) B50) | 2112 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch vLLM · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0058 undated |
| RedHat_Intel system (Intel Xeon 6972P, CPU only) CPU-only | 1966 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-2S-GNR_96C undated |
| Quanta_Cloud_Technology system (Intel Xeon 6788P, CPU only) CPU-only | 1548 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-2S-GNR_86C undated |
| Dell system (Intel Xeon 6787P, CPU only) CPU-only | 1539 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-2S-GNR_86C undated |
| Cisco system (Intel Xeon 6787P, CPU only) CPU-only | 1423 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, datacenter) ID 6.0-0007 undated |
| Oracle system (Intel Xeon 6987P-C, CPU only) CPU-only | 1100 FPS throughput_offline · b=unknown | MLPerf whisper INT8 · PyTorch · unknown | A | unknown power n/a no power published | MLPerf Inference v6.1 (closed, datacenter) ID 1-node-1S-GNR_120C undated |
| HPE ProLiant DL145 Gen11 (1x NVIDIA RTX PRO 4500 Blackwell) | 917 FPS throughput_offline · b=unknown | MLPerf whisper fp16 · TensorRT 10.11.0.33, CUDA 13.0 · unknown | A | unknown power n/a no power published | MLPerf Inference v6.0 (closed, edge) ID 6.0-0055 undated |
How to read these numbers
Every number is labelled. Confidence: A MLPerf closed, audited · B documented third-party method · C community / review · D vendor-published, not independently verified · E estimate. Power basis: measured with the result · spec vendor spec/envelope · power n/a not published.
Machine-readable: /find.json (see llms.txt); full tables are in “Show data”. Confidence grades: A audited MLPerf closed · B documented third-party · C community · D vendor-published · E estimate. Unknown values are shown as unknown, never estimated. Generated 2026-10-16 from public sources.