Find edge AI hardware

Task LLM generation · model-size class by model · batch per group · 2 platforms · edge hardware only · model_key "llama328b"

Results are compared only within the same model. 1 model have results for this task.

How to read power: results are compared only within the same model. By efficiency ranks only results whose source publishes a power figure; By speed ranks every result. ● Measured = measured for this workload. Vendor-measured (typical/avg) = the vendor’s own measurement at typical or average load, not an independent test and not a maximum, so fps/W looks better than at max TDP. Chip, Module or Board TDP = the vendor’s published maximum, not measured. Typical power, not max TDP = only a typical figure is published, so fps/W looks better than at full load. Scope says what the watts cover: Accelerator only (host not included), Module, or Whole system. Max of range = top of a published range, so fps/W is a floor (≥). “— not published” = the source gives no power; we never estimate one.