Verified by MLPerf Inference 4.1 benchmarks, speedAI accelerator cards exhibit industry leading performance, up to 6X the energy efficiency and 3X lower latency than other submitters
Untether AI®, the leader in energy-centric AI inference acceleration, today announced its world-leading verified submission to the MLPerf® Inference benchmarks, version 4.1. The speedAI®240 Preview accelerator card submission demonstrated the highest throughput of any single PCIe card in Datacenter and Edge categories for the image classification benchmark, ResNet-50[1]. The speedAI240 Slim submission exhibited greater than 3X the energy efficiency of other accelerators in the Datacenter Closed Power category [2], 6X the energy efficiency in the Edge Closed category [3], and 3X lower latency in the Edge Closed category [4].
The MLPerf benchmarks are the only objective, peer-reviewed set of AI performance and power benchmarks in existence. MLPerf is supported by MLCommons®, a consortium of industry leading AI chip developers such as Nvidia, Google, and Intel. Benchmark submissions are audited, and peer reviewed by all the submitters for performance, accuracy, latency, and power consumption, ensuring fair and accurate reporting. It is such a high barrier to entry that few startups have attempted to submit results. Untether AI not only submitted, but demonstrated that it has the highest performance, most energy efficient, and lowest latency PCIe AI accelerator cards.
“AI’s potential is being hamstrung by technologies that force customers to choose between performance and power efficiency. Demonstrating leadership across both vectors in a stringent test environment like MLPerf speaks volumes about the unique value of Untether AI’s At-Memory compute architecture as well as the maturity of our hardware and software,” said Chris Walker, CEO of Untether AI.
“MLPerf submission requires operating hardware, shipments to customers, computational accuracy, and a mature software stack that can be peer reviewed. It also requires companies to declare how many accelerators are used in their submission. These factors are what makes these benchmarks so objective, but also creates a high bar that many companies can’t meet in performance, accuracy or transparency of their solution,” said Bob Beachler, VP of Product at Untether AI.
Untether AI submitted two different cards and multiple system configurations in the Datacenter Closed, Datacenter Power, Edge Closed, and Edge Power categories for the MLPerf v4.1 ResNet-50 benchmark. In the Datacenter Closed offline performance submission of ResNet-50, it had a verified result of 70,348 samples/s[1], the highest throughput of any PCIe card submitted. In the Datacenter Closed Power, its 309,752 Server Queries/S at 986 Watts was 3X more energy efficient than the closest competitor [5].
In the Edge submission, it had a verified result of 0.12ms for single stream latency, 0.17ms for multi stream latency, and 70,348 samples/s for offline throughput [4]. These latency values are the fastest ever recorded for an MLPerf ResNet-50 submission. In the Edge Closed Power category, Untether AI was the only company that submitted so no direct comparison is available. However, normalizing to single cards and their published TDPs, the speedAI 240 Slim card demonstrated a 6X improvement in energy efficiency [3].
Untether AI enlisted the aid of Krai, a company that provides AI computer systems and has been trusted to submit MLPerf benchmarks by companies such as Dell, Lenovo, Qualcomm, Google, and HPE. Anton Lokhmotov, CEO of Krai, said, “We were impressed not only by the record-breaking performance and energy efficiency of the speedAI 240 accelerators, but also the readiness and robustness of the imAIgine SDK which facilitated creating the benchmark submissions.”
Untether AI is excited to have its At-Memory technology available in shipping hardware and verified by MLPerf. The speedAI240 accelerator cards’ world-leading performance and energy efficiency will transform AI inference, making it faster and more energy efficient for markets including datacenters, video surveillance, vision guided robotics, agricultural technology, machine inspection, and autonomous vehicles. To find out more about Untether AI acceleration solutions and its recent MLPerf benchmark scores, please visit https://www.untether.ai/mlperf-results/.
Explore AITechPark for the latest advancements in AI, IOT, Cybersecurity, AITech News, and insightful updates from industry experts!
