DAWNBench

An End-to-End Deep Learning Benchmark and Competition

ImageNet Inference

Submission Date Model 1-example Latency (milliseconds) 10,000 batch classification cost (USD) Max Accuracy Hardware Framework

Apr 2018

ResNet50

Intel(R) Corporation

source

9.9600 N/A 93.07% Amazon EC2 [c5.18xlarge] Intel(R) Optimized Caffe

Nov 2017

ResNet 152

Stanford DAWN

source

26.8200 $0.13 93.00% 1 P100 / 30 GB / 8 CPU (Google Compute) MXNet 0.11.0

Mar 2019

ResNet50

ModelArts Service of Huawei Cloud

source

2.4500 N/A 93.40% Huawei Cloud [pi1.2xlarge.4] TensorFlow 1.8.0

Jun 2019

ResNet50

InferenceX Team of Didi Cloud

source

1.5439 $0.00 93.35% Didi Cloud [1 P4 / 16 GB / 8 vCPU] ifx

Nov 2017

ResNet 152

Stanford DAWN

source

29.2400 $0.07 93.00% 1 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge]) MXNet 0.11.0

Apr 2018

ResNet50

Intel(R) Corporation

source

17.3800 $0.02 93.07% Amazon EC2 [c5.2xlarge] Intel(R) Optimized Caffe

Jun 2019

ResNet50

InferenceX Team of Didi Cloud

source

0.9695 N/A 93.35% Didi Cloud [1 T4 / 16 GB / 8 vCPU] ifx

Nov 2017

ResNet 152

Stanford DAWN

source

49.5100 $0.12 93.35% 1 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge]) TensorFlow v1.2

May 2018

ResNet50

Perseus AI Cloud Acceleration team in Alibaba Cloud

source

2.2700 $0.01 93.25% Alibaba Cloud [ecs.gn5i-c8g1.2xlarge] TensorFlow 1.12.2

Jun 2019

ResNet50

Iluvatar CoreX, P.S.R team: Gang Xu, Yu Song, Tao Yang, Fanwu Han

source

1.4328 N/A 93.73% 1 P4 / 128 GB / 32 CPU TensorRT 5.1.5

Apr 2018

ResNet50

Intel(R) Corporation

source

12.4000 $0.02 93.07% Amazon EC2 [c5.4xlarge] Intel(R) Optimized Caffe

Dec 2018

ResNet50

Perseus AI Cloud Acceleration team in Alibaba Cloud

source

4.2180 $0.02 93.16% Alibaba Cloud [ecs.gn5i-c8g1.2xlarge] TensorFlow 1.10.0

Nov 2017

ResNet 152

Stanford DAWN

source

22.2700 $0.11 93.35% 1 P100 / 30 GB / 8 CPU (Google Compute) TensorFlow v1.2

Jul 2019

ResNet50

PingAn GammaLab & PingAn Cloud team

source

0.8200 N/A 93.01% PingAn Cloud {1 T4 / 128 GB / Xeon(G) Gold 6130} Caffe 1.0
Disclosure: The Stanford DAWN research project is a five-year industrial affiliates program at Stanford University and is financially supported in part by founding members including Intel, Microsoft, NEC, Teradata, VMWare, and Google. For more information, including information regarding Stanford’s policies on openness in research and policies affecting industrial affiliates program membership, please see DAWN's membership page.