DAWNBench

An End-to-End Deep Learning Benchmark and Competition

ImageNet Inference

Submission Date Model 1-example Latency (milliseconds) 10,000 batch classification cost (USD) Max Accuracy Hardware Framework

Apr 2018

 ResNet50

Intel(R) Corporation

source

9.9600 N/A 93.07% Amazon EC2 [c5.18xlarge] Intel(R) Optimized Caffe

Nov 2017

 ResNet 152

Stanford DAWN

source

26.8200 $0.13 93.00% 1 P100 / 30 GB / 8 CPU (Google Compute) MXNet 0.11.0

Nov 2017

 ResNet 152

Stanford DAWN

source

29.2400 $0.07 93.00% 1 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge]) MXNet 0.11.0

Apr 2018

 ResNet50

Intel(R) Corporation

source

17.3800 $0.02 93.07% Amazon EC2 [c5.2xlarge] Intel(R) Optimized Caffe

Nov 2017

 ResNet 152

Stanford DAWN

source

49.5100 $0.12 93.35% 1 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge]) TensorFlow v1.2

Apr 2018

 ResNet50

Intel(R) Corporation

source

12.4000 $0.02 93.07% Amazon EC2 [c5.4xlarge] Intel(R) Optimized Caffe

Nov 2017

 ResNet 152

Stanford DAWN

source

22.2700 $0.11 93.35% 1 P100 / 30 GB / 8 CPU (Google Compute) TensorFlow v1.2
Disclosure: The Stanford DAWN research project is a five-year industrial affiliates program at Stanford University and is financially supported in part by founding members including Intel, Microsoft, NEC, Teradata, VMWare, and Google. For more information, including information regarding Stanford’s policies on openness in research and policies affecting industrial affiliates program membership, please see DAWN's membership page.