DAWNBench

An End-to-End Deep Learning Benchmark and Competition

SQuAD Training

Submission Date Model Time to 0.75 F1 Cost (USD) Max F1 Score Hardware Framework

Oct 2017

BiDAF

Stanford DAWN

source

10:50:22 $5.78 0.7579 60 GB / 16 CPU (Google Cloud [n1-standard-16]) TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

7:51:22 N/A 0.7530 1 P100 / 512 GB / 56 CPU (DAWN Internal Cluster) TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

8:43:40 $8.44 0.7524 1 K80 / 30 GB / 8 CPU (Google Cloud) TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

7:38:10 $6.87 0.7533 1 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge]) TensorFlow v1.2