Announcing Rolling Submissions for DAWNBench

by Cody Coleman, Deepak Narayanan, Daniel Kang, Peter Bailis, and Matei Zaharia

Following the successful completion of the DAWNBench v1 competition, we are re-opening DAWNBench to allow rolling submissions. We’re eager to see the community continue to innovate and improve on optimizing for time-to-accuracy in deep learning, so starting today, we will accept new pull requests to dawn-bench-entries. The tasks, thresholds, metrics, and instructions are still the same as DAWNBench v1, but with two changes to the reviewing process:

  1. We will only review submissions that are in the top 5 results for at least one leaderboard.
  2. New submissions will be highlighted on the leaderboard to indicate they were made after the v1 deadline.

For a more comprehensive benchmark, please consider submitting to the updated MLPerf benchmark, which continues DAWNBench’s focus on time-to-accuracy for common machine learning workloads.