by Daniel Kang, Nikos Arechiga, Sudeep Pillai, Peter Bailis, and Matei Zaharia
Machine learning (ML) is increasingly being deployed in mission-critical settings, where errors can have disastrous consequences: autonomous vehicles have already been involved in fatal accidents. As a result, auditing ML deployments is becoming increasingly important. An emerging body of work has shown the importance of one aspect of the ML deployment pipeline: training data. Much work in ML assumes that provided labels are ground truth and measure performance against this data. For example, benchmarks on domains ranging from autonomous vehicles...
by Daniel Kang, John Guibas, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia
In this blog post, we’ll describe our recent work on constructing indexes for unstructured data. As a sneak preview, our index can reduce ingest costs by 10x while simultaneously improving query costs by up to 24x over prior work! While this blog post will be self-contained, please see our other blog posts (part 1, part 2, part 3, part 4 for other exciting developments and more context! As we described in part 1, unstructured data records (e.g., videos, text) are...
by Daniel Kang, John Guibas, Peter Bailis, Tatsunori Hashimoto, Yi Sun, and Matei Zaharia
In this blog post, we’ll describe our recent work on accelerating approximate aggregation queries with expensive predicates. While this blog post will be self-contained, please see our other blog posts for other exciting developments and more context (part 1, part 2, part 3)! Analysts and researchers are increasingly interested in using powerful machine learning models and human labeling services (which we’ll refer to as “oracles”) to compute statistics over their datasets. For example, a media studies researcher may want to...
by Daniel Kang, Ankit Mathur, Teja Veeramacheneni, Peter Bailis, and Matei Zaharia
In this blog post, we’ll describe our recent work on benchmarking recent progress on deep neural network (DNN) execution and optimizing end-to-end DNN inference for visual DNNs. While this blog post will be self-contained, please see our other blog posts (part 1, part 2) for other exciting developments and more context! DNNs now power a range of visual analytics applications because they can produce high quality annotations over visual data. However, they can cost billions of float-point operations to execute...
by Daniel Kang, Edward Gan, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia
In this blog post, we’ll describe our recent work on approximate selection queries with statistical guarantees. While this blog post will be self-contained, please see our other blog posts for other exciting developments and more context (part 1)! As we described in part 1, unstructured data records (e.g., videos, text) are becoming increasingly possible to automatically query with the proliferation of powerful deep neural networks (DNNs) and human-powered labeling services (which we collectively refer to as “oracle methods”). As a...
by Daniel Kang, Peter Bailis, and Matei Zaharia
Unstructured data (e.g., videos, text) has become increasingly possible to automatically query with the proliferation of powerful deep neural networks (DNNs) and human-powered labeling services (which we collectively refer to as “oracle methods”). For example, an urban planner may query videos of street cameras to count vehicles to understand traffic patterns. As another example, a lawyer may be interested in extracting emails mentioning employee/employer information (“relation extraction”) for legal discovery. One naive method to execute such queries is to use...
by Deepak Narayanan, Cody Coleman, Peter Bailis, and Matei Zaharia
MLPerf announced its v0.7 results recently, with sub-minute submissions from companies such as Nvidia, Google, Alibaba, Intel, and others. While this is exciting, many of these entries are fundamentally hard to compare, since they use different numbers and types of accelerators. Consequently, it is still challenging for an end user to determine the right accelerator type to use for their model and with what scale factor, particularly if the user has time or budget constraints. In this blogpost, we use...
by Cody Coleman, Peter Bailis, and Matei Zaharia
Given massive amounts of data available to train deep networks for many tasks, how can we quickly determine which data should actually be used in training? Data selection methods like active learning and core-set selection techniques are powerful ways to curate data for training, but these approaches can be computationally expensive and struggle to scale. In recent work at ICLR 2020, we show how to speed up data selection by up to 41.9x: we use a small, less accurate model...
by Peter Kraft, Daniel Kang, Deepak Narayanan, Shoumik Palkar, Peter Bailis, Matei Zaharia
Machine learning inference, which involves making predictions from machine learning models, is an increasingly important problem today, crucial for many systems like spam detection and content recommendation. Serving ML predictions is fundamentally different from serving other workloads, such as web pages or database queries, because, unlike those workloads, ML applications have unique statistical properties, like an amenability to approximation. However, existing systems for ML inference serving, like Clipper or AWS Sagemaker, neglect these statistical properties and approach ML inference workloads...
by Cody Coleman, Daniel Kang, Deepti Raghavan, Megan Leszczynski, Peter Kraft, Zhihao Jia, and Matei Zaharia
We are excited to present some of our latest research at the MLSys 2020 conference in Austin next week! DAWN researchers are involved in five conference papers and several workshop papers, and on top of that, our PI Chris Ré is giving a keynote on Monday, and PI Matei Zaharia is co-organizing the MLOps workshop. Be sure to check out the the following talks on our papers at MLSys next week: Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference...