Skip to content

Commit 1cc1859

Browse files
authored
Add new benchmark numbers (#339)
1 parent 41794b9 commit 1cc1859

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -50,12 +50,12 @@ and a Raspberry Pi 4 Model B ([BCM2711](https://www.raspberrypi.org/documentatio
5050

5151
| Model | Top-1 Accuracy | RPi 4 B, ms (1 thread) | Pixel 1, ms (1 thread) |
5252
| ------------------------------------------------------------------------------------------------ | :------------: | :--------------------: | :--------------------: |
53-
| [QuickNet](https://docs.larq.dev/zoo/api/sota/#quicknet) ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.2.0/quicknet_weights.h5)) | 58.6 % | 45.6 | 21.2 |
54-
| [QuickNet-Large](https://docs.larq.dev/zoo/api/sota/#quicknetlarge) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.2.0/quicknet_large_weights.h5)) | 62.7 % | 66.5 | 32.0 |
55-
| [QuickNet-XL](https://docs.larq.dev/zoo/api/sota/#quicknetxl) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_xl-v0.1.0/quicknet_xl_weights.h5)) | 67.0 % | 121.2 | 55.8 |
53+
| [QuickNet](https://docs.larq.dev/zoo/api/sota/#quicknet) ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.2.0/quicknet_weights.h5)) | 58.6 % | 38.5 | 20.2 |
54+
| [QuickNet-Large](https://docs.larq.dev/zoo/api/sota/#quicknetlarge) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.2.0/quicknet_large_weights.h5)) | 62.7 % | 58.3 | 30.9 |
55+
| [QuickNet-XL](https://docs.larq.dev/zoo/api/sota/#quicknetxl) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_xl-v0.1.0/quicknet_xl_weights.h5)) | 67.0 % | 102.0 | 54.5 |
5656

5757
For reference, [dabnn](https://github.com/JDAI-CV/dabnn) (the other main BNN library) reports an inference time of 61.3 ms for [Bi-RealNet](https://docs.larq.dev/zoo/api/literature/#birealnet) (56.4% accuracy) on the Pixel 1 phone,
58-
while LCE achieves an inference time of 47.7 ms for Bi-RealNet on the same device.
58+
while LCE achieves an inference time of 46.8 ms for Bi-RealNet on the same device.
5959
They furthermore present a modified version, BiRealNet-Stem, which achieves the same accuracy of 56.4% in 43.2 ms.
6060

6161
The following table presents **multi-threaded** performance of Larq Compute Engine on
@@ -64,12 +64,12 @@ board:
6464

6565
| Model | Top-1 Accuracy | RPi 4 B, ms (4 threads) | Pixel 1, ms (4 threads) |
6666
| ------------------------------------------------------------------------------------------------ | :------------: | :---------------------: | :---------------------: |
67-
| [QuickNet](https://docs.larq.dev/zoo/api/sota/#quicknet) ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.2.0/quicknet_weights.h5)) | 58.6 % | 21.7 | 11.6 |
68-
| [QuickNet-Large](https://docs.larq.dev/zoo/api/sota/#quicknetlarge) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.2.0/quicknet_large_weights.h5)) | 62.7 % | 31.8 | 16.9 |
69-
| [QuickNet-XL](https://docs.larq.dev/zoo/api/sota/#quicknetxl) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_xl-v0.1.0/quicknet_xl_weights.h5)) | 67.0 % | 52.4 | 29.7 |
67+
| [QuickNet](https://docs.larq.dev/zoo/api/sota/#quicknet) ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.2.0/quicknet_weights.h5)) | 58.6 % | 20.0 | 11.5 |
68+
| [QuickNet-Large](https://docs.larq.dev/zoo/api/sota/#quicknetlarge) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.2.0/quicknet_large_weights.h5)) | 62.7 % | 30.4 | 16.9 |
69+
| [QuickNet-XL](https://docs.larq.dev/zoo/api/sota/#quicknetxl) ([.h5](https://github.com/larq/zoo/releases/download/quicknet_xl-v0.1.0/quicknet_xl_weights.h5)) | 67.0 % | 46.6 | 28.3 |
7070

7171

72-
Benchmarked on March 20th, 2020 with LCE custom
72+
Benchmarked on April 20th, 2020 with LCE custom
7373
[TFLite Model Benchmark Tool](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark)
7474
(see [here](https://github.com/larq/compute-engine/tree/master/larq_compute_engine/tflite/benchmark))
7575
and BNN models with randomized weights and inputs.

0 commit comments

Comments
 (0)