Skip to content

Commit 8b083f9

Browse files
authored
update the pixel phone benchmark results (#249)
1 parent dd321f0 commit 8b083f9

File tree

1 file changed

+12
-10
lines changed

1 file changed

+12
-10
lines changed

README.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -46,25 +46,27 @@ different versions of a novel BNN model called Quicknet (trained on ImageNet dat
4646
on a [Pixel 1 phone (2016)](https://support.google.com/pixelphone/answer/7158570?hl=en-GB)
4747
and a Raspberry Pi 4 Model B ([BCM2711](https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711/README.md)) board:
4848

49-
| Model | Top-1 Accuracy | RPi 4 B, ms (1 thread) | Pixel 1, ms (1 thread) |
50-
| ------------------------------------------------------------------------------------------------ | :------------: | :--------------------: | :--------------------: |
51-
| Quicknet ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.1.0/quicknet_weights.h5)) | 58.3 % | 60.5 | 27.9 |
52-
| Quicknet-Large ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.1.0/quicknet_large_weights.h5)) | 62.5 % | 89.9 | 41.8 |
49+
| Model | Top-1 Accuracy | RPi 4 B, ms (1 thread) | Pixel 1, ms (1 thread) |
50+
| ------------------------------------------------------------------------------------------------ | :------------: | :--------------------: | :--------------------: |
51+
| Quicknet ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.1.0/quicknet_weights.h5)) | 58.3 % | 50.7 | 22.1 |
52+
| Quicknet-Large ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.1.0/quicknet_large_weights.h5)) | 62.5 % | 79.0 | 35.6 |
53+
5354

5455
For reference, [dabnn](https://github.com/JDAI-CV/dabnn) (the other main BNN library) reports an inference time of 61.3 ms for [Bi-RealNet](https://docs.larq.dev/larq/api/larq_zoo/#birealnet) (56.4% accuracy) on the Pixel 1 phone,
55-
while LCE achieves an inference time of 54.0 ms for Bi-RealNet on the same device.
56+
while LCE achieves an inference time of 47.1 ms for Bi-RealNet on the same device.
5657
They furthermore present a modified version, BiRealNet-Stem, which achieves the same accuracy of 56.4% in 43.2 ms.
5758

5859
The following table presents **multi-threaded** performance of Larq Compute Engine on
5960
a Pixel 1 phone and a Raspberry Pi 4 Model B ([BCM2711](https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711/README.md))
6061
board:
6162

62-
| Model | Top-1 Accuracy | RPi 4 B, ms (4 threads) | Pixel 1, ms (4 threads) |
63-
| ------------------------------------------------------------------------------------------------ | :------------: | :---------------------: | :---------------------: |
64-
| Quicknet ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.1.0/quicknet_weights.h5)) | 58.3 % | 37.9 | 19.1 |
65-
| Quicknet-Large ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.1.0/quicknet_large_weights.h5)) | 62.5 % | 55.8 | 28.0 |
63+
| Model | Top-1 Accuracy | RPi 4 B, ms (4 threads) | Pixel 1, ms (4 threads) |
64+
| ------------------------------------------------------------------------------------------------ | :------------: | :---------------------: | :---------------------: |
65+
| Quicknet ([.h5](https://github.com/larq/zoo/releases/download/quicknet-v0.1.0/quicknet_weights.h5)) | 58.3 % | 27.6 | 13.2 |
66+
| Quicknet-Large ([.h5](https://github.com/larq/zoo/releases/download/quicknet_large-v0.1.0/quicknet_large_weights.h5)) | 62.5 % | 44.7 | 21.6 |
67+
6668

67-
Benchmarked on February 14th, 2020 with LCE custom
69+
Benchmarked on February 26th, 2020 with LCE custom
6870
[TFLite Model Benchmark Tool](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark)
6971
(see [here](https://github.com/larq/compute-engine/tree/master/larq_compute_engine/tflite/benchmark))
7072
and BNN models with randomized weights and inputs.

0 commit comments

Comments
 (0)