Update to API v2.0

README.md

@@ -1,40 +1,45 @@
-# Plumerai People Detection OpenCV pipeline example
+# Plumerai Video Intelligence OpenCV pipeline example
 
-This is the source code for a simple version of using the Plumerai People Detection library in an end-to-end demo setting. It runs on Linux, uses OpenCV and V4L, and retrieves input data from a webcam (or optionally an RTSP stream) and writes box coordinates to console and to an output video stream displayed on screen. It can be modified as needed for proto-typing with the Plumerai People Detection library.
+This is the source code for a simple version of using the Plumerai Video Intelligence library in an end-to-end demo setting. It runs on Linux, uses OpenCV and V4L, and retrieves input data from a webcam (or optionally an RTSP stream) and writes box coordinates to console and to an output video stream displayed on screen. It can be modified as needed for proto-typing with the Plumerai Video Intelligence library.
 
-The Plumerai People Detection library itself is not included: this repository can't be used without access to the library. If you do not have access to the library and would like to evaluate it, then contact us at [plumerai.com/contact_us](https://plumerai.com/contact_us). For more information, see [plumerai.com/people-detection](https://plumerai.com/people-detection).
+The Plumerai Video Intelligence library itself is not included: this repository can't be used without access to the library. If you do not have access to the library and would like to evaluate it, then contact us at [plumerai.com/contact_us](https://plumerai.com/contact_us). For more information, see [plumerai.com/people-detection](https://plumerai.com/people-detection).
 
 ## Linux system requirements
 
 The example application uses [OpenCV](https://opencv.org/). Using the `CAP_V4L` OpenCV API, it connects to the webcam with [V4L (video 4 linux)](https://en.wikipedia.org/wiki/Video4Linux). Most (USB) webcams compatible with V4L should work with this example demo application, as long as they support the Motion-JPEG (MJPEG) input compression format. If you have V4L installed on your system (see below), this can be verified by running `v4l2-ctl --list-formats-ext`.
 
 On a Debian-based system (e.g. Ubuntu) the pre-requisites (OpenCV with V4L) can be installed as follows:
+
 ```bash
 apt install libopencv-dev libopencv-highgui-dev libopencv-videoio-dev libv4l-dev
 ```
+
 In case of issues or for other Linux systems, see the [official OpenCV release page](https://opencv.org/releases/).
 
 ## Configuring the camera settings
 
 First, make sure the camera settings (`camera_height`, `camera_width`, and `camera_id`) are set correctly near the top of the `src/opencv_example.cc` file.
 
-The higher the input resolution, the better the results can become. However, note that this might slow down the entire example application, because the camera capture and video-displaying might take up more resources. The framerate reported is purely for the Plumerai People Detection algorithm itself, and does not count camera capture or displaying of the results.
+The higher the input resolution, the better the results can become. However, note that this might slow down the entire example application, because the camera capture and video-displaying might take up more resources. The framerate reported is purely for the Plumerai Video Intelligence algorithm itself, and does not count camera capture or displaying of the results.
 
-Note that the example application itself (the camera capture / video decoding and displaying) is not optimized for speed. Furthermore, the people detection algorithm in its current form is not optimized for speed on x86 systems, only for targets such as Arm Cortex-A.
+Note that the example application itself (the camera capture / video decoding and displaying) is not optimized for speed. Furthermore, the video intelligence algorithm in its current form is not optimized for speed on x86 systems, only for targets such as Arm Cortex-A.
 
 ## Optional: use an RTSP stream as input
 
-This repository also contains example code to use an RTSP video stream instead of camera data as input. The code changes are minimal, and can be enabled by uncommenting the `#define USE_RTSP_INPUT` macro  near the top of the `src/opencv_example.cc` file. Then, a little bit below the RTSP stream settings (`camera_height`, `camera_width`, and `rtsp_url`) can be changed as needed.
+This repository also contains example code to use an RTSP video stream instead of camera data as input. The code changes are minimal, and can be enabled by uncommenting the `#define USE_RTSP_INPUT` macro near the top of the `src/opencv_example.cc` file. Then, a little bit below the RTSP stream settings (`camera_height`, `camera_width`, and `rtsp_url`) can be changed as needed.
 
 ## Compiling the example application
 
-The OpenCV example can be compiled with a C++11 compiler linking to both OpenCV4 and the Plumerai People Detection library, e.g.:
+The OpenCV example can be compiled with a C++11 compiler linking to both OpenCV4 and the Plumerai Video Intelligence library, e.g.:
+
 ```bash
-g++ -std=c++11 -O3 src/opencv_example.cc -I/path/to/plumeraipeopledetection/include/ /path/to/plumeraipeopledetection/lib/<PLATFORM>/libplumeraipeopledetection.a `pkg-config --cflags --libs opencv4` -ldl -pthread -o opencv_example
+g++ -std=c++11 -O3 src/opencv_example.cc -I/path/to/plumeraivideointelligence/include/ /path/to/plumeraivideointelligence/lib/<PLATFORM>/libplumeraivideointelligence.a `pkg-config --cflags --libs opencv4` -ldl -pthread -o opencv_example
 ```
+
 Here, `<PLATFORM>` above refers to the platform you are on, e.g. `x86_64` or `aarch64`. In case of an older OpenCV version, remove the `4` in the `opencv4` part above.
 
 The example app can then be executed without any command-line arguments, e.g.:
+
 ```
 ./opencv_example
 ```

src/opencv_example.cc

@@ -1,6 +1,6 @@
-// Copyright (C) 2022 Plumerai Ltd under the Apache 2.0 license.
+// Copyright (C) 2024 Plumerai Ltd under the Apache 2.0 license.
 //
-// This code implements a simple demo around the Plumerai People Detection
+// This code implements a simple demo around the Plumerai Video Intelligence
 // library using OpenCV to capture webcam input and display to screen. You can
 // modify the code as needed for your prototyping use-case, it might not work
 // out-of-the-box depending on your system and camera used. See the README for
@@ -16,14 +16,16 @@
 #include "opencv2/highgui/highgui.hpp"
 #include "opencv2/videoio.hpp"
 
-// The Plumerai People Detection library. If you do not have access to this
+// The Plumerai Video Intelligence library. If you do not have access to this
 // library and would like to evaluate it, then contact us via
 // https://plumerai.com/contact_us. For more information, see our website
 // https://plumerai.com/people-detection, and docs.plumerai.com for information
 // how to use it.
-#include "plumerai/people_detection.h"
+#include "plumerai/video_intelligence.h"
 
-// Enable this marco to switch the input of this example application from
+using namespace plumerai;
+
+// Enable this macro to switch the input of this example application from
 // camera input to RTSP stream:
 // #define USE_RTSP_INPUT
 
@@ -61,11 +63,11 @@ const int colours[num_colours][3] = {
     {128, 128, 0},  {255, 215, 180}, {0, 0, 128}};
 
 int main() {
-  // Set-up the Plumerai People Detection algorithm
-  auto ppd = plumerai::PeopleDetection(height, width);
+  // Set-up the Plumerai Video Intelligence algorithm
+  auto pvi = plumerai::VideoIntelligence(height, width);
 
   // Set-up the output window
-  const auto window_text = "Plumerai People Detection";
+  const auto window_text = "Plumerai Video Intelligence";
   cv::namedWindow(window_text, cv::WINDOW_AUTOSIZE);
 
   // Set-up the camera, see the above constants for the settings
@@ -94,27 +96,35 @@ int main() {
       return -1;
     }
 
-    // Process the frame using the Plumerai People Detection library. This
+    // Process the frame using the Plumerai Video Intelligence library. This
     // function is timed and framerate in FPS is reported. Note that the
-    // Plumerai People Detection library is not optimized for all systems,
+    // Plumerai Video Intelligence library is not optimized for all systems,
     // see the README for more details.
     auto start_time = std::chrono::steady_clock::now();
-    auto predictions = ppd.process_frame(cv_image.data);
+    auto error_code = pvi.process_frame(
+        ImagePointer<ImageFormat::PACKED_RGB888>(cv_image.data));
     auto elapsed_time = std::chrono::steady_clock::now() - start_time;
 
+    if (error_code != ErrorCode::SUCCESS) {
+      printf("Error: Could not process the frame.\n");
+      return -1;
+    }
+
     // Report the time and framerate the above function took
     auto time_ms =
         std::chrono::duration<double, std::milli>(elapsed_time).count();
     auto fps = static_cast<int>(1000 / time_ms);
     printf("Processing took %.1f ms (%d fps)\n", time_ms, fps);
 
     // Process the resulting bounding-boxes if there are any
+    std::vector<BoxPrediction> predictions;
+    error_code = pvi.object_detection().get_detections(predictions);
+    if (error_code != ErrorCode::SUCCESS) {
+      printf("Error: Could not get detections.\n");
+      return -1;
+    }
     for (auto &p : predictions) {
-      // Filter out low-confidence predictions, adjust the threshold as needed
-      if (p.confidence < 0.5f) {
-        continue;
-      }
-
+      // Convert the relative bounding-box coordinates to the image size
       auto x_min =
           std::max(0, std::min(static_cast<int>(p.x_min * width), width - 1));
       auto y_min =

testing/fake_libplumerai.cc

@@ -1,20 +1,20 @@
-// Copyright (C) 2022 Plumerai Ltd under the Apache 2.0 license.
+// Copyright (C) 2024 Plumerai Ltd under the Apache 2.0 license.
 //
-// Implements a fake version of the Plumerai People Detection library such that
-// we can test linking and running the application without needing access to the
-// actual library. This file does not implement the regular Plumerai People
-// Detection library header, do not use it outside of (automated) testing.
-#include "plumerai/people_detection.h"
+// Implements a fake version of the Plumerai Video Intelligence library such
+// that we can test linking and running the application without needing access
+// to the actual library. This file does not implement the regular Plumerai
+// Video Intelligence library header, do not use it outside of (automated)
+// testing.
+#include "plumerai/video_intelligence.h"
 
 namespace plumerai {
 
-PeopleDetection::PeopleDetection(int height, int width) {}
-PeopleDetection::~PeopleDetection() {}
+VideoIntelligence::VideoIntelligence(int height, int width) {}
+VideoIntelligence::~VideoIntelligence() {}
 
-std::vector<BoxPrediction> PeopleDetection::process_frame(
-    const std::uint8_t *image_data, float delta_t) {
-  // This does nothing: it returns an empty vector
-  return std::vector<BoxPrediction>();
+ErrorCode ObjectDetection::get_detections(std::vector<BoxPrediction>& results) {
+  results.clear();
+  return ErrorCode::SUCCESS;
 }
 
 }  // namespace plumerai

testing/plumerai/video_intelligence.h

@@ -0,0 +1,52 @@
+// Copyright (C) 2024 Plumerai Ltd under the Apache 2.0 license.
+//
+// Implements a fake version of the Plumerai Video Intelligence header such that
+// we can test linking and running the application without needing access to the
+// actual library. This file is not the regular Plumerai Video Intelligence
+// header, do not use it outside of (automated) testing.
+#pragma once
+
+#include <cstdint>
+#include <vector>
+
+typedef struct BoxPrediction {
+  float y_min;
+  float x_min;
+  float y_max;
+  float x_max;
+  float confidence;
+  unsigned int id;
+  unsigned int class_id;
+} BoxPrediction;
+
+namespace plumerai {
+
+enum class ErrorCode { SUCCESS = 0 };
+
+enum class ImageFormat { PACKED_RGB888 };
+
+template <ImageFormat format>
+struct ImagePointer {
+  ImagePointer(const std::uint8_t* _data) {}
+};
+
+class ObjectDetection {
+ public:
+  ErrorCode get_detections(std::vector<BoxPrediction>& results);
+};
+
+class VideoIntelligence {
+ public:
+  VideoIntelligence(int height, int width);
+  ~VideoIntelligence();
+
+  template <ImageFormat image_format>
+  ErrorCode process_frame(const ImagePointer<image_format> image_data,
+                          float delta_t = 0.f) {
+    return ErrorCode::SUCCESS;
+  }
+
+  ObjectDetection object_detection() const { return ObjectDetection(); }
+};
+
+}  // namespace plumerai