[Not an issue] Curious on Distance Calculation

[firmanjml]

Hi, thank you for sharing this project, it’s an excellent resource for the community to learn from.

I just have a quick question (this isn’t an issue report).
I noticed that you're using ObjectBox to store vectors, and I came across a function in your code.

cosineDistance(x1: FloatArray, x2: FloatArray) {}

Since ObjectBox provides a built-in nearestNeighbour function that can be used with cosine distance to determine vector similarity, I'm wondering...

Is it necessary to use the cosineDistance function again after calling nearestNeighbour?
From what I understand, nearestNeighbour already handles the cosine distance calculation to find the most similar vectors.

Objectbox Data Model & Vector Index
VectorDistanceType COSINE
consineDistance function ln 113

[shubham0204]

@firmanjml The distance calculated by ObjectBox when performing a nearest neighbor search is not in the range [-1, 1] (as implied by the definition of cosine similarity) as mentioned in the docs for VectorDistanceType.COSINE,

Cosine similarity compares two vectors irrespective of their magnitude (compares the angle of two vectors).
Often used for document or semantic similarity.
Value range: 0.0 - 2.0 (0.0: same direction, 1.0: orthogonal, 2.0: opposite direction)

The threshold of 0.4, that determines whether an exact match was found, does not work correctly when scaled in the range [0, 2]. To keep using the threshold of 0.4 and to make the pipeline identical to that in my older project, I decided to calculate the cosine similarity myself using the 'nearest neighbor' and query embeddings.

[firmanjml]

Amazing, thank you for the quick answer.