similarity

class sentence_transformers.util.similarity.SimilarityFunction(value)[source]

Enum class for supported similarity functions. The following functions are supported:

  • SimilarityFunction.COSINE ("cosine"): Cosine similarity

  • SimilarityFunction.DOT_PRODUCT ("dot", dot_product): Dot product similarity

  • SimilarityFunction.EUCLIDEAN ("euclidean"): Euclidean distance

  • SimilarityFunction.MANHATTAN ("manhattan"): Manhattan distance

static possible_values() → list[str][source]

Returns a list of possible values for the SimilarityFunction enum.

Returns:

A list of possible values for the SimilarityFunction enum.

Return type:

list

Example

>>> possible_values = SimilarityFunction.possible_values()
>>> possible_values
['cosine', 'dot', 'euclidean', 'manhattan']
static to_similarity_fn(similarity_function: str | SimilarityFunction) → Callable[[Tensor | ndarray, Tensor | ndarray], Tensor][source]

Converts a similarity function name or enum value to the corresponding similarity function.

Parameters:

similarity_function (Union[str, SimilarityFunction]) – The name or enum value of the similarity function.

Returns:

The corresponding similarity function.

Return type:

Callable[[Union[Tensor, ndarray], Union[Tensor, ndarray]], Tensor]

Raises:

ValueError – If the provided function is not supported.

Example

>>> similarity_fn = SimilarityFunction.to_similarity_fn("cosine")
>>> similarity_scores = similarity_fn(embeddings1, embeddings2)
>>> similarity_scores
tensor([[0.3952, 0.0554],
        [0.0992, 0.1570]])
static to_similarity_pairwise_fn(similarity_function: str | SimilarityFunction) → Callable[[Tensor | ndarray, Tensor | ndarray], Tensor][source]

Converts a similarity function into a pairwise similarity function.

The pairwise similarity function returns the diagonal vector from the similarity matrix, i.e. it only computes the similarity(a[i], b[i]) for each i in the range of the input tensors, rather than computing the similarity between all pairs of a and b.

Parameters:

similarity_function (Union[str, SimilarityFunction]) – The name or enum value of the similarity function.

Returns:

The pairwise similarity function.

Return type:

Callable[[Union[Tensor, ndarray], Union[Tensor, ndarray]], Tensor]

Raises:

ValueError – If the provided similarity function is not supported.

Example

>>> pairwise_fn = SimilarityFunction.to_similarity_pairwise_fn("cosine")
>>> similarity_scores = pairwise_fn(embeddings1, embeddings2)
>>> similarity_scores
tensor([0.3952, 0.1570])
sentence_transformers.util.similarity.cos_sim(a: list | ndarray | Tensor, b: list | ndarray | Tensor) → Tensor[source]

Computes the cosine similarity between two tensors.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Matrix with res[i][j] = cos_sim(a[i], b[j])

Return type:

Tensor

sentence_transformers.util.similarity.dot_score(a: list | ndarray | Tensor, b: list | ndarray | Tensor) → Tensor[source]

Computes the dot-product dot_prod(a[i], b[j]) for all i and j.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Matrix with res[i][j] = dot_prod(a[i], b[j])

Return type:

Tensor

sentence_transformers.util.similarity.euclidean_sim(a: list | ndarray | Tensor, b: list | ndarray | Tensor) → Tensor[source]

Computes the euclidean similarity (i.e., negative distance) between two tensors. Handles sparse tensors without converting to dense when possible.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Matrix with res[i][j] = -euclidean_distance(a[i], b[j])

Return type:

Tensor

sentence_transformers.util.similarity.manhattan_sim(a: list | ndarray | Tensor, b: list | ndarray | Tensor) → Tensor[source]

Computes the manhattan similarity (i.e., negative distance) between two tensors. Handles sparse tensors without converting to dense when possible.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Matrix with res[i][j] = -manhattan_distance(a[i], b[j])

Return type:

Tensor

sentence_transformers.util.similarity.pairwise_angle_sim(x: Tensor, y: Tensor) → Tensor[source]

Computes the absolute normalized angle distance. See AnglELoss or https://huggingface.co/papers/2309.12871 for more information.

Parameters:

  • x (Tensor) – The first tensor.

  • y (Tensor) – The second tensor.

Returns:

Vector with res[i] = angle_sim(a[i], b[i])

Return type:

Tensor

sentence_transformers.util.similarity.pairwise_cos_sim(a: Tensor, b: Tensor) → Tensor[source]

Computes the pairwise cosine similarity cos_sim(a[i], b[i]).

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Vector with res[i] = cos_sim(a[i], b[i])

Return type:

Tensor

sentence_transformers.util.similarity.pairwise_dot_score(a: Tensor, b: Tensor) → Tensor[source]

Computes the pairwise dot-product dot_prod(a[i], b[i]).

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Vector with res[i] = dot_prod(a[i], b[i])

Return type:

Tensor

sentence_transformers.util.similarity.pairwise_euclidean_sim(a: list | ndarray | Tensor, b: list | ndarray | Tensor) → Tensor[source]

Computes the euclidean distance (i.e., negative distance) between pairs of tensors.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Vector with res[i] = -euclidean_distance(a[i], b[i])

Return type:

Tensor

sentence_transformers.util.similarity.pairwise_manhattan_sim(a: list | ndarray | Tensor, b: list | ndarray | Tensor) → Tensor[source]

Computes the manhattan similarity (i.e., negative distance) between pairs of tensors.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Vector with res[i] = -manhattan_distance(a[i], b[i])

Return type:

Tensor

sentence_transformers.util.similarity.pytorch_cos_sim(a: Tensor, b: Tensor) → Tensor[source]

Computes the cosine similarity between two tensors.

Parameters:

  • a (Union[list, np.ndarray, Tensor]) – The first tensor.

  • b (Union[list, np.ndarray, Tensor]) – The second tensor.

Returns:

Matrix with res[i][j] = cos_sim(a[i], b[j])

Return type:

Tensor