RyanCodrai/turbovec · warning · UserWarning

similarity='dot_product' produces unbounded raw inner produc

Error message

similarity='dot_product' produces unbounded raw inner products, so relevance scores are not calibrated to [0, 1]; score_threshold filtering is only meaningful if your embeddings are unit-normalized upstream. Use similarity='cosine' (the default) for calibrated relevance scores.

What it means

This is a Python UserWarning (not an exception) from the LangChain vector store's _select_relevance_score_fn: dot_product similarity yields unbounded raw inner products, so relevance scores are not calibrated to [0,1] and score_threshold filtering is unreliable unless embeddings are unit-normalized upstream. The affine mapping is kept for continuity, but the clamp was removed because it silently collapsed scores >= 1.0 onto 1.0 and let score_threshold retrievers admit unrelated documents (issue #322).

Source

Thrown at turbovec-python/python/turbovec/langchain.py:171

        return self._similarity

    # ---- Relevance score normalization --------------------------------

    def _select_relevance_score_fn(self) -> Callable[[float], float]:
        # Under the default cosine mode both sides are unit vectors, so
        # the engine's raw inner product is true cosine similarity in
        # [-1, 1]; (sim + 1) / 2 maps it onto LangChain's [0, 1]
        # relevance scale and the clamp only absorbs quantization noise.
        if self._similarity == COSINE:
            return lambda sim: max(0.0, min(1.0, (sim + 1.0) / 2.0))
        # Under dot_product mode scores are raw inner products with no
        # fixed range, so no mapping onto [0, 1] is meaningful. The same
        # affine mapping is kept for continuity with earlier releases,
        # but WITHOUT the clamp: clamping silently collapsed every raw
        # score >= 1.0 onto exactly 1.0, which made score_threshold
        # retrievers admit unrelated documents and suppressed the
        # out-of-range warning VectorStore itself emits (issue #322).
        warnings.warn(
            "similarity='dot_product' produces unbounded raw inner products, "
            "so relevance scores are not calibrated to [0, 1]; "
            "score_threshold filtering is only meaningful if your embeddings "
            "are unit-normalized upstream. Use similarity='cosine' (the "
            "default) for calibrated relevance scores.",
            UserWarning,
            stacklevel=2,
        )
        return lambda sim: (sim + 1.0) / 2.0

    # ---- Embedder-output validation -----------------------------------

    @staticmethod
    def _check_embedded_batch(vectors: np.ndarray, n_texts: int) -> None:
        """Validate the shape of an embedder's document-batch output.

        Only 2D outputs are inspected here — any other ndim falls through
        to ``_store_texts_and_vectors``, whose existing guard names the bad

View on GitHub (pinned to ccab9f325e)

Solutions

1. Use similarity='cosine' (the default) for calibrated [0,1] relevance scores
2. Unit-normalize embeddings before insertion/querying if you must use dot_product
3. Drop score_threshold filtering when using raw dot_product scores
4. Suppress or route the UserWarning only after confirming scores are bounded

Example fix

// before
vs = Turbovec.from_documents(docs, emb, similarity="dot_product")
r = vs.as_retriever(search_kwargs={"score_threshold": 0.7})
// after
vs = Turbovec.from_documents(docs, emb)  # cosine, calibrated
r = vs.as_retriever(search_kwargs={"score_threshold": 0.7})

import numpy as np
def embeddings_unit_norm(X):
    n = np.linalg.norm(X, axis=-1)
    return bool(np.all(np.abs(n - 1.0) < 1e-3))
# only use dot_product + score_threshold if embeddings_unit_norm(X)

import numpy as np
def is_unit_normalized(x) -> bool:
    a = np.asarray(x)
    return a.ndim >= 1 and bool(np.allclose(np.linalg.norm(a, axis=-1), 1.0, atol=1e-3))

import warnings
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    scores = vs._select_relevance_score_fn("dot_product")
for w in caught:
    if "dot_product" in str(w.message):
        logging.warning("unnormalized embeddings with dot_product: %s", w.message)

When it happens

Trigger: Constructing the turbovec LangChain VectorStore with similarity='dot_product' and then using a score_threshold retriever (or any relevance-score mapping path).

Common situations: Switching similarity from the default 'cosine' to 'dot_product' for speed without normalizing embeddings; copying retriever configs that assume calibrated scores; upgrading to a release that removed the clamp.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

• module {__name__!r} has no attribute {name!r}
• duplicate id in batch: {k!r}
• {prefix} {version}; this turbovec accepts versions {list(com
• side-car key {key!r} at {_crumb_path(entry)} is {type(key)._
• side-car value at {_crumb_path(entry)} is {obj!r}, which JSO

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/8568c11018ae2780. Report an issue: GitHub.