pathwaycom/pathway · error · ValueError

Either `dimensions` or `embedder` must be provided to index

Error message

Either `dimensions` or `embedder` must be provided to index factory.

What it means

KNN index factories in pathway.stdlib.indexing.nearest_neighbors need to know the vector dimensionality to allocate and configure the underlying index (USearch reserves space per vector, brute force allocates accordingly). The dimension is either stated explicitly via `dimensions` or inferred from an `embedder`; __post_init__ raises this ValueError when both are absent, i.e. dimensions is None and embedder is None.

Source

Thrown at python/pathway/stdlib/indexing/nearest_neighbors.py:426

    dimensions: int | None = None
    embedder: pw.UDF | None = None

    def _get_embed_dimensions(self) -> int:
        # import is here to prevent cyclical imports
        from pathway.xpacks.llm.embedders import BaseEmbedder

        if isinstance(self.embedder, BaseEmbedder):
            return self.embedder.get_embedding_dimension()
        elif isinstance(self.embedder, pw.UDF):
            return len(_coerce_sync(self.embedder.__wrapped__)("."))
        else:
            raise TypeError("Embedder is not a valid `pw.UDF`.")

    def __post_init__(self):
        if self.dimensions is None and self.embedder is not None:
            self.dimensions: int = self._get_embed_dimensions()
        elif self.dimensions is None and self.embedder is None:
            raise ValueError(
                "Either `dimensions` or `embedder` must be provided to index factory."
            )


@dataclass(kw_only=True)
class UsearchKnnFactory(KnnIndexFactory):
    """
    Factory for creating UsearchKNN indices.

    Args:
        dimensions (int): number of dimensions of vectors that are used by the index and
            queries. This is only needed if the `embedder` is not provided.
        reserved_space (int): initial capacity (in the number of entries) of the index
        metric (USearchMetricKind): metric kind that is used to determine distance.
            Defaults to cosine similarity.
        connectivity (int): maximum number of edges for a node in the HNSW index, setting
            this value to 0 tells usearch to configure it on its own
        expansion_add (int): indicates amount of work spent while adding elements to the index

View on GitHub (pinned to fa2f74a464)

Solutions

1. Pass the exact vector size if vectors are precomputed: UsearchKnnFactory(dimensions=1536).
2. Or pass an embedder (a pw.UDF or BaseEmbedder) so the dimension is inferred by embedding a probe string.
3. Centralize the dimension in one constant shared with your embedding model config to avoid drift.

Example fix

# before
factory = BruteForceKnnFactory()  # vectors already in the table

# after
factory = BruteForceKnnFactory(dimensions=384)

def make_knn_factory(cls, *, dimensions=None, embedder=None, **kw):
    if dimensions is None and embedder is None:
        raise ValueError("provide dimensions (precomputed vectors) or an embedder")
    return cls(dimensions=dimensions, embedder=embedder, **kw)

When it happens

Trigger: Constructing UsearchKnnFactory() or BruteForceKnnFactory() with neither dimensions nor embedder — e.g. when indexing precomputed vector columns the author assumes the dimension is read from the data, which it is not.

Common situations: Switching a pipeline from text embedding to precomputed vectors and deleting the embedder argument; copy-pasting a factory constructor and trimming arguments; defaults picked up from a config dict where both keys are missing.

Related errors

• Failed to install dependencies
• Column {api.TIME_PSEUDOCOLUMN} cannot contain negative times
• Column {api.DIFF_PSEUDOCOLUMN} can only have 1 and -1 values
• schema does not match given dataframe
• negative timestamp cannot be used

AI-assisted analysis of pathwaycom/pathway@fa2f74a464 (2026-08-15). Data as JSON: /api/errors/50b1daf3fa1fd09e. Report an issue: GitHub.