deepset-ai/haystack · error · DocumentStoreError

The embedding size of all Documents should be the same. Plea

Error message

The embedding size of all Documents should be the same. Please make sure that the Documents have been embedded with the same model.

What it means

_compute_query_embedding_similarity_scores raises DocumentStoreError when the stored documents' embeddings have differing sizes. numpy cannot build a rectangular array from an inhomogeneous list ('inhomogeneous shape'), so the store translates that into a clear error telling you to re-embed with one model.

Source

Thrown at haystack/document_stores/in_memory/document_store.py:881

    ) -> list[float]:
        """
        Computes the similarity scores between the query embedding and the embeddings of the documents.

        :param embedding: Embedding of the query.
        :param documents: A list of Documents.
        :param scale_score: Whether to scale the scores of the Documents. Default is False.
        :returns: A list of scores.
        """

        query_embedding = np.array(embedding)
        if query_embedding.ndim == 1:
            query_embedding = np.expand_dims(a=query_embedding, axis=0)

        try:
            document_embeddings = np.array([doc.embedding for doc in documents])
        except ValueError as e:
            if "inhomogeneous shape" in str(e):
                raise DocumentStoreError(
                    "The embedding size of all Documents should be the same. "
                    "Please make sure that the Documents have been embedded with the same model."
                ) from e
            raise e
        if document_embeddings.ndim == 1:
            document_embeddings = np.expand_dims(a=document_embeddings, axis=0)

        if self.embedding_similarity_function == "cosine":
            # cosine similarity is a normed dot product; guard against zero-norm vectors
            # (e.g. a zero embedding) which would otherwise divide by zero and yield NaN scores.
            query_norm = np.linalg.norm(x=query_embedding, axis=1, keepdims=True)
            document_norms = np.linalg.norm(x=document_embeddings, axis=1, keepdims=True)
            query_embedding /= np.where(query_norm == 0.0, 1.0, query_norm)
            document_embeddings /= np.where(document_norms == 0.0, 1.0, document_norms)

        try:
            scores = np.dot(a=query_embedding, b=document_embeddings.T)[0].tolist()
        except ValueError as e:

View on GitHub (pinned to e318778c9b)

Solutions

1. Re-embed ALL documents with the same model (re-run the Embedder/DocumentWriter indexing pipeline into a fresh store)
2. Delete documents embedded with the old model before re-indexing
3. Verify dimensions are uniform before writing: len(set(len(d.embedding) for d in docs)) == 1

Example fix

// before
# docs partly embedded with old 384-dim model, partly with new 768-dim model
store.embedding_retrieval(query_embedding=query_vec)
// after
# re-index everything with one model into a fresh store, then retrieve
store = InMemoryDocumentStore()
pipe = Pipeline().add_component("embedder", embedder).add_component("writer", DocumentWriter(store, policy=DuplicatePolicy.OVERWRITE))
pipe.run(...)
store.embedding_retrieval(query_embedding=query_vec)

dims = {len(d.embedding) for d in store.filter_documents() if d.embedding is not None}
if len(dims) > 1:
    raise RuntimeError(f"Store has mixed embedding sizes: {dims}; re-embed all documents with one model")

from haystack.errors import DocumentStoreError
try:
    docs = store.embedding_retrieval(query_embedding=query_embedding)
except DocumentStoreError as e:
    if "embedding size of all Documents" in str(e):
        # re-index the whole corpus with a single embedding model, then retry
        ...
    else:
        raise

When it happens

Trigger: Calling embedding_retrieval when documents in the store were embedded with different models or different truncation/dimension settings (e.g. mixed 384-dim and 768-dim vectors).

Common situations: Switching embedding models without re-indexing, re-embedding only a subset of documents, mixing documents from two corpora indexed with different models.

Related errors

• The embedding size of the query should be the same as the em
• Document store {type(self.document_store).__name__} does not
• Expected 1 parent document with id {parent_id}, found {len(p
• Parent document with id {parent_id} does not have any childr
• document_store must be an instance of InMemoryDocumentStore

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/e80c8b4ac9a52c21. Report an issue: GitHub.