deepset-ai/haystack · error · DocumentStoreError

The embedding size of the query should be the same as the em

Error message

The embedding size of the query should be the same as the embedding size of the Documents. Please make sure that the query has been embedded with the same model as the Documents.

What it means

_compute_query_embedding_similarity_scores raises DocumentStoreError when the query embedding's dimension differs from the document embeddings', so np.dot fails with a 'shapes not aligned' error. The store wraps it in a message telling you to embed the query with the same model as the documents.

Source

Thrown at haystack/document_stores/in_memory/document_store.py:901

                    "Please make sure that the Documents have been embedded with the same model."
                ) from e
            raise e
        if document_embeddings.ndim == 1:
            document_embeddings = np.expand_dims(a=document_embeddings, axis=0)

        if self.embedding_similarity_function == "cosine":
            # cosine similarity is a normed dot product; guard against zero-norm vectors
            # (e.g. a zero embedding) which would otherwise divide by zero and yield NaN scores.
            query_norm = np.linalg.norm(x=query_embedding, axis=1, keepdims=True)
            document_norms = np.linalg.norm(x=document_embeddings, axis=1, keepdims=True)
            query_embedding /= np.where(query_norm == 0.0, 1.0, query_norm)
            document_embeddings /= np.where(document_norms == 0.0, 1.0, document_norms)

        try:
            scores = np.dot(a=query_embedding, b=document_embeddings.T)[0].tolist()
        except ValueError as e:
            if "shapes" in str(e) and "not aligned" in str(e):
                raise DocumentStoreError(
                    "The embedding size of the query should be the same as the embedding size of the Documents. "
                    "Please make sure that the query has been embedded with the same model as the Documents."
                ) from e
            raise e

        if scale_score:
            if self.embedding_similarity_function == "dot_product":
                scores = [expit(float(score / DOT_PRODUCT_SCALING_FACTOR)) for score in scores]
            elif self.embedding_similarity_function == "cosine":
                scores = [(score + 1) / 2 for score in scores]

        return scores

    async def count_documents_async(self) -> int:
        """
        Returns the number of documents present in the DocumentStore.
        """
        return len(self.storage.keys())

View on GitHub (pinned to e318778c9b)

Solutions

1. Embed the query with the exact same model used for the documents (same name, version, and dimension)
2. Re-index documents if the query-side model must change
3. Assert dimension match before retrieval: len(query_embedding) == len(docs[0].embedding)

Example fix

// before
query_vec = embedder_query_model.encode(query)  # 384-dim, docs are 768-dim
store.embedding_retrieval(query_embedding=query_vec)
// after
query_vec = same_model_as_documents.encode(query)
assert len(query_vec) == len(store.storage[next(iter(store.storage))].embedding)
store.embedding_retrieval(query_embedding=query_vec)

sample_doc = next(iter(store.storage.values()), None)
if sample_doc and sample_doc.embedding and len(query_embedding) != len(sample_doc.embedding):
    raise RuntimeError(
        f"Query embedding dim {len(query_embedding)} != document dim {len(sample_doc.embedding)}; "
        "use the same embedding model for query and documents"
    )

from haystack.errors import DocumentStoreError
try:
    docs = store.embedding_retrieval(query_embedding=query_embedding)
except DocumentStoreError as e:
    if "embedding size of the query" in str(e):
        query_embedding = query_embedder.run(query)["embedding"]  # same model as documents
        docs = store.embedding_retrieval(query_embedding=query_embedding)
    else:
        raise

When it happens

Trigger: Calling embedding_retrieval with a query vector of different length than the stored document vectors — e.g. documents embedded with a 768-dim model but the query embedded with a 384-dim model (or vice versa).

Common situations: Swapping the retriever's embedder without re-indexing documents, using different embedding models for indexing and query time, config drift between indexing and query pipelines.

Related errors

• query_embedding should be a non-empty list of floats.
• The embedding size of all Documents should be the same. Plea
• Query should be a non-empty string

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/b79eefba3f3b548c. Report an issue: GitHub.