MemPalace/mempalace · error · ValueError

embeddings length {len(embeddings)} does not match ids lengt

Error message

embeddings length {len(embeddings)} does not match ids length {n}

What it means

_validate_write_batch requires that when embeddings is provided its length matches len(ids). It fails fast with ValueError before normalization or any Qdrant write, preventing batches where vectors and documents are misaligned.

Source

Thrown at mempalace/backends/qdrant.py:233

            continue
        raise UnsupportedFilterError(f"where_document operator {key!r} not supported")
    return True


def _validate_write_batch(
    *,
    documents: list[str],
    ids: list[str],
    metadatas: Optional[list[dict]],
    embeddings: Optional[list[list[float]]],
) -> None:
    n = len(ids)
    if len(documents) != n:
        raise ValueError(f"documents length {len(documents)} does not match ids length {n}")
    if metadatas is not None and len(metadatas) != n:
        raise ValueError(f"metadatas length {len(metadatas)} does not match ids length {n}")
    if embeddings is not None and len(embeddings) != n:
        raise ValueError(f"embeddings length {len(embeddings)} does not match ids length {n}")


def _as_vector_array(vector: list[float]) -> np.ndarray:
    arr = np.asarray(vector, dtype=np.float32)
    if arr.ndim != 1 or arr.size == 0:
        raise ValueError("embedding must be a non-empty 1D vector")
    return arr


def _normalize_vectors(embeddings: list[list[float]]) -> tuple[list[list[float]], int]:
    vectors = []
    dims = set()
    for embedding in embeddings:
        arr = _as_vector_array(embedding)
        vectors.append(arr.astype(float).tolist())
        dims.add(int(arr.size))
    if len(dims) > 1:
        raise DimensionMismatchError(f"qdrant batch cannot mix embedding dimensions {sorted(dims)}")

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Verify len(embeddings) == len(ids) right after the embedding step and fail loudly there.
2. Make your embed function always return exactly one vector per input (embed empty strings to a zero vector or filter inputs and ids together).
3. Log input/output counts from the embedder to catch truncation early.

Example fix

# before
vecs = embed_client.embed([t for t in texts if t])  # may be shorter
col.add(ids=ids, documents=texts, embeddings=vecs)

# after
vecs = embed_client.embed(texts)
assert len(vecs) == len(ids) == len(texts)
col.add(ids=ids, documents=texts, embeddings=vecs)

vecs = embed(texts)
if len(vecs) != len(ids):
    raise RuntimeError(f"embedder returned {len(vecs)} vectors for {len(ids)} ids")
col.add(ids=ids, documents=texts, embeddings=vecs)

def embeddings_aligned(ids, embeddings) -> bool:
    return embeddings is None or len(embeddings) == len(ids)

try:
    col.add(ids=ids, documents=docs, embeddings=vecs)
except ValueError as e:
    if "embeddings length" in str(e):
        logger.error("embedding batch truncated: %d/%d", len(vecs), len(ids))
        raise

When it happens

Trigger: upsert(ids=["1","2"], documents=[d1,d2], embeddings=[[...]]) — e.g. embedding call returned fewer vectors than texts because one input was skipped or an API returned a truncated batch.

Common situations: Batch embedding endpoints that drop empty strings; retry logic re-requesting embeddings for a subset; caching layers that return partial results.

Related errors

• documents length {len(documents)} does not match ids length
• metadatas length {len(metadatas)} does not match ids length
• embedding must be a non-empty 1D vector
• qdrant batch cannot mix embedding dimensions {sorted(dims)}
• embedding dimension must be positive

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/e5af2951b9ea687f. Report an issue: GitHub.