MemPalace/mempalace · error · DimensionMismatchError

milvus batch cannot mix embedding dimensions {sorted(dims)}

Error message

milvus batch cannot mix embedding dimensions {sorted(dims)}

What it means

Raised by _normalize_vectors() when a single add/upsert batch contains embeddings of differing dimensions. Milvus collections are fixed-dimension, so a mixed batch cannot be inserted; the backend collects all dimensions first and raises DimensionMismatchError listing them. Distinct from collection-vs-batch mismatch (a different error raised later with the collection's expected dim) — this is strictly batch-internal inconsistency.

Source

Thrown at mempalace/backends/milvus.py:224

    return "(" + ") and (".join(present) + ")"


def _as_vector_array(vector: list[float]) -> np.ndarray:
    arr = np.asarray(vector, dtype=np.float32)
    if arr.ndim != 1 or arr.size == 0:
        raise ValueError("embedding must be a non-empty 1D vector")
    return arr


def _normalize_vectors(embeddings: list[list[float]]) -> tuple[list[list[float]], int]:
    vectors = []
    dims = set()
    for embedding in embeddings:
        arr = _as_vector_array(embedding)
        vectors.append(arr.astype(float).tolist())
        dims.add(int(arr.size))
    if len(dims) > 1:
        raise DimensionMismatchError(f"milvus batch cannot mix embedding dimensions {sorted(dims)}")
    return vectors, dims.pop() if dims else 0


def _clean_text(value: Any) -> str:
    text = "" if value is None else str(value)
    return strip_lone_surrogates(text).replace("\x00", "")


def _utf8_len(value: str) -> int:
    return len(value.encode("utf-8"))


def _jsonable_metadata(meta: dict | None) -> dict:
    cleaned = {}
    for key, value in (meta or {}).items():
        if key in RESERVED_FIELDS:
            raise ValueError(f"metadata key {key!r} clashes with a reserved Milvus field")
        try:

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Verify all embeddings in a batch come from the same model/dimension before calling add
2. Re-embed stale data after changing models, or clear the collection first
3. Group by dimension and insert as separate batches/collections

Example fix

# before
collection.add(ids=ids, documents=docs, embeddings=all_embs)  # mixed 384/768

# after
dims = {len(e) for e in all_embs}
if len(dims) > 1:
    raise ValueError(f"mixed embedding dimensions: {dims}")
collection.add(ids=ids, documents=docs, embeddings=all_embs)

def uniform_dimension(embeddings) -> int | None:
    dims = {len(e) for e in embeddings}
    if len(dims) != 1:
        raise ValueError(f"mixed embedding dimensions: {sorted(dims)}")
    return dims.pop() if dims else None

from mempalace.backends.base import DimensionMismatchError
try:
    collection.add(ids=ids, documents=docs, embeddings=embs)
except DimensionMismatchError as e:
    if "cannot mix" in str(e):
        groups = {}
        for i, d, e_ in zip(ids, docs, embs):
            groups.setdefault(len(e_), []).append((i, d, e_))
        for batch in groups.values():
            collection.add(ids=[b[0] for b in batch], documents=[b[1] for b in batch], embeddings=[b[2] for b in batch])
    else:
        raise

When it happens

Trigger: add(embeddings=[[0.1]*384, [0.1]*768]) — one embedder call returned 384-d and another 768-d vectors concatenated into one batch.

Common situations: Switching embedding models mid-stream; mixing cached embeddings from an old model with fresh ones; multiple embedders (dense vs sparse) merged accidentally.

Related errors

• embedding must be a non-empty 1D vector
• embedding dimension must be positive
• pgvector collection {self._collection_name!r} expects embedd
• qdrant batch cannot mix embedding dimensions {sorted(dims)}
• qdrant collection {self._collection_name!r} expects embeddin

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/6d74037e32bc001f. Report an issue: GitHub.