MemPalace/mempalace · error · DimensionMismatchError

qdrant batch cannot mix embedding dimensions {sorted(dims)}

Error message

qdrant batch cannot mix embedding dimensions {sorted(dims)}

What it means

Raised by _normalize_vectors() when a single write batch (add/upsert/update) contains embeddings of differing dimensions. The Qdrant collection is created with one fixed vector size, so a mixed-dimension batch could never be stored consistently; the backend raises DimensionMismatchError (a BackendError subclass) before contacting the server.

Source

Thrown at mempalace/backends/qdrant.py:251

        raise ValueError(f"embeddings length {len(embeddings)} does not match ids length {n}")


def _as_vector_array(vector: list[float]) -> np.ndarray:
    arr = np.asarray(vector, dtype=np.float32)
    if arr.ndim != 1 or arr.size == 0:
        raise ValueError("embedding must be a non-empty 1D vector")
    return arr


def _normalize_vectors(embeddings: list[list[float]]) -> tuple[list[list[float]], int]:
    vectors = []
    dims = set()
    for embedding in embeddings:
        arr = _as_vector_array(embedding)
        vectors.append(arr.astype(float).tolist())
        dims.add(int(arr.size))
    if len(dims) > 1:
        raise DimensionMismatchError(f"qdrant batch cannot mix embedding dimensions {sorted(dims)}")
    return vectors, dims.pop() if dims else 0


def _jsonable_metadata(meta: dict | None) -> dict:
    try:
        value = json.loads(json.dumps(meta or {}, ensure_ascii=False))
    except (TypeError, ValueError):
        value = {}
    return value if isinstance(value, dict) else {}


def _point_id(doc_id: str) -> str:
    return str(uuid.uuid5(_POINT_NAMESPACE, str(doc_id)))


def _slug(value: str, fallback: str = "palace") -> str:
    safe = re.sub(r"[^A-Za-z0-9_-]+", "_", value).strip("_")
    safe = safe or fallback

View on GitHub (pinned to 06cb6987f0)

Solutions

Check which model produced each row: log {len(e) for e in embeddings} before submit to find the offending dimension
Re-embed the entire batch with a single model so all vectors share one dimension
If you intentionally changed models, create a new collection (or palace) rather than mixing dimensions
Pin the embedder identity recorded with the collection so mismatches surface at config time, not write time

Example fix

// before
 collection.upsert(documents=docs, ids=ids, embeddings=old_rows + new_rows)  # 768-dim + 384-dim
// after
 dims = {len(e) for e in embeddings}
 assert len(dims) == 1, f"mixed dimensions: {dims}"
 collection.upsert(documents=docs, ids=ids, embeddings=embeddings)

Defensive patterns

Strategy: validation

Validate before calling

dims = {len(e) for e in embeddings}
if len(dims) != 1:
    raise ValueError(f"refusing mixed-dimension batch: {sorted(dims)}")

Try / catch

from mempalace.backends.base import DimensionMismatchError
try:
    collection.upsert(documents=docs, ids=ids, embeddings=embs)
except DimensionMismatchError as e:
    logger.error("mixed dimensions: %s", e)
    # re-embed batch with a single model and retry once

Prevention

Pin one embedding model per palace/collection; record it via set_embedder_identity
Verify all rows share a dimension before every batch write
Never merge vectors embedded by different models into one collection

When it happens

Trigger: Calling upsert() or add() with embeddings like [[...768 dims...], [...384 dims...]] — e.g. mixing outputs from two embedding models (nomic-embed-text vs all-MiniLM), or one embedding truncated/corrupted during serialization.

Common situations: Switching the local embedding model without recreating the collection, merging chunks embedded at different times with different models, or a partial pipeline migration that left some rows embedded with the old model.

Related errors

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/9259abf272a74822. Report an issue: GitHub.