{"record":{"id":"9259abf272a74822","repo":"MemPalace/mempalace","slug":"qdrant-batch-cannot-mix-embedding-dimensions-sort","errorCode":null,"errorMessage":"qdrant batch cannot mix embedding dimensions {sorted(dims)}","messagePattern":"qdrant batch cannot mix embedding dimensions (.+?)","errorType":"exception","errorClass":"DimensionMismatchError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/qdrant.py","lineNumber":251,"sourceCode":"        raise ValueError(f\"embeddings length {len(embeddings)} does not match ids length {n}\")\n\n\ndef _as_vector_array(vector: list[float]) -> np.ndarray:\n    arr = np.asarray(vector, dtype=np.float32)\n    if arr.ndim != 1 or arr.size == 0:\n        raise ValueError(\"embedding must be a non-empty 1D vector\")\n    return arr\n\n\ndef _normalize_vectors(embeddings: list[list[float]]) -> tuple[list[list[float]], int]:\n    vectors = []\n    dims = set()\n    for embedding in embeddings:\n        arr = _as_vector_array(embedding)\n        vectors.append(arr.astype(float).tolist())\n        dims.add(int(arr.size))\n    if len(dims) > 1:\n        raise DimensionMismatchError(f\"qdrant batch cannot mix embedding dimensions {sorted(dims)}\")\n    return vectors, dims.pop() if dims else 0\n\n\ndef _jsonable_metadata(meta: dict | None) -> dict:\n    try:\n        value = json.loads(json.dumps(meta or {}, ensure_ascii=False))\n    except (TypeError, ValueError):\n        value = {}\n    return value if isinstance(value, dict) else {}\n\n\ndef _point_id(doc_id: str) -> str:\n    return str(uuid.uuid5(_POINT_NAMESPACE, str(doc_id)))\n\n\ndef _slug(value: str, fallback: str = \"palace\") -> str:\n    safe = re.sub(r\"[^A-Za-z0-9_-]+\", \"_\", value).strip(\"_\")\n    safe = safe or fallback","sourceCodeStart":233,"sourceCodeEnd":269,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/qdrant.py#L233-L269","documentation":"Raised by _normalize_vectors() when a single write batch (add/upsert/update) contains embeddings of differing dimensions. The Qdrant collection is created with one fixed vector size, so a mixed-dimension batch could never be stored consistently; the backend raises DimensionMismatchError (a BackendError subclass) before contacting the server.","triggerScenarios":"Calling upsert() or add() with embeddings like [[...768 dims...], [...384 dims...]] — e.g. mixing outputs from two embedding models (nomic-embed-text vs all-MiniLM), or one embedding truncated/corrupted during serialization.","commonSituations":"Switching the local embedding model without recreating the collection, merging chunks embedded at different times with different models, or a partial pipeline migration that left some rows embedded with the old model.","solutions":["Check which model produced each row: log {len(e) for e in embeddings} before submit to find the offending dimension","Re-embed the entire batch with a single model so all vectors share one dimension","If you intentionally changed models, create a new collection (or palace) rather than mixing dimensions","Pin the embedder identity recorded with the collection so mismatches surface at config time, not write time"],"exampleFix":"// before\n collection.upsert(documents=docs, ids=ids, embeddings=old_rows + new_rows)  # 768-dim + 384-dim\n// after\n dims = {len(e) for e in embeddings}\n assert len(dims) == 1, f\"mixed dimensions: {dims}\"\n collection.upsert(documents=docs, ids=ids, embeddings=embeddings)","handlingStrategy":"validation","validationCode":"dims = {len(e) for e in embeddings}\nif len(dims) != 1:\n    raise ValueError(f\"refusing mixed-dimension batch: {sorted(dims)}\")","typeGuard":null,"tryCatchPattern":"from mempalace.backends.base import DimensionMismatchError\ntry:\n    collection.upsert(documents=docs, ids=ids, embeddings=embs)\nexcept DimensionMismatchError as e:\n    logger.error(\"mixed dimensions: %s\", e)\n    # re-embed batch with a single model and retry once","preventionTips":["Pin one embedding model per palace/collection; record it via set_embedder_identity","Verify all rows share a dimension before every batch write","Never merge vectors embedded by different models into one collection"],"tags":["embeddings","dimension-mismatch","qdrant","validation"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}