{"record":{"id":"e5af2951b9ea687f","repo":"MemPalace/mempalace","slug":"embeddings-length-len-embeddings-does-not-match-e5af29","errorCode":null,"errorMessage":"embeddings length {len(embeddings)} does not match ids length {n}","messagePattern":"embeddings length (.+?) does not match ids length (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/qdrant.py","lineNumber":233,"sourceCode":"            continue\n        raise UnsupportedFilterError(f\"where_document operator {key!r} not supported\")\n    return True\n\n\ndef _validate_write_batch(\n    *,\n    documents: list[str],\n    ids: list[str],\n    metadatas: Optional[list[dict]],\n    embeddings: Optional[list[list[float]]],\n) -> None:\n    n = len(ids)\n    if len(documents) != n:\n        raise ValueError(f\"documents length {len(documents)} does not match ids length {n}\")\n    if metadatas is not None and len(metadatas) != n:\n        raise ValueError(f\"metadatas length {len(metadatas)} does not match ids length {n}\")\n    if embeddings is not None and len(embeddings) != n:\n        raise ValueError(f\"embeddings length {len(embeddings)} does not match ids length {n}\")\n\n\ndef _as_vector_array(vector: list[float]) -> np.ndarray:\n    arr = np.asarray(vector, dtype=np.float32)\n    if arr.ndim != 1 or arr.size == 0:\n        raise ValueError(\"embedding must be a non-empty 1D vector\")\n    return arr\n\n\ndef _normalize_vectors(embeddings: list[list[float]]) -> tuple[list[list[float]], int]:\n    vectors = []\n    dims = set()\n    for embedding in embeddings:\n        arr = _as_vector_array(embedding)\n        vectors.append(arr.astype(float).tolist())\n        dims.add(int(arr.size))\n    if len(dims) > 1:\n        raise DimensionMismatchError(f\"qdrant batch cannot mix embedding dimensions {sorted(dims)}\")","sourceCodeStart":215,"sourceCodeEnd":251,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/qdrant.py#L215-L251","documentation":"_validate_write_batch requires that when embeddings is provided its length matches len(ids). It fails fast with ValueError before normalization or any Qdrant write, preventing batches where vectors and documents are misaligned.","triggerScenarios":"upsert(ids=[\"1\",\"2\"], documents=[d1,d2], embeddings=[[...]]) — e.g. embedding call returned fewer vectors than texts because one input was skipped or an API returned a truncated batch.","commonSituations":"Batch embedding endpoints that drop empty strings; retry logic re-requesting embeddings for a subset; caching layers that return partial results.","solutions":["Verify len(embeddings) == len(ids) right after the embedding step and fail loudly there.","Make your embed function always return exactly one vector per input (embed empty strings to a zero vector or filter inputs and ids together).","Log input/output counts from the embedder to catch truncation early."],"exampleFix":"# before\nvecs = embed_client.embed([t for t in texts if t])  # may be shorter\ncol.add(ids=ids, documents=texts, embeddings=vecs)\n\n# after\nvecs = embed_client.embed(texts)\nassert len(vecs) == len(ids) == len(texts)\ncol.add(ids=ids, documents=texts, embeddings=vecs)","handlingStrategy":"validation","validationCode":"vecs = embed(texts)\nif len(vecs) != len(ids):\n    raise RuntimeError(f\"embedder returned {len(vecs)} vectors for {len(ids)} ids\")\ncol.add(ids=ids, documents=texts, embeddings=vecs)","typeGuard":"def embeddings_aligned(ids, embeddings) -> bool:\n    return embeddings is None or len(embeddings) == len(ids)","tryCatchPattern":"try:\n    col.add(ids=ids, documents=docs, embeddings=vecs)\nexcept ValueError as e:\n    if \"embeddings length\" in str(e):\n        logger.error(\"embedding batch truncated: %d/%d\", len(vecs), len(ids))\n        raise","preventionTips":["Check the embedder's output count immediately after every batch call.","Do not pre-filter texts before embedding while keeping the full ids list.","Wrap the embedder so it guarantees one vector per input."],"tags":["qdrant","validation","batch-mismatch","embeddings"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}