{"record":{"id":"72a6e095751918ee","repo":"RyanCodrai/turbovec","slug":"expected-2d-embedding-batch-got-vectors-ndim-d-72a6e0","errorCode":null,"errorMessage":"expected 2D embedding batch, got {vectors.ndim}D","messagePattern":"expected 2D embedding batch, got (.+?)D","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"turbovec-python/python/turbovec/haystack.py","lineNumber":337,"sourceCode":"        add the vectors to the index (dropping ``to_remove``'s old\n        handles last, index first).\n\n        Maps BEFORE the index add: a concurrent retrieval can only learn\n        a handle from the index, so an entry that is resolvable but not\n        yet searchable is invisible to readers (issue #161). Still\n        all-or-nothing with respect to the given batch: validation\n        precedes any mutation, and if the index add fails the\n        pre-inserted map entries are unwound (restoring the previous\n        mapping of any overwritten id), so a failure leaves the store\n        exactly as it was (issue #89). The FAIL path calls this with\n        single-document batches to get per-document commit semantics.\n        Callers hold the writer lock.\n        \"\"\"\n        vectors = np.asarray(\n            [doc.embedding for doc in to_write], dtype=np.float32\n        )\n        if vectors.ndim != 2:\n            raise ValueError(\n                f\"expected 2D embedding batch, got {vectors.ndim}D\"\n            )\n        # A batch of empty per-document embeddings has shape (N, 0) — 2D,\n        # so it passes the ndim guard, then dies deep in the index kernel\n        # with an opaque buffer-length error. Name the real cause instead.\n        if vectors.shape[1] == 0:\n            raise ValueError(\n                \"documents have empty embeddings (dim 0); check the \"\n                \"embedder that produced them\"\n            )\n        # IdMapIndex.add_with_ids handles both eager (dim must match) and\n        # lazy (locks dim on first call) cases. Surface its mismatch\n        # panic as a clean ValueError for parity with previous behaviour.\n        existing_dim = self._index.dim\n        if existing_dim is not None and vectors.shape[1] != existing_dim:\n            raise ValueError(\n                f\"embedding dim {vectors.shape[1]} does not match store dim {existing_dim}\"\n            )","sourceCodeStart":319,"sourceCodeEnd":355,"githubUrl":"https://github.com/RyanCodrai/turbovec/blob/ccab9f325e6ce2a270a87daf01ae4e443bcf2d49/turbovec-python/python/turbovec/haystack.py#L319-L355","documentation":"Raised in _commit_batch after stacking the batch's embeddings with np.asarray: the result is not a 2D (N, dim) matrix. Since write_documents already rejects missing embeddings, this points at malformed embedding payloads — ragged lists, scalars, or non-uniform shapes — that cannot form a uniform matrix.","triggerScenarios":"Thrown at turbovec-python/python/turbovec/haystack.py:337 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Inspect the offending documents' embeddings for ragged lengths or nested arrays and re-embed with a healthy embedder.","Ensure every document's embedding is a flat 1D sequence of floats of equal length.","Catch the ValueError to abort the write before any index mutation (validation precedes mutation, so state is unchanged)."],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"ccab9f325e6ce2a270a87daf01ae4e443bcf2d49","analyzedAt":"2026-09-06T08:39:18.516Z","contentChangedAt":"2026-09-06T08:39:18.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}