{"record":{"id":"0bcbb2c0f449b52c","repo":"MemPalace/mempalace","slug":"embeddings-length-len-embeddings-does-not-match-0bcbb2","errorCode":null,"errorMessage":"embeddings length {len(embeddings)} does not match ids length {n}","messagePattern":"embeddings length (.+?) does not match ids length (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/sqlite_exact.py","lineNumber":248,"sourceCode":"            continue\n        raise UnsupportedFilterError(f\"where_document operator {key!r} not supported\")\n    return True\n\n\ndef _validate_write_batch(\n    *,\n    documents: list[str],\n    ids: list[str],\n    metadatas: Optional[list[dict]],\n    embeddings: Optional[list[list[float]]],\n) -> None:\n    n = len(ids)\n    if len(documents) != n:\n        raise ValueError(f\"documents length {len(documents)} does not match ids length {n}\")\n    if metadatas is not None and len(metadatas) != n:\n        raise ValueError(f\"metadatas length {len(metadatas)} does not match ids length {n}\")\n    if embeddings is not None and len(embeddings) != n:\n        raise ValueError(f\"embeddings length {len(embeddings)} does not match ids length {n}\")\n\n\nclass _SQLiteExactHandle:\n    def __init__(\n        self,\n        conn: sqlite3.Connection,\n        lock: threading.RLock,\n        palace_path: str,\n        *,\n        read_only: bool = False,\n        immutable: bool = False,\n    ):\n        self.conn = conn\n        self.lock = lock\n        self.palace_path = palace_path\n        self.read_only = read_only\n        # True when opened with ``immutable=1`` because no WAL existed at connect\n        # time. A later writer can create WAL sidecars that this connection will","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/sqlite_exact.py#L230-L266","documentation":"Raised by sqlite_exact's `_validate_write_batch`: the optional `embeddings` list was supplied but its length differs from `len(ids)`. Embeddings are optional, but when provided they must align with ids so each document row stores its own vector; a mismatch aborts the whole batch before any write.","triggerScenarios":"Calling upsert with `embeddings=[vec]` (single vector) while `ids` has N entries; computing embeddings only for texts that passed a filter; passing a 2-D numpy array whose first dimension does not equal the batch size without converting to a properly sized list.","commonSituations":"Refactoring from a single-record API to a batch API and forgetting to wrap or repeat the vector; embedding cache misses that shrink the embeddings list; slicing mismatch after deduplication of ids.","solutions":["Embed per document in the same loop that builds ids: `embeddings.append(embed(text))`.","If a single vector applies to one row only, make it a one-element batch or align lengths explicitly.","Pre-check `len(embeddings) == len(ids)` and log both values before calling upsert."],"exampleFix":"# before\ncol.upsert(ids=ids, documents=docs, embeddings=[embed(docs[0])])  # 1 != N\n\n# after\ncol.upsert(ids=ids, documents=docs, embeddings=[embed(d) for d in docs])","handlingStrategy":"validation","validationCode":"def align_embeddings(ids, documents, embeddings):\n    if embeddings is None:\n        return None\n    if len(embeddings) != len(ids):\n        raise ValueError(f\"embeddings {len(embeddings)} != ids {len(ids)}\")\n    return embeddings","typeGuard":null,"tryCatchPattern":"try:\n    col.upsert(ids=ids, documents=docs, embeddings=embs)\nexcept ValueError as e:\n    if \"embeddings length\" in str(e):\n        embs = [embed(d) for d in docs]  # recompute, aligned by construction\n        col.upsert(ids=ids, documents=docs, embeddings=embs)","preventionTips":["Compute embeddings with a list comprehension over the same documents list used for the write.","Never slice or dedupe embeddings independently of ids.","Convert numpy (N, D) arrays with .tolist() and check len == len(ids)."],"tags":["sqlite-exact","batch-write","embeddings","validation"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}