MemPalace/mempalace · error · ValueError

embeddings length {len(embeddings)} does not match ids lengt

Error message

embeddings length {len(embeddings)} does not match ids length {n}

What it means

Raised by sqlite_exact's `_validate_write_batch`: the optional `embeddings` list was supplied but its length differs from `len(ids)`. Embeddings are optional, but when provided they must align with ids so each document row stores its own vector; a mismatch aborts the whole batch before any write.

Source

Thrown at mempalace/backends/sqlite_exact.py:248

            continue
        raise UnsupportedFilterError(f"where_document operator {key!r} not supported")
    return True


def _validate_write_batch(
    *,
    documents: list[str],
    ids: list[str],
    metadatas: Optional[list[dict]],
    embeddings: Optional[list[list[float]]],
) -> None:
    n = len(ids)
    if len(documents) != n:
        raise ValueError(f"documents length {len(documents)} does not match ids length {n}")
    if metadatas is not None and len(metadatas) != n:
        raise ValueError(f"metadatas length {len(metadatas)} does not match ids length {n}")
    if embeddings is not None and len(embeddings) != n:
        raise ValueError(f"embeddings length {len(embeddings)} does not match ids length {n}")


class _SQLiteExactHandle:
    def __init__(
        self,
        conn: sqlite3.Connection,
        lock: threading.RLock,
        palace_path: str,
        *,
        read_only: bool = False,
        immutable: bool = False,
    ):
        self.conn = conn
        self.lock = lock
        self.palace_path = palace_path
        self.read_only = read_only
        # True when opened with ``immutable=1`` because no WAL existed at connect
        # time. A later writer can create WAL sidecars that this connection will

View on GitHub (pinned to 06cb6987f0)

Solutions

Embed per document in the same loop that builds ids: `embeddings.append(embed(text))`.
If a single vector applies to one row only, make it a one-element batch or align lengths explicitly.
Pre-check `len(embeddings) == len(ids)` and log both values before calling upsert.

Example fix

# before
col.upsert(ids=ids, documents=docs, embeddings=[embed(docs[0])])  # 1 != N

# after
col.upsert(ids=ids, documents=docs, embeddings=[embed(d) for d in docs])

Defensive patterns

Strategy: validation

Validate before calling

def align_embeddings(ids, documents, embeddings):
    if embeddings is None:
        return None
    if len(embeddings) != len(ids):
        raise ValueError(f"embeddings {len(embeddings)} != ids {len(ids)}")
    return embeddings

Try / catch

try:
    col.upsert(ids=ids, documents=docs, embeddings=embs)
except ValueError as e:
    if "embeddings length" in str(e):
        embs = [embed(d) for d in docs]  # recompute, aligned by construction
        col.upsert(ids=ids, documents=docs, embeddings=embs)

Prevention

Compute embeddings with a list comprehension over the same documents list used for the write.
Never slice or dedupe embeddings independently of ids.
Convert numpy (N, D) arrays with .tolist() and check len == len(ids).

When it happens

Trigger: Calling upsert with `embeddings=[vec]` (single vector) while `ids` has N entries; computing embeddings only for texts that passed a filter; passing a 2-D numpy array whose first dimension does not equal the batch size without converting to a properly sized list.

Common situations: Refactoring from a single-record API to a batch API and forgetting to wrap or repeat the vector; embedding cache misses that shrink the embeddings list; slicing mismatch after deduplication of ids.

Related errors

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/0bcbb2c0f449b52c. Report an issue: GitHub.