MemPalace/mempalace · error · DimensionMismatchError

sqlite_exact collection {self._collection_name!r} expects em

Error message

sqlite_exact collection {self._collection_name!r} expects embedding dimension {stored}, got {dim}

What it means

Raised by `_ensure_collection_dimension` during writes: every embedding in the batch has a consistent dimension, but it differs from the dimension already recorded in the `collections` table for this collection. The stored dimension is set once (first write) and then enforced, because a collection's vectors must be mutually comparable for cosine search.

Source

Thrown at mempalace/backends/sqlite_exact.py:351

    def _ensure_collection_dimension(self, cur, collection_id: int, dims: list[int]) -> None:
        distinct = {int(dim) for dim in dims}
        if not distinct:
            return
        if len(distinct) > 1:
            raise DimensionMismatchError(
                f"sqlite_exact collection {self._collection_name!r} cannot mix "
                f"embedding dimensions {sorted(distinct)}"
            )
        dim = distinct.pop()
        stored = self._collection_dimension(cur, collection_id)
        if stored is None:
            cur.execute(
                "UPDATE collections SET dimension = ? WHERE id = ?",
                (dim, collection_id),
            )
        elif stored != dim:
            raise DimensionMismatchError(
                f"sqlite_exact collection {self._collection_name!r} expects "
                f"embedding dimension {stored}, got {dim}"
            )

    def _fts_available(self, cur) -> bool:
        row = cur.execute("SELECT value FROM meta WHERE key = 'fts5_available'").fetchone()
        return bool(row and row[0] == "1")

    def _embedder_meta_key(self) -> str:
        return f"embedder_model:{self._collection_name}"

    def get_stored_embedder_identity(self):
        from .base import EmbedderIdentity

        with self._cursor() as cur:
            try:
                cid = self._collection_id(cur)
            except CollectionNotInitializedError:

View on GitHub (pinned to 06cb6987f0)

Solutions

1. If the model change is intentional, start a fresh collection/palace (or delete and recreate the collection) and re-ingest all content with the new model.
2. Otherwise, revert to the embedder whose dimension matches the stored one — check `get_stored_embedder_identity()` / the collection's dimension to find it.
3. Pin the embedding model in config so palace and pipeline cannot drift.
4. Never mix old cached vectors with a new model's vectors in the same collection.

Example fix

# before
# palace built with 768-dim model, now:
col.upsert(ids=ids, documents=docs, embeddings=[embed_mxbai(d) for d in docs])  # 1024-dim

# after
backend.delete_collection(palace, "drawers")
col = backend.get_collection(palace, "drawers", create=True)
col.upsert(ids=all_ids, documents=all_docs, embeddings=[embed_mxbai(d) for d in all_docs])

def check_dims_match_collection(col, embeddings):
    # decode expected dim from the collection
    with col._cursor() as cur:
        cid = col._collection_id(cur)
        expected = col._collection_dimension(cur, cid)
    if expected is not None and any(len(e) != expected for e in embeddings):
        raise ValueError(f"collection expects dim {expected}; re-embed or rebuild collection")

try:
    col.upsert(ids=ids, documents=docs, embeddings=embs)
except DimensionMismatchError as e:
    logger.warning("dimension change detected (%s); rebuilding collection", e)
    backend.delete_collection(palace, name)
    col = backend.get_collection(palace, name, create=True)
    col.upsert(ids=all_ids, documents=all_docs, embeddings=[embed(d) for d in all_docs])

When it happens

Trigger: Writing to an existing collection with a new embedder of different size — e.g. collection built with 768-dim vectors, now upserting 1536-dim vectors; switching from one Ollama model to another and re-mining into the same palace.

Common situations: Upgrading the local embedding model and continuing incremental ingest into an existing palace; test fixtures that create collections with one embedder then run the suite with another; copying a palace built elsewhere with a different model config.

Related errors

• sqlite_exact collection {self._collection_name!r} cannot mix
• sqlite_exact collection {self._collection_name!r} expects em
• milvus batch cannot mix embedding dimensions {sorted(dims)}
• pgvector collection {self._collection_name!r} expects embedd
• qdrant batch cannot mix embedding dimensions {sorted(dims)}

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/6fd7b2ad74b3dc1f. Report an issue: GitHub.