MemPalace/mempalace · error · DimensionMismatchError

sqlite_exact collection {self._collection_name!r} cannot mix

Error message

sqlite_exact collection {self._collection_name!r} cannot mix embedding dimensions {sorted(distinct)}

What it means

Raised by `_ensure_collection_dimension` during writes: a single upsert/add batch contains embeddings of two or more different dimensionalities. sqlite_exact enforces one embedding dimension per collection so cosine-similarity search stays meaningful; mixing dimensions in one batch is rejected before any row is inserted.

Source

Thrown at mempalace/backends/sqlite_exact.py:339

        if row is None:
            raise CollectionNotInitializedError(self._collection_name)
        return int(row[0])

    def _collection_dimension(self, cur, collection_id: int) -> Optional[int]:
        row = cur.execute(
            "SELECT dimension FROM collections WHERE id = ?",
            (collection_id,),
        ).fetchone()
        if row is None or row[0] is None:
            return None
        return int(row[0])

    def _ensure_collection_dimension(self, cur, collection_id: int, dims: list[int]) -> None:
        distinct = {int(dim) for dim in dims}
        if not distinct:
            return
        if len(distinct) > 1:
            raise DimensionMismatchError(
                f"sqlite_exact collection {self._collection_name!r} cannot mix "
                f"embedding dimensions {sorted(distinct)}"
            )
        dim = distinct.pop()
        stored = self._collection_dimension(cur, collection_id)
        if stored is None:
            cur.execute(
                "UPDATE collections SET dimension = ? WHERE id = ?",
                (dim, collection_id),
            )
        elif stored != dim:
            raise DimensionMismatchError(
                f"sqlite_exact collection {self._collection_name!r} expects "
                f"embedding dimension {stored}, got {dim}"
            )

    def _fts_available(self, cur) -> bool:
        row = cur.execute("SELECT value FROM meta WHERE key = 'fts5_available'").fetchone()

View on GitHub (pinned to 06cb6987f0)

Solutions

Re-embed all vectors in the batch with one model so every vector shares a dimension, then retry the upsert.
Key your embedding cache by (text, model) so stale-dimension vectors are never reused after a model switch.
Split the batch by dimension and write each group to a differently-named collection (e.g. `drawers_384`, `drawers_1536`).
Check `get_stored_embedder_identity()` before writing to confirm the collection's model matches the current embedder.

Example fix

# before
embeddings = cached_old_vectors + [embed_new(d) for d in new_docs]  # 384 + 1536 mixed
col.upsert(ids=ids, documents=docs, embeddings=embeddings)

# after
embeddings = [embed_new(d) for d in all_docs]  # one model, one dimension
col.upsert(ids=all_ids, documents=all_docs, embeddings=embeddings)

Defensive patterns

Strategy: validation

Validate before calling

def check_uniform_dimensions(embeddings, ids):
    dims = {len(e) for e in embeddings}
    if len(dims) > 1:
        raise ValueError(f"batch mixes dimensions {sorted(dims)}; re-embed with one model")
    return dims.pop() if dims else None

Try / catch

try:
    col.upsert(ids=ids, documents=docs, embeddings=embs)
except DimensionMismatchError:
    embs = [embed(d) for d in docs]  # one model, one dimension
    col.upsert(ids=ids, documents=docs, embeddings=embs)

Prevention

Pin one embedding model per palace and record it via get_stored_embedder_identity().
Key embedding caches by (text, model_name) so cross-model vectors never mix.
After switching local models, re-embed everything instead of replaying cached vectors.

When it happens

Trigger: Calling upsert with `embeddings` where vectors have differing lengths — e.g. some from a 384-dim model (MiniLM) and some from a 768/1536-dim model; rows with `None`/empty embeddings mixed with real ones in ways that decode to different sizes; concatenating batches produced under different embedder configs.

Common situations: Changing the local embedding model (Ollama/LM Studio model swap) between incremental ingest runs and then replaying old cached vectors together with new ones; an embedding cache keyed only by text, not by model; multi-source ingest where adapters use different embedders.

Related errors

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/958d6319f1c644a0. Report an issue: GitHub.