MemPalace/mempalace · error · ValueError

add ids must be unique

Error message

add ids must be unique

What it means

Error "add ids must be unique" thrown in MemPalace/mempalace.

Source

Thrown at mempalace/backends/milvus.py:534

                raise ValueError(
                    f"row {idx}: document byte length {document_bytes} exceeds "
                    f"Milvus VARCHAR limit {DOCUMENT_MAX_LENGTH}; chunk before storing"
                )
            row = {
                FIELD_ID: doc_id,
                FIELD_DOCUMENT: document,
                FIELD_METADATA: _jsonable_metadata(metadata),
                FIELD_VECTOR: vector,
            }
            row.update(row[FIELD_METADATA])
            rows.append(row)
        return rows, dimension

    def add(self, *, documents, ids, metadatas=None, embeddings=None):
        if embeddings is None:
            raise ValueError("milvus requires explicit embeddings")
        if len(set(ids)) != len(ids):
            raise ValueError("add ids must be unique")
        rows, dimension = self._prepare_rows(
            documents=documents,
            ids=ids,
            metadatas=metadatas,
            embeddings=embeddings,
        )
        if not rows:
            return
        if self._remote_exists():
            existing = self.get(ids=list(ids), include=[])
            if existing.ids:
                raise ValueError(f"ids already exist in milvus collection: {existing.ids}")
        self._ensure_remote_collection(dimension)
        self._client.insert(collection_name=self._remote_collection, data=rows)
        self._backend._write_marker(self._palace, self._config)

    def upsert(self, *, documents, ids, metadatas=None, embeddings=None):
        if embeddings is None:

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Deduplicate the ids list before calling add; each id may appear only once

When it happens

Trigger: Thrown at mempalace/backends/milvus.py:534 when the library encounters an invalid state.

Common situations: Upstream chunker produced colliding drawer ids within a single batch.

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/6ba72b89b418446f. Report an issue: GitHub.