MemPalace/mempalace · error · ValueError

ids already exist in qdrant collection: {existing.ids}

Error message

ids already exist in qdrant collection: {existing.ids}

What it means

Raised by QdrantCollection.add() when some of the requested ids already exist in the collection (checked via a get() pre-flight). add() is insert-only; overwriting existing points requires upsert(). The message includes the offending ids.

Source

Thrown at mempalace/backends/qdrant.py:827

            and _matches_where(row["metadata"], where)
            and _matches_where_document(row["document"], where_document)
        ]
        return rows

    def add(self, *, documents, ids, metadatas=None, embeddings=None):
        _validate_write_batch(
            documents=documents,
            ids=ids,
            metadatas=metadatas,
            embeddings=embeddings,
        )
        if embeddings is None:
            raise ValueError("qdrant requires explicit embeddings")
        if len(set(ids)) != len(ids):
            raise ValueError("add ids must be unique")
        existing = self.get(ids=list(ids), include=[])
        if existing.ids:
            raise ValueError(f"ids already exist in qdrant collection: {existing.ids}")
        self.upsert(documents=documents, ids=ids, metadatas=metadatas, embeddings=embeddings)

    def upsert(self, *, documents, ids, metadatas=None, embeddings=None):
        _validate_write_batch(
            documents=documents,
            ids=ids,
            metadatas=metadatas,
            embeddings=embeddings,
        )
        if embeddings is None:
            raise ValueError("qdrant requires explicit embeddings")
        vectors, dimension = _normalize_vectors(embeddings)
        self._ensure_remote_collection(dimension)
        metadatas = metadatas or [{} for _ in ids]
        points = []
        for doc_id, doc, meta, vector in zip(ids, documents, metadatas, vectors):
            points.append(
                {

View on GitHub (pinned to 06cb6987f0)

Solutions

1. Switch to upsert() if overwriting is the intended behavior (idempotent re-ingest)
2. Or skip existing ids: filter out the ones named in the error before retrying add()
3. If duplicate ingest indicates a bug (same content filed twice), fix the upstream trigger (e.g. hook firing twice) instead of the write path
4. Keep a manifest of ingested ids per source to make re-ingest checks cheap

Example fix

# before
collection.add(documents=docs, ids=ids, embeddings=embs)  # ValueError: ids already exist
// after
existing = set(collection.get(ids=ids, include=[]).ids)
new = [(d,i,e) for d,i,e in zip(docs,ids,embs) if i not in existing]
if new:
    collection.upsert(documents=[d for d,_,_ in new], ids=[i for _,i,_ in new], embeddings=[e for _,_,e in new])

existing = set(collection.get(ids=ids, include=[]).ids)
fresh = [(d, i, e) for d, i, e in zip(docs, ids, embs) if i not in existing]
if fresh:
    collection.upsert(*[list(x) for x in zip(*fresh)])

try:
    collection.add(documents=docs, ids=ids, embeddings=embs)
except ValueError as e:
    if "already exist" in str(e):
        collection.upsert(documents=docs, ids=ids, embeddings=embs)  # idempotent retry

When it happens

Trigger: Calling add() with ids that were stored in a previous run — typical in retry logic after a partial failure, or idempotent re-ingest of the same transcript/file without a freshness check.

Common situations: Re-running an ingest pipeline after a crash; a hook firing twice for the same session; deterministic chunk ids colliding with yesterday's data; duplicate file processing.

Related errors

• add ids must be unique
• qdrant requires explicit embeddings
• update requires at least one of documents, metadatas, embedd
• {label} length {len(value)} does not match ids length {n}
• qdrant requires query_embeddings; use palace.get_collection

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/5ebd9c6ffd95b0c1. Report an issue: GitHub.