chroma-core/chroma · error · InvalidArgumentError

transactional write request contains duplicate id "{id}"

Error message

transactional write request contains duplicate id "{id}"

What it means

Raised by _validate_buffered_write (chromadb/api/conditional_http.py:287) when the same id appears more than once within the ids list of a single transactional add/update/upsert/delete call. The transaction buffer keys writes by id to maintain read-after-write consistency and OCC preconditions, so one request cannot carry two conflicting payloads for the same record. The check uses a per-call set (call_ids) and fires before anything is buffered.

Source

Thrown at chromadb/api/conditional_http.py:287

                f"{self._read_token} to {actual_read_token}"
            )

    def _buffer_write(
        self,
        operation: str,
        ids: IDs,
        payload: ConditionalHttpJsonPayload,
    ) -> None:
        self._validate_buffered_write(operation, ids)
        for id in ids:
            self._buffered_write_ids.add(id)
        self._operations.append({"operation": operation, "payload": payload})

    def _validate_buffered_write(self, operation: str, ids: IDs) -> None:
        call_ids: Set[str] = set()
        for id in ids:
            if id in call_ids:
                raise InvalidArgumentError(
                    f'transactional write request contains duplicate id "{id}"'
                )
            call_ids.add(id)
            if id in self._buffered_write_ids:
                raise InvalidArgumentError(
                    f'transaction already has a buffered write for id "{id}"'
                )
            self._validate_write_precondition(operation, id)

    def _validate_write_precondition(self, operation: str, id: str) -> None:
        if operation == "add" and id not in self._known_absent:
            raise InvalidArgumentError(
                f'transactional add for id "{id}" requires a prior read '
                "proving the id is absent"
            )
        if operation == "update" and id not in self._known_present:
            raise InvalidArgumentError(
                f'transactional update for id "{id}" requires a prior read '

View on GitHub (pinned to aecdd12c8a)

Solutions

1. Deduplicate ids (preserving order) before the call and align the parallel embeddings/documents/metadata lists to the deduped ids
2. If the later occurrence should win, split into sequential calls only after a read, or use upsert semantics
3. Fix the upstream id generation so duplicates cannot be produced

Example fix

# before
collection.add(ids=["a", "b", "a"], embeddings=[e1, e2, e3])
# after
ids = list(dict.fromkeys(["a", "b", "a"]))   # ordered dedupe -> ["a", "b"]
collection.add(ids=ids, embeddings=[e1, e2])

def dedupe_ids(ids):
    return list(dict.fromkeys(ids))  # order-preserving unique ids

ids = dedupe_ids(ids)
assert len(ids) == len(embeddings), "parallel arrays must match deduped ids"

When it happens

Trigger: A single transactional write whose ids argument contains duplicates, e.g. collection.add(ids=["a", "b", "a"], ...) inside a transaction; duplicates introduced by concatenating lists or generating ids with a non-unique key without deduping.

Common situations: Merging batches from multiple sources into one add() call without deduping; ID generation collisions (same hash or slug produced twice) feeding one call; off-by-one slicing that repeats the tail of a list.

Related errors

• transaction already has a buffered write for id "{id}"
• conditional transaction has no collection scope
• transactional filter reads require a positive limit
• transactional add for id "{id}" requires a prior read provin
• transactional update for id "{id}" requires a prior read pro

AI-assisted analysis of chroma-core/chroma@aecdd12c8a (2026-08-16). Data as JSON: /api/errors/5c649faed97c0e33. Report an issue: GitHub.