run-llama/llama_index · error · ValueError

node_id {node_id} already exists. Set allow_update to True t

Error message

node_id {node_id} already exists. Set allow_update to True to overwrite.

What it means

KeyValDocumentStore.add_documents(nodes, allow_update=False) (sync) refuses to overwrite: for each node it checks document_exists(node.node_id) and raises if the ID is already stored. The default call path in DocumentStore.add() passes allow_update=True, so this fires only when allow_update=False was explicitly chosen.

Source

Thrown at llama-index-core/llama_index/core/storage/docstore/keyval_docstore.py:176

        Returns:
            Tuple[
                list,          # List of key-value pairs for nodes
                list,          # List of key-value pairs for metadata
                List[Tuple[str, dict]]  # Dictionary of key-value pairs for reference documents, keyed by ref_doc_id
            ]

        Raises:
            ValueError: If a node already exists in the store and `allow_update` is False.

        """
        node_kv_pairs = []
        metadata_kv_pairs = []
        ref_doc_kv_pairs: Dict[str, List[Tuple[str, dict]]] = {}

        for node in nodes:
            # NOTE: doc could already exist in the store, but we overwrite it
            if not allow_update and self.document_exists(node.node_id):
                raise ValueError(
                    f"node_id {node.node_id} already exists. "
                    "Set allow_update to True to overwrite."
                )
            ref_doc_info = None
            if node.source_node is not None:
                ref_doc_info = (
                    self.get_ref_doc_info(node.source_node.node_id) or RefDocInfo()
                )

            (
                node_kv_pair,
                metadata_kv_pair,
                ref_doc_kv_pair,
            ) = self._get_kv_pairs_for_insert(node, ref_doc_info, store_text)

            if node_kv_pair is not None:
                node_kv_pairs.append(node_kv_pair)
            if metadata_kv_pair is not None:

View on GitHub (pinned to afd0fef371)

Solutions

1. Pass allow_update=True when re-ingesting unchanged documents so existing nodes are overwritten.
2. Delete before re-adding: docstore.delete_document(doc_id) for the stale document, then add again.
3. Use docstore.document_exists(node.node_id) (skip-if-exists) per node when you want dedupe instead of failure.
4. For ingestion pipelines, rely on the document management / docstore strategy (upserts) instead of manual add with allow_update=False.

Example fix

# before
nodes = splitter.get_nodes_from_documents([doc])
docstore.add_documents(nodes, allow_update=False)  # ValueError on re-run

# after
if not docstore.document_exists(nodes[0].node_id):
    docstore.add_documents(nodes)
else:
    docstore.delete_document(doc.doc_id)
    docstore.add_documents(nodes)  # allow_update defaults to True

def add_documents_idempotent(docstore, nodes) -> None:
    new_nodes = [n for n in nodes if not docstore.document_exists(n.node_id)]
    if new_nodes:
        docstore.add_documents(new_nodes)  # allow_update defaults to True

def all_nodes_absent(docstore, nodes) -> bool:
    return not any(docstore.document_exists(n.node_id) for n in nodes)

try:
    docstore.add_documents(nodes, allow_update=False)
except ValueError as e:
    if 'already exists' in str(e):
        docstore.add_documents(nodes, allow_update=True)  # overwrite on conflict
    else:
        raise

When it happens

Trigger: docstore.add_documents(nodes, allow_update=False) (or DocumentStore.add(..., allow_update=False)) where any node's node_id already exists -- typical when re-ingesting the same document without deleting it first, since node IDs are deterministic hashes of content+metadata.

Common situations: Idempotency-guarded ingestion jobs that intentionally set allow_update=False to detect duplicates; re-running a failed pipeline; deterministic SentenceSplitter producing identical node IDs on unchanged documents.

Related errors

• First argument to Readability constructor should be a docume
• Aborting parsing document; {numTags} elements found
• Command failed: {command} {result.stderr}
• Git command failed: {result.stderr}
• Node ID {node_id_str} not found in fetched nodes.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/b2dd09a1215b2191. Report an issue: GitHub.