{"record":{"id":"b2dd09a1215b2191","repo":"run-llama/llama_index","slug":"node-id-node-id-already-exists-set-allow-update","errorCode":null,"errorMessage":"node_id {node_id} already exists. Set allow_update to True to overwrite.","messagePattern":"node_id (.+?) already exists\\. Set allow_update to True to overwrite\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/storage/docstore/keyval_docstore.py","lineNumber":176,"sourceCode":"        Returns:\n            Tuple[\n                list,          # List of key-value pairs for nodes\n                list,          # List of key-value pairs for metadata\n                List[Tuple[str, dict]]  # Dictionary of key-value pairs for reference documents, keyed by ref_doc_id\n            ]\n\n        Raises:\n            ValueError: If a node already exists in the store and `allow_update` is False.\n\n        \"\"\"\n        node_kv_pairs = []\n        metadata_kv_pairs = []\n        ref_doc_kv_pairs: Dict[str, List[Tuple[str, dict]]] = {}\n\n        for node in nodes:\n            # NOTE: doc could already exist in the store, but we overwrite it\n            if not allow_update and self.document_exists(node.node_id):\n                raise ValueError(\n                    f\"node_id {node.node_id} already exists. \"\n                    \"Set allow_update to True to overwrite.\"\n                )\n            ref_doc_info = None\n            if node.source_node is not None:\n                ref_doc_info = (\n                    self.get_ref_doc_info(node.source_node.node_id) or RefDocInfo()\n                )\n\n            (\n                node_kv_pair,\n                metadata_kv_pair,\n                ref_doc_kv_pair,\n            ) = self._get_kv_pairs_for_insert(node, ref_doc_info, store_text)\n\n            if node_kv_pair is not None:\n                node_kv_pairs.append(node_kv_pair)\n            if metadata_kv_pair is not None:","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/storage/docstore/keyval_docstore.py#L158-L194","documentation":"KeyValDocumentStore.add_documents(nodes, allow_update=False) (sync) refuses to overwrite: for each node it checks document_exists(node.node_id) and raises if the ID is already stored. The default call path in DocumentStore.add() passes allow_update=True, so this fires only when allow_update=False was explicitly chosen.","triggerScenarios":"docstore.add_documents(nodes, allow_update=False) (or DocumentStore.add(..., allow_update=False)) where any node's node_id already exists -- typical when re-ingesting the same document without deleting it first, since node IDs are deterministic hashes of content+metadata.","commonSituations":"Idempotency-guarded ingestion jobs that intentionally set allow_update=False to detect duplicates; re-running a failed pipeline; deterministic SentenceSplitter producing identical node IDs on unchanged documents.","solutions":["Pass allow_update=True when re-ingesting unchanged documents so existing nodes are overwritten.","Delete before re-adding: docstore.delete_document(doc_id) for the stale document, then add again.","Use docstore.document_exists(node.node_id) (skip-if-exists) per node when you want dedupe instead of failure.","For ingestion pipelines, rely on the document management / docstore strategy (upserts) instead of manual add with allow_update=False."],"exampleFix":"# before\nnodes = splitter.get_nodes_from_documents([doc])\ndocstore.add_documents(nodes, allow_update=False)  # ValueError on re-run\n\n# after\nif not docstore.document_exists(nodes[0].node_id):\n    docstore.add_documents(nodes)\nelse:\n    docstore.delete_document(doc.doc_id)\n    docstore.add_documents(nodes)  # allow_update defaults to True","handlingStrategy":"validation","validationCode":"def add_documents_idempotent(docstore, nodes) -> None:\n    new_nodes = [n for n in nodes if not docstore.document_exists(n.node_id)]\n    if new_nodes:\n        docstore.add_documents(new_nodes)  # allow_update defaults to True","typeGuard":"def all_nodes_absent(docstore, nodes) -> bool:\n    return not any(docstore.document_exists(n.node_id) for n in nodes)","tryCatchPattern":"try:\n    docstore.add_documents(nodes, allow_update=False)\nexcept ValueError as e:\n    if 'already exists' in str(e):\n        docstore.add_documents(nodes, allow_update=True)  # overwrite on conflict\n    else:\n        raise","preventionTips":["Default to allow_update=True for re-runnable ingestion.","Delete the source document (delete_document/delete_ref_doc) before re-ingesting it.","Filter already-present node IDs with document_exists before adding."],"tags":["docstore","ingestion","duplicate","idempotency","llama-index"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}