{"record":{"id":"85b8ca1e0f18de1a","repo":"RyanCodrai/turbovec","slug":"id-doc-id-already-exists-in-the-document-store","errorCode":null,"errorMessage":"ID '{doc.id}' already exists in the document store.","messagePattern":"ID '(.+?)' already exists in the document store\\.","errorType":"exception","errorClass":"DuplicateDocumentError","httpStatus":null,"severity":"error","filePath":"turbovec-python/python/turbovec/haystack.py","lineNumber":262,"sourceCode":"        if policy == DuplicatePolicy.FAIL:\n            # Reference parity (issue #167): InMemoryDocumentStore commits\n            # each document as it iterates and raises on the *first*\n            # duplicate, so every non-duplicate document preceding it\n            # stays persisted — a partial write. Mirror that observable\n            # state exactly: validate and commit per document, raising on\n            # the first collision. A repeated id within a single call\n            # collides with its already-committed first instance, the same\n            # way a cross-call repeat would. Each individual commit is\n            # still all-or-nothing — validation precedes any mutation, so\n            # a failing document mid-batch never leaves the index and the\n            # id maps inconsistent (#89/#139 apply per document).\n            written = 0\n            for doc in documents:\n                if doc.id in self._str_to_u64:\n                    # Checked before embedding validation: the reference\n                    # raises DuplicateDocumentError for a colliding id\n                    # regardless of the document's other fields.\n                    raise DuplicateDocumentError(\n                        f\"ID '{doc.id}' already exists in the document store.\"\n                    )\n                if doc.embedding is None:\n                    raise ValueError(\n                        f\"Document {doc.id!r} has no embedding. \"\n                        \"TurboQuantDocumentStore only stores documents with precomputed \"\n                        \"embeddings — run an embedder component before writing.\"\n                    )\n                self._commit_batch([doc])\n                written += 1\n            return written\n\n        # SKIP / OVERWRITE: first pass validates and resolves duplicates\n        # against the batch-so-far as well as the existing store:\n        # InMemoryDocumentStore writes into its dict as it iterates, so a\n        # repeated id *within a single call* is resolved the same way a\n        # cross-call repeat would be. Without tracking the batch, every\n        # duplicate row still gets its own vector while _str_to_u64 keeps","sourceCodeStart":244,"sourceCodeEnd":280,"githubUrl":"https://github.com/RyanCodrai/turbovec/blob/ccab9f325e6ce2a270a87daf01ae4e443bcf2d49/turbovec-python/python/turbovec/haystack.py#L244-L280","documentation":"DuplicateDocumentError raised in _write_documents_locked when policy is FAIL (the default: NONE resolves to FAIL) and a document's id already exists in the store. Committed per document in iteration order, matching InMemoryDocumentStore's partial-write semantics: documents before the collision stay persisted.","triggerScenarios":"Thrown at turbovec-python/python/turbovec/haystack.py:262 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Pass policy=DuplicatePolicy.SKIP to silently skip existing ids, or OVERWRITE to replace them.","Deduplicate the batch against existing ids (filter_documents / count_documents) before writing.","Catch DuplicateDocumentError and retry the remaining documents under a non-FAIL policy."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"ccab9f325e6ce2a270a87daf01ae4e443bcf2d49","analyzedAt":"2026-09-06T08:39:18.516Z","contentChangedAt":"2026-09-06T08:39:18.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}