{"record":{"id":"7068ee8cc6ce6bbc","repo":"langchain-ai/langchain","slug":"source-id-key-is-required-when-cleanup-mode-is-inc","errorCode":null,"errorMessage":"Source id key is required when cleanup mode is incremental or scoped_full.","messagePattern":"Source id key is required when cleanup mode is incremental or scoped_full\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"libs/core/langchain_core/indexing/api.py","lineNumber":422,"sourceCode":"            (should be unreachable code).\n    \"\"\"\n    # Behavior is deprecated, but we keep it for backwards compatibility.\n    # # Warn only once per process.\n    if key_encoder == \"sha1\":\n        _warn_about_sha1()\n\n    if cleanup not in {\"incremental\", \"full\", \"scoped_full\", None}:\n        msg = (\n            f\"cleanup should be one of 'incremental', 'full', 'scoped_full' or None. \"\n            f\"Got {cleanup}.\"\n        )\n        raise ValueError(msg)\n\n    if (cleanup in {\"incremental\", \"scoped_full\"}) and source_id_key is None:\n        msg = (\n            \"Source id key is required when cleanup mode is incremental or scoped_full.\"\n        )\n        raise ValueError(msg)\n\n    destination = vector_store  # Renaming internally for clarity\n\n    # If it's a vectorstore, let's check if it has the required methods.\n    if isinstance(destination, VectorStore):\n        # Check that the Vectorstore has required methods implemented\n        methods = [\"delete\", \"add_documents\"]\n\n        for method in methods:\n            if not hasattr(destination, method):\n                msg = (\n                    f\"Vectorstore {destination} does not have required method {method}\"\n                )\n                raise ValueError(msg)\n\n        if type(destination).delete == VectorStore.delete:\n            # Checking if the VectorStore has overridden the default delete method\n            # implementation which just raises a NotImplementedError","sourceCodeStart":404,"sourceCodeEnd":440,"githubUrl":"https://github.com/langchain-ai/langchain/blob/e32fa9a52eab3b61ad7a45399bfde59b3e580fc4/libs/core/langchain_core/indexing/api.py#L404-L440","documentation":"Raised in `index()` when cleanup mode is 'incremental' or 'scoped_full' but `source_id_key` is None. These two cleanup modes delete documents whose upstream source disappeared, which is only possible if every indexed document carries a source identifier. Without `source_id_key` the indexer cannot map documents to sources, so it refuses to run rather than silently skipping cleanup.","triggerScenarios":"Calling `index(vs, docs, rm, cleanup=\"incremental\")` with no `source_id_key`; switching cleanup from \"full\"/None to \"incremental\" in existing code without adding the key.","commonSituations":"Incremental sync jobs over a directory or website where files/pages get removed; upgrading a prototype (cleanup=None) to production incremental mode.","solutions":["Pass a metadata key present on every document: `source_id_key=\"source\"` (and populate it in your loader).","Or use a callable: `source_id_key=lambda doc: doc.metadata[\"url\"]`.","If you genuinely cannot attribute sources, use `cleanup=\"full\"` instead and accept the cost."],"exampleFix":"# before\nindex(vs, docs, rm, cleanup=\"incremental\")\n\n# after\n# loader sets metadata={\"source\": str(path)}\nindex(vs, docs, rm, cleanup=\"incremental\", source_id_key=\"source\")","handlingStrategy":"validation","validationCode":"if cleanup in {\"incremental\", \"scoped_full\"} and source_id_key is None:\n    raise ValueError(\"source_id_key required for this cleanup mode — check config\")\nindex(vs, docs, rm, cleanup=cleanup, source_id_key=source_id_key)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pair incremental/scoped_full cleanup with a source_id_key in one config struct so they cannot diverge.","Ensure the loader stamps every document with the source metadata key."],"tags":["indexing","cleanup","source-id"],"backgroundTag":null,"analyzedSha":"e32fa9a52eab3b61ad7a45399bfde59b3e580fc4","analyzedAt":"2026-08-14T18:42:09.092Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}