langchain-ai/langchain · error · ValueError

Source id key is required when cleanup mode is incremental o

Error message

Source id key is required when cleanup mode is incremental or scoped_full.

What it means

Raised in `index()` when cleanup mode is 'incremental' or 'scoped_full' but `source_id_key` is None. These two cleanup modes delete documents whose upstream source disappeared, which is only possible if every indexed document carries a source identifier. Without `source_id_key` the indexer cannot map documents to sources, so it refuses to run rather than silently skipping cleanup.

Source

Thrown at libs/core/langchain_core/indexing/api.py:422

            (should be unreachable code).
    """
    # Behavior is deprecated, but we keep it for backwards compatibility.
    # # Warn only once per process.
    if key_encoder == "sha1":
        _warn_about_sha1()

    if cleanup not in {"incremental", "full", "scoped_full", None}:
        msg = (
            f"cleanup should be one of 'incremental', 'full', 'scoped_full' or None. "
            f"Got {cleanup}."
        )
        raise ValueError(msg)

    if (cleanup in {"incremental", "scoped_full"}) and source_id_key is None:
        msg = (
            "Source id key is required when cleanup mode is incremental or scoped_full."
        )
        raise ValueError(msg)

    destination = vector_store  # Renaming internally for clarity

    # If it's a vectorstore, let's check if it has the required methods.
    if isinstance(destination, VectorStore):
        # Check that the Vectorstore has required methods implemented
        methods = ["delete", "add_documents"]

        for method in methods:
            if not hasattr(destination, method):
                msg = (
                    f"Vectorstore {destination} does not have required method {method}"
                )
                raise ValueError(msg)

        if type(destination).delete == VectorStore.delete:
            # Checking if the VectorStore has overridden the default delete method
            # implementation which just raises a NotImplementedError

View on GitHub (pinned to e32fa9a52e)

Solutions

1. Pass a metadata key present on every document: `source_id_key="source"` (and populate it in your loader).
2. Or use a callable: `source_id_key=lambda doc: doc.metadata["url"]`.
3. If you genuinely cannot attribute sources, use `cleanup="full"` instead and accept the cost.

Example fix

# before
index(vs, docs, rm, cleanup="incremental")

# after
# loader sets metadata={"source": str(path)}
index(vs, docs, rm, cleanup="incremental", source_id_key="source")

if cleanup in {"incremental", "scoped_full"} and source_id_key is None:
    raise ValueError("source_id_key required for this cleanup mode — check config")
index(vs, docs, rm, cleanup=cleanup, source_id_key=source_id_key)

When it happens

Trigger: Calling `index(vs, docs, rm, cleanup="incremental")` with no `source_id_key`; switching cleanup from "full"/None to "incremental" in existing code without adding the key.

Common situations: Incremental sync jobs over a directory or website where files/pages get removed; upgrading a prototype (cleanup=None) to production incremental mode.

Related errors

• Source IDs are required when cleanup mode is incremental or
• source_id_key should be either None, a string or a callable.
• The delete operation to VectorStore failed.
• The delete operation to DocumentIndex failed.
• cleanup should be one of 'incremental', 'full', 'scoped_full

AI-assisted analysis of langchain-ai/langchain@e32fa9a52e (2026-08-14). Data as JSON: /api/errors/7068ee8cc6ce6bbc. Report an issue: GitHub.