deepset-ai/haystack · error · ValueError

The matched leaf documents do not have the required meta fie

Error message

The matched leaf documents do not have the required meta field '__level'

What it means

Same validation as '__parent_id': each leaf document must carry a '__level' meta entry recording its position in the document hierarchy. If any matched leaf lacks '__level', _check_valid_documents raises this ValueError before merging.

Source

Thrown at haystack/components/retrievers/auto_merging_retriever.py:108

    def from_dict(cls, data: dict[str, Any]) -> "AutoMergingRetriever":
        """
        Deserializes the component from a dictionary.

        :param data:
            Dictionary with serialized data.
        :returns:
            An instance of the component.
        """
        return default_from_dict(cls, data)

    @staticmethod
    def _check_valid_documents(matched_leaf_documents: list[Document]) -> None:
        # check if the matched leaf documents have the required meta fields
        if not all(doc.meta.get("__parent_id") for doc in matched_leaf_documents):
            raise ValueError("The matched leaf documents do not have the required meta field '__parent_id'")

        if not all(doc.meta.get("__level") for doc in matched_leaf_documents):
            raise ValueError("The matched leaf documents do not have the required meta field '__level'")

        if not all(doc.meta.get("__block_size") for doc in matched_leaf_documents):
            raise ValueError("The matched leaf documents do not have the required meta field '__block_size'")

    @component.output_types(documents=list[Document])
    def run(self, documents: list[Document]) -> dict[str, list[Document]]:
        """
        Run the AutoMergingRetriever.

        Recursively groups documents by their parents and merges them if they meet the threshold,
        continuing up the hierarchy until no more merges are possible.

        :param documents: List of leaf documents that were matched by a retriever
        :returns:
            List of documents (could be a mix of different hierarchy levels)
        """

        AutoMergingRetriever._check_valid_documents(documents)

View on GitHub (pinned to e318778c9b)

Solutions

1. Use HierarchicalDocumentSplitter output so leaves include '__level'
2. Add '__level' to each leaf's meta before run()
3. Re-index documents missing hierarchy meta

Example fix

// before
doc.meta = {"__parent_id": pid, "__block_size": 5}
// after
doc.meta = {"__parent_id": pid, "__level": 2, "__block_size": 5}

missing = [d.id for d in docs if not d.meta.get("__level")]
if missing:
    raise ValueError(f"Leaves missing '__level': {missing}")

def has_level(doc: Document) -> bool:
    return bool(doc.meta.get("__level"))

try:
    result = retriever.run(documents=docs)
except ValueError as e:
    logger.error("Leaf docs invalid for auto-merging: %s", e)
    result = {"documents": docs}

When it happens

Trigger: run() receives leaf documents whose meta lacks '__level' — e.g. hand-built Documents, partially migrated stores, or meta keys stripped during serialization.

Common situations: Custom ingestion pipelines that copy docs but drop dunder meta keys; mixing documents from old/new indexing code paths.

Related errors

• The matched leaf documents do not have the required meta fie
• The matched leaf documents do not have the required meta fie
• top_k must be > 0, but got {top_k}
• Parameter <weight> must be in range [0,1] but is currently s
• The threshold parameter must be between 0 and 1.

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/6e89e4b88b5718d3. Report an issue: GitHub.