deepset-ai/haystack · error · ValueError

The matched leaf documents do not have the required meta fie

Error message

The matched leaf documents do not have the required meta field '__block_size'

What it means

Each leaf document must also store '__block_size' in its meta so the retriever knows how many leaves form one full block for merging. A leaf missing this key triggers this ValueError in _check_valid_documents.

Source

Thrown at haystack/components/retrievers/auto_merging_retriever.py:111

        :param data:
            Dictionary with serialized data.
        :returns:
            An instance of the component.
        """
        return default_from_dict(cls, data)

    @staticmethod
    def _check_valid_documents(matched_leaf_documents: list[Document]) -> None:
        # check if the matched leaf documents have the required meta fields
        if not all(doc.meta.get("__parent_id") for doc in matched_leaf_documents):
            raise ValueError("The matched leaf documents do not have the required meta field '__parent_id'")

        if not all(doc.meta.get("__level") for doc in matched_leaf_documents):
            raise ValueError("The matched leaf documents do not have the required meta field '__level'")

        if not all(doc.meta.get("__block_size") for doc in matched_leaf_documents):
            raise ValueError("The matched leaf documents do not have the required meta field '__block_size'")

    @component.output_types(documents=list[Document])
    def run(self, documents: list[Document]) -> dict[str, list[Document]]:
        """
        Run the AutoMergingRetriever.

        Recursively groups documents by their parents and merges them if they meet the threshold,
        continuing up the hierarchy until no more merges are possible.

        :param documents: List of leaf documents that were matched by a retriever
        :returns:
            List of documents (could be a mix of different hierarchy levels)
        """

        AutoMergingRetriever._check_valid_documents(documents)

        def _get_parent_doc(parent_id: str) -> Document:
            parent_docs = self.document_store.filter_documents({"field": "id", "operator": "==", "value": parent_id})

View on GitHub (pinned to e318778c9b)

Solutions

1. Index with HierarchicalDocumentSplitter so '__block_size' is present
2. Set '__block_size' explicitly on each leaf's meta before run()
3. Rebuild the document store with the standard hierarchical indexing pipeline

Example fix

// before
doc.meta = {"__parent_id": pid, "__level": 1}
// after
doc.meta = {"__parent_id": pid, "__level": 1, "__block_size": 5}

missing = [d.id for d in docs if not d.meta.get("__block_size")]
if missing:
    raise ValueError(f"Leaves missing '__block_size': {missing}")

def has_block_size(doc: Document) -> bool:
    return bool(doc.meta.get("__block_size"))

try:
    result = retriever.run(documents=docs)
except ValueError as e:
    logger.error("Leaf docs invalid for auto-merging: %s", e)
    result = {"documents": docs}

When it happens

Trigger: run() with leaf documents whose meta lacks '__block_size' — typically documents not produced by the hierarchical splitting/indexing flow.

Common situations: Manually constructed Document objects, stores built by custom scripts, or post-processing that rebuilds meta and omits internal keys.

Related errors

• The matched leaf documents do not have the required meta fie
• The matched leaf documents do not have the required meta fie
• top_k must be > 0, but got {top_k}
• Parameter <weight> must be in range [0,1] but is currently s
• The threshold parameter must be between 0 and 1.

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/445beabf5ba192a3. Report an issue: GitHub.