ground_truth_documents and retrieved_documents must be provi

Error message

ground_truth_documents and retrieved_documents must be provided.

What it means

DocumentNDCGEvaluator.validate_inputs raises ValueError when either ground_truth_documents or retrieved_documents is empty, because NDCG cannot be computed without at least one question's documents. It is called at the start of run().

Source

Thrown at haystack/components/evaluators/document_ndcg.py:150

    @staticmethod
    def validate_inputs(gt_docs: list[list[Document]], ret_docs: list[list[Document]]) -> None:
        """
        Validate the input parameters.

        :param gt_docs:
            The ground_truth_documents to validate.
        :param ret_docs:
            The retrieved_documents to validate.

        :raises ValueError:
            If the ground_truth_documents or the retrieved_documents are an empty list.
            If the length of ground_truth_documents and retrieved_documents differs.
            If any list of documents in ground_truth_documents contains a mix of documents with and without a score.
        """
        if len(gt_docs) == 0 or len(ret_docs) == 0:
            msg = "ground_truth_documents and retrieved_documents must be provided."
            raise ValueError(msg)

        if len(gt_docs) != len(ret_docs):
            msg = "The length of ground_truth_documents and retrieved_documents must be the same."
            raise ValueError(msg)

        for docs in gt_docs:
            if any(doc.score is not None for doc in docs) and any(doc.score is None for doc in docs):
                msg = "Either none or all documents in each list of ground_truth_documents must have a score."
                raise ValueError(msg)

    def calculate_dcg(self, gt_docs: list[Document], ret_docs: list[Document]) -> float:
        """
        Calculate the discounted cumulative gain (DCG) of the retrieved documents.

        :param gt_docs:
            The ground truth documents.
        :param ret_docs:
            The retrieved documents.

View on GitHub (pinned to e318778c9b)

Solutions

Ensure both lists contain at least one entry (one per question) before calling run
Check upstream data loading / retrieval for empty results
Skip evaluation gracefully when there is no data instead of calling run

Example fix

// before
ndcg.run(ground_truth_documents=[], retrieved_documents=results)
// after
if gt and results:
    ndcg.run(ground_truth_documents=gt, retrieved_documents=results)

Defensive patterns

Strategy: validation

Validate before calling

if not ground_truth_documents or not retrieved_documents:
    raise ValueError("evaluation inputs are empty")

Type guard

def has_data(gt: list, ret: list) -> bool:
    return len(gt) > 0 and len(ret) > 0

Try / catch

try:
    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)
except ValueError as e:
    if "must be provided" in str(e):
        result = {"score": 0.0, "individual_scores": []}  # skip empty eval
    else:
        raise

Prevention

Check that data loading actually produced rows before evaluating
Investigate retrievers that return nothing (filters, index empty)
Guard evaluation runs behind a non-empty dataset check
Fail fast at pipeline setup when inputs are empty

When it happens

Trigger: Calling `run(ground_truth_documents=[], retrieved_documents=[...])`, both empty, or retrieving an empty list for the whole dataset.

Common situations: Empty evaluation datasets due to failed data loading; a retriever filter returning nothing so the whole retrieved list collapses; slicing bugs producing empty lists.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/cd02592722aef338. Report an issue: GitHub.