ground_truth_documents and retrieved_documents must be…

Error message

ground_truth_documents and retrieved_documents must be provided.

What it means

DocumentNDCGEvaluator.validate_inputs raises ValueError when either ground_truth_documents or retrieved_documents is empty, because NDCG cannot be computed without at least one question's documents. It is called at the start of run().

Solutions

Ensure both lists contain at least one entry (one per question) before calling run
Check upstream data loading / retrieval for empty results
Skip evaluation gracefully when there is no data instead of calling run

Example fix

// before
ndcg.run(ground_truth_documents=[], retrieved_documents=results)
// after
if gt and results:
    ndcg.run(ground_truth_documents=gt, retrieved_documents=results)

Defensive patterns

Strategy: validation

Validate before calling

if not ground_truth_documents or not retrieved_documents:
    raise ValueError("evaluation inputs are empty")

Type guard

def has_data(gt: list, ret: list) -> bool:
    return len(gt) > 0 and len(ret) > 0

Try / catch

try:
    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)
except ValueError as e:
    if "must be provided" in str(e):
        result = {"score": 0.0, "individual_scores": []}  # skip empty eval
    else:
        raise

Prevention

Check that data loading actually produced rows before evaluating
Investigate retrievers that return nothing (filters, index empty)
Guard evaluation runs behind a non-empty dataset check
Fail fast at pipeline setup when inputs are empty

When it happens

Trigger: Calling `run(ground_truth_documents=[], retrieved_documents=[...])`, both empty, or retrieving an empty list for the whole dataset.

Common situations: Empty evaluation datasets due to failed data loading; a retriever filter returning nothing so the whole retrieved list collapses; slicing bugs producing empty lists.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/cd02592722aef338. Report an issue: GitHub.

Appendix: source

Thrown at haystack/components/evaluators/document_ndcg.py:150

    @staticmethod
    def validate_inputs(gt_docs: list[list[Document]], ret_docs: list[list[Document]]) -> None:
        """
        Validate the input parameters.

        :param gt_docs:
            The ground_truth_documents to validate.
        :param ret_docs:
            The retrieved_documents to validate.

        :raises ValueError:
            If the ground_truth_documents or the retrieved_documents are an empty list.
            If the length of ground_truth_documents and retrieved_documents differs.
            If any list of documents in ground_truth_documents contains a mix of documents with and without a score.
        """
        if len(gt_docs) == 0 or len(ret_docs) == 0:
            msg = "ground_truth_documents and retrieved_documents must be provided."
            raise ValueError(msg)

        if len(gt_docs) != len(ret_docs):
            msg = "The length of ground_truth_documents and retrieved_documents must be the same."
            raise ValueError(msg)

        for docs in gt_docs:
            if any(doc.score is not None for doc in docs) and any(doc.score is None for doc in docs):
                msg = "Either none or all documents in each list of ground_truth_documents must have a score."
                raise ValueError(msg)

    def calculate_dcg(self, gt_docs: list[Document], ret_docs: list[Document]) -> float:
        """
        Calculate the discounted cumulative gain (DCG) of the retrieved documents.

        :param gt_docs:
            The ground truth documents.
        :param ret_docs:
            The retrieved documents.

View on GitHub (pinned to e318778c9b)