{"record":{"id":"cd02592722aef338","repo":"deepset-ai/haystack","slug":"ground-truth-documents-and-retrieved-documents-mus","errorCode":null,"errorMessage":"ground_truth_documents and retrieved_documents must be provided.","messagePattern":"ground_truth_documents and retrieved_documents must be provided\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/document_ndcg.py","lineNumber":150,"sourceCode":"\n    @staticmethod\n    def validate_inputs(gt_docs: list[list[Document]], ret_docs: list[list[Document]]) -> None:\n        \"\"\"\n        Validate the input parameters.\n\n        :param gt_docs:\n            The ground_truth_documents to validate.\n        :param ret_docs:\n            The retrieved_documents to validate.\n\n        :raises ValueError:\n            If the ground_truth_documents or the retrieved_documents are an empty list.\n            If the length of ground_truth_documents and retrieved_documents differs.\n            If any list of documents in ground_truth_documents contains a mix of documents with and without a score.\n        \"\"\"\n        if len(gt_docs) == 0 or len(ret_docs) == 0:\n            msg = \"ground_truth_documents and retrieved_documents must be provided.\"\n            raise ValueError(msg)\n\n        if len(gt_docs) != len(ret_docs):\n            msg = \"The length of ground_truth_documents and retrieved_documents must be the same.\"\n            raise ValueError(msg)\n\n        for docs in gt_docs:\n            if any(doc.score is not None for doc in docs) and any(doc.score is None for doc in docs):\n                msg = \"Either none or all documents in each list of ground_truth_documents must have a score.\"\n                raise ValueError(msg)\n\n    def calculate_dcg(self, gt_docs: list[Document], ret_docs: list[Document]) -> float:\n        \"\"\"\n        Calculate the discounted cumulative gain (DCG) of the retrieved documents.\n\n        :param gt_docs:\n            The ground truth documents.\n        :param ret_docs:\n            The retrieved documents.","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/document_ndcg.py#L132-L168","documentation":"DocumentNDCGEvaluator.validate_inputs raises ValueError when either ground_truth_documents or retrieved_documents is empty, because NDCG cannot be computed without at least one question's documents. It is called at the start of run().","triggerScenarios":"Calling `run(ground_truth_documents=[], retrieved_documents=[...])`, both empty, or retrieving an empty list for the whole dataset.","commonSituations":"Empty evaluation datasets due to failed data loading; a retriever filter returning nothing so the whole retrieved list collapses; slicing bugs producing empty lists.","solutions":["Ensure both lists contain at least one entry (one per question) before calling run","Check upstream data loading / retrieval for empty results","Skip evaluation gracefully when there is no data instead of calling run"],"exampleFix":"// before\nndcg.run(ground_truth_documents=[], retrieved_documents=results)\n// after\nif gt and results:\n    ndcg.run(ground_truth_documents=gt, retrieved_documents=results)","handlingStrategy":"validation","validationCode":"if not ground_truth_documents or not retrieved_documents:\n    raise ValueError(\"evaluation inputs are empty\")","typeGuard":"def has_data(gt: list, ret: list) -> bool:\n    return len(gt) > 0 and len(ret) > 0","tryCatchPattern":"try:\n    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)\nexcept ValueError as e:\n    if \"must be provided\" in str(e):\n        result = {\"score\": 0.0, \"individual_scores\": []}  # skip empty eval\n    else:\n        raise","preventionTips":["Check that data loading actually produced rows before evaluating","Investigate retrievers that return nothing (filters, index empty)","Guard evaluation runs behind a non-empty dataset check","Fail fast at pipeline setup when inputs are empty"],"tags":["python","value-error","evaluators","empty-input"],"backgroundTag":"empty-input-list","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}