{"record":{"id":"bd2f557b850b63b2","repo":"deepset-ai/haystack","slug":"the-length-of-ground-truth-documents-and-retrieved-bd2f55","errorCode":null,"errorMessage":"The length of ground_truth_documents and retrieved_documents must be the same.","messagePattern":"The length of ground_truth_documents and retrieved_documents must be the same\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/document_ndcg.py","lineNumber":154,"sourceCode":"        Validate the input parameters.\n\n        :param gt_docs:\n            The ground_truth_documents to validate.\n        :param ret_docs:\n            The retrieved_documents to validate.\n\n        :raises ValueError:\n            If the ground_truth_documents or the retrieved_documents are an empty list.\n            If the length of ground_truth_documents and retrieved_documents differs.\n            If any list of documents in ground_truth_documents contains a mix of documents with and without a score.\n        \"\"\"\n        if len(gt_docs) == 0 or len(ret_docs) == 0:\n            msg = \"ground_truth_documents and retrieved_documents must be provided.\"\n            raise ValueError(msg)\n\n        if len(gt_docs) != len(ret_docs):\n            msg = \"The length of ground_truth_documents and retrieved_documents must be the same.\"\n            raise ValueError(msg)\n\n        for docs in gt_docs:\n            if any(doc.score is not None for doc in docs) and any(doc.score is None for doc in docs):\n                msg = \"Either none or all documents in each list of ground_truth_documents must have a score.\"\n                raise ValueError(msg)\n\n    def calculate_dcg(self, gt_docs: list[Document], ret_docs: list[Document]) -> float:\n        \"\"\"\n        Calculate the discounted cumulative gain (DCG) of the retrieved documents.\n\n        :param gt_docs:\n            The ground truth documents.\n        :param ret_docs:\n            The retrieved documents.\n        :returns:\n            The discounted cumulative gain (DCG) of the retrieved\n            documents based on the ground truth documents.\n        \"\"\"","sourceCodeStart":136,"sourceCodeEnd":172,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/document_ndcg.py#L136-L172","documentation":"DocumentNDCGEvaluator.validate_inputs raises ValueError when ground_truth_documents and retrieved_documents have different lengths, since NDCG is computed per aligned question pair. This is the same alignment contract as MAP/MRR evaluators but enforced inside validate_inputs alongside other checks.","triggerScenarios":"Calling run() with, e.g., 5 ground-truth question lists and 6 retrieved question lists; any off-by-one from partial batch evaluation.","commonSituations":"Appending retrieved results in a loop that runs more/fewer iterations than the GT set; datasets where some questions failed and were dropped from one side only.","solutions":["Build both lists from the same iteration over question ids","Pad with [] or drop from both sides together so lengths match","Assert `len(gt) == len(ret)` before run()"],"exampleFix":"// before\nndcg.run(ground_truth_documents=gt[:5], retrieved_documents=retrieved)  # 5 vs 6\n// after\nn = min(len(gt), len(retrieved))\nndcg.run(ground_truth_documents=gt[:n], retrieved_documents=retrieved[:n])","handlingStrategy":"validation","validationCode":"assert len(ground_truth_documents) == len(retrieved_documents), (len(ground_truth_documents), len(retrieved_documents))","typeGuard":"def aligned(a: list, b: list) -> bool:\n    return len(a) == len(b)","tryCatchPattern":"try:\n    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)\nexcept ValueError as e:\n    if \"must be the same\" in str(e):\n        n = min(len(gt), len(ret)); result = evaluator.run(ground_truth_documents=gt[:n], retrieved_documents=ret[:n])\n    else:\n        raise","preventionTips":["Build both lists in the same loop over question ids","Never drop failed questions from only one side","Keep per-question results keyed by id so you can re-align","Sanity-check lengths before batch evaluation runs"],"tags":["python","value-error","evaluators","dataset-alignment"],"backgroundTag":"input-list-length-mismatch","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}