{"record":{"id":"20bb91f1f4aee6c9","repo":"deepset-ai/haystack","slug":"the-length-of-ground-truth-documents-and-retrieved-20bb91","errorCode":null,"errorMessage":"The length of ground_truth_documents and retrieved_documents must be the same.","messagePattern":"The length of ground_truth_documents and retrieved_documents must be the same\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/document_mrr.py","lineNumber":111,"sourceCode":"    ) -> dict[str, Any]:\n        \"\"\"\n        Run the DocumentMRREvaluator on the given inputs.\n\n        `ground_truth_documents` and `retrieved_documents` must have the same length.\n\n        :param ground_truth_documents:\n            A list of expected documents for each question.\n        :param retrieved_documents:\n            A list of retrieved documents for each question.\n        :returns:\n            A dictionary with the following outputs:\n            - `score` - The average of calculated scores.\n            - `individual_scores` - A list of numbers from 0.0 to 1.0 that represents how high the first retrieved\n                document is ranked.\n        \"\"\"\n        if len(ground_truth_documents) != len(retrieved_documents):\n            msg = \"The length of ground_truth_documents and retrieved_documents must be the same.\"\n            raise ValueError(msg)\n\n        individual_scores = []\n\n        for ground_truth, retrieved in zip(ground_truth_documents, retrieved_documents, strict=True):\n            reciprocal_rank = 0.0\n\n            ground_truth_values = [val for doc in ground_truth if (val := self._get_comparison_value(doc)) is not None]\n            for rank, retrieved_document in enumerate(retrieved):\n                retrieved_value = self._get_comparison_value(retrieved_document)\n                if retrieved_value is None:\n                    continue\n                if retrieved_value in ground_truth_values:\n                    reciprocal_rank = 1 / (rank + 1)\n                    break\n            individual_scores.append(reciprocal_rank)\n\n        score = sum(individual_scores) / len(ground_truth_documents)\n","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/document_mrr.py#L93-L129","documentation":"DocumentMRREvaluator.run requires ground_truth_documents and retrieved_documents to be equal-length parallel lists of per-question document lists, computing reciprocal rank per question. It raises ValueError on length mismatch because strict per-question pairing is required.","triggerScenarios":"Calling run() with different numbers of ground-truth question entries vs retrieval result entries, e.g. one list has 20 items and the other 19 after a failed query.","commonSituations":"Evaluation harnesses that skip queries with no retrieval output; loading GT and predictions from independent JSONL files with diverging row counts.","solutions":["Keep both lists one-entry-per-question, using [] for missing document sets","Filter/pad both lists together by question id","Add a pre-run length assertion in your evaluation script"],"exampleFix":"// before\nmrr.run(ground_truth_documents=gt, retrieved_documents=retrieved[1:])  # dropped first row\n// after\nmrr.run(ground_truth_documents=gt, retrieved_documents=retrieved)  # both length N","handlingStrategy":"validation","validationCode":"assert len(ground_truth_documents) == len(retrieved_documents), (len(ground_truth_documents), len(retrieved_documents))","typeGuard":"def aligned(a: list, b: list) -> bool:\n    return len(a) == len(b)","tryCatchPattern":"try:\n    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)\nexcept ValueError as e:\n    if \"must be the same\" in str(e):\n        n = min(len(gt), len(ret)); result = evaluator.run(ground_truth_documents=gt[:n], retrieved_documents=ret[:n])\n    else:\n        raise","preventionTips":["Keep GT and retrieved lists indexed by the same question ids","Use [] for questions without documents rather than omitting rows","Slice both lists together when subsetting","Validate lengths before running large evaluation sweeps"],"tags":["python","value-error","evaluators","dataset-alignment"],"backgroundTag":"input-list-length-mismatch","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}