{"record":{"id":"1e2f0c2f7c10bc45","repo":"deepset-ai/haystack","slug":"the-length-of-ground-truth-documents-and-retrieved","errorCode":null,"errorMessage":"The length of ground_truth_documents and retrieved_documents must be the same.","messagePattern":"The length of ground_truth_documents and retrieved_documents must be the same\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/document_map.py","lineNumber":113,"sourceCode":"    ) -> dict[str, Any]:\n        \"\"\"\n        Run the DocumentMAPEvaluator on the given inputs.\n\n        All lists must have the same length.\n\n        :param ground_truth_documents:\n            A list of expected documents for each question.\n        :param retrieved_documents:\n            A list of retrieved documents for each question.\n        :returns:\n            A dictionary with the following outputs:\n            - `score` - The average of calculated scores.\n            - `individual_scores` - A list of numbers from 0.0 to 1.0 that represents how high retrieved documents\n                are ranked.\n        \"\"\"\n        if len(ground_truth_documents) != len(retrieved_documents):\n            msg = \"The length of ground_truth_documents and retrieved_documents must be the same.\"\n            raise ValueError(msg)\n\n        individual_scores = []\n\n        for ground_truth, retrieved in zip(ground_truth_documents, retrieved_documents, strict=True):\n            average_precision = 0.0\n            average_precision_numerator = 0.0\n            retrieved_relevant_documents = 0\n\n            # A list keeps the deduplication working for unhashable comparison values, for example when\n            # document_comparison_field points to a meta key holding a list.\n            uncredited_ground_truth_values: list[Any] = []\n            for doc in ground_truth:\n                value = self._get_comparison_value(doc)\n                if value is not None and value not in uncredited_ground_truth_values:\n                    uncredited_ground_truth_values.append(value)\n\n            total_relevant_documents = len(uncredited_ground_truth_values)\n            for rank, retrieved_document in enumerate(retrieved):","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/document_map.py#L95-L131","documentation":"DocumentMAPEvaluator.run requires ground_truth_documents and retrieved_documents to be parallel lists of per-question document lists. It raises ValueError when their lengths differ, because average precision must be computed per question pair and a strict zip would silently drop or misalign questions.","triggerScenarios":"Calling run() where `len(ground_truth_documents) != len(retrieved_documents)`, e.g. 10 ground-truth entries but only 9 retrieval results after a failed retrieval.","commonSituations":"Building evaluation sets from separate files where a query was skipped; retriever returning results only for successful queries; concatenating batch results unevenly.","solutions":["Make both lists contain one entry per question, using empty lists [] for questions with no documents","Filter both lists together (same indices) before calling run","Validate `len(a) == len(b)` in your eval harness before invoking the evaluator"],"exampleFix":"// before\nevaluator.run(ground_truth_documents=gt[:9], retrieved_documents=retrieved)  # 9 vs 10\n// after\nevaluator.run(ground_truth_documents=gt, retrieved_documents=retrieved)  # keep aligned, use [] where empty","handlingStrategy":"validation","validationCode":"if len(ground_truth_documents) != len(retrieved_documents):\n    raise ValueError(f\"gt={len(ground_truth_documents)} retrieved={len(retrieved_documents)}\")","typeGuard":"def aligned(a: list, b: list) -> bool:\n    return len(a) == len(b)","tryCatchPattern":"try:\n    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)\nexcept ValueError as e:\n    if \"must be the same\" in str(e):\n        n = min(len(gt), len(ret)); result = evaluator.run(ground_truth_documents=gt[:n], retrieved_documents=ret[:n])\n    else:\n        raise","preventionTips":["Emit one entry per question even when documents are missing (use [])","Build both lists from a single loop over the dataset","Keep retrieval failures as empty sub-lists, not missing rows","Assert lengths in the evaluation harness"],"tags":["python","value-error","evaluators","dataset-alignment"],"backgroundTag":"input-list-length-mismatch","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}