{"record":{"id":"bf4028fbfd0e713f","repo":"deepset-ai/haystack","slug":"the-length-of-ground-truth-answers-and-predicted-a","errorCode":null,"errorMessage":"The length of ground_truth_answers and predicted_answers must be the same.","messagePattern":"The length of ground_truth_answers and predicted_answers must be the same\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/answer_exact_match.py","lineNumber":57,"sourceCode":"    def run(self, ground_truth_answers: list[str], predicted_answers: list[str]) -> dict[str, Any]:\n        \"\"\"\n        Run the AnswerExactMatchEvaluator on the given inputs.\n\n        The `ground_truth_answers` and `retrieved_answers` must have the same length.\n\n        :param ground_truth_answers:\n            A list of expected answers.\n        :param predicted_answers:\n            A list of predicted answers.\n        :returns:\n            A dictionary with the following outputs:\n            - `individual_scores` - A list of 0s and 1s, where 1 means that the predicted answer matched one of the\n                ground truth.\n            - `score` - A number from 0.0 to 1.0 that represents the proportion of questions where any predicted\n                         answer matched one of the ground truth answers.\n        \"\"\"\n        if not len(ground_truth_answers) == len(predicted_answers):\n            raise ValueError(\"The length of ground_truth_answers and predicted_answers must be the same.\")\n\n        matches = []\n        for truth, extracted in zip(ground_truth_answers, predicted_answers, strict=True):\n            if truth == extracted:\n                matches.append(1)\n            else:\n                matches.append(0)\n\n        # The proportion of questions where any predicted answer matched one of the ground truth answers\n        average = sum(matches) / len(predicted_answers)\n\n        return {\"individual_scores\": matches, \"score\": average}\n","sourceCodeStart":39,"sourceCodeEnd":70,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/answer_exact_match.py#L39-L70","documentation":"SASEvaluator.run (answer exact match) zips ground_truth_answers with predicted_answers using strict=True after asserting equal lengths. It raises ValueError when the two lists differ in length, since each predicted answer needs its ground-truth counterpart to compute the per-question match scores.","triggerScenarios":"Calling `evaluator.run(ground_truth_answers=[\"a\", \"b\"], predicted_answers=[\"x\"])` — any length mismatch between the two lists, including empty vs non-empty.","commonSituations":"Evaluation datasets where some questions have no model answer (LLM refused/failed) so the predictions list is shorter; loading answers from separate files or pipeline runs that dropped rows.","solutions":["Ensure both lists are built from the same dataset rows with the same length","Align the lists by question id, padding or filtering consistently on both sides","Assert lengths match before calling run: `assert len(gt) == len(pred)`"],"exampleFix":"// before\nevaluator.run(ground_truth_answers=[\"Paris\", \"Berlin\"], predicted_answers=[\"Paris\"])\n// after\n# keep answers from failed runs, e.g. as empty string\nevaluator.run(ground_truth_answers=[\"Paris\", \"Berlin\"], predicted_answers=[\"Paris\", \"\"])","handlingStrategy":"validation","validationCode":"assert len(ground_truth_answers) == len(predicted_answers), (len(ground_truth_answers), len(predicted_answers))","typeGuard":"def aligned(gt: list, pred: list) -> bool:\n    return len(gt) == len(pred)","tryCatchPattern":"try:\n    result = evaluator.run(ground_truth_answers=gt, predicted_answers=pred)\nexcept ValueError as e:\n    if \"must be the same\" in str(e):\n        n = min(len(gt), len(pred)); result = evaluator.run(ground_truth_answers=gt[:n], predicted_answers=pred[:n])\n    else:\n        raise","preventionTips":["Iterate over the same dataset rows to build both lists","Keep failed predictions as empty strings instead of dropping rows","Align by question id before evaluation","Log list lengths before every evaluator.run call"],"tags":["python","value-error","evaluators","dataset-alignment"],"backgroundTag":"input-list-length-mismatch","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}