{"record":{"id":"de2f353bb7e8a3fb","repo":"deepset-ai/haystack","slug":"predicted-answers-must-not-contain-none-values","errorCode":null,"errorMessage":"Predicted answers must not contain None values.","messagePattern":"Predicted answers must not contain None values\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/sas_evaluator.py","lineNumber":148,"sourceCode":"        SASEvaluator component run method.\n\n        Run the SASEvaluator to compute the Semantic Answer Similarity (SAS) between a list of predicted answers\n        and a list of ground truth answers. Both must be list of strings of same length.\n\n        :param ground_truth_answers:\n            A list of expected answers for each question.\n        :param predicted_answers:\n            A list of generated answers for each question.\n        :returns:\n            A dictionary with the following outputs:\n                - `score`: Mean SAS score over all the predictions/ground-truth pairs.\n                - `individual_scores`: A list of similarity scores for each prediction/ground-truth pair.\n        \"\"\"\n        if len(ground_truth_answers) != len(predicted_answers):\n            raise ValueError(\"The number of predictions and labels must be the same.\")\n\n        if any(answer is None for answer in predicted_answers):\n            raise ValueError(\"Predicted answers must not contain None values.\")\n\n        if len(predicted_answers) == 0:\n            return {\"score\": 0.0, \"individual_scores\": [0.0]}\n\n        if not self._similarity_model:\n            self.warm_up()\n\n        if isinstance(self._similarity_model, CrossEncoder):\n            # For Cross Encoders we create a list of pairs of predictions and labels\n            sentence_pairs = list(zip(predicted_answers, ground_truth_answers, strict=True))\n            similarity_scores = self._similarity_model.predict(\n                sentence_pairs, batch_size=self._batch_size, convert_to_numpy=True\n            )\n\n            # All Cross Encoders do not return a set of logits scores that are normalized\n            # We normalize scores if they are larger than 1\n            if (similarity_scores > 1).any():\n                similarity_scores = expit(similarity_scores)","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/sas_evaluator.py#L130-L166","documentation":"SASEvaluator cannot score None predicted answers, so it raises ValueError before invoking the similarity model to avoid None crashing the embedding step.","triggerScenarios":"Calling run() where predicted_answers contains at least one None element, e.g. an upstream extraction/generation component that returned None for a failed answer.","commonSituations":"Pipeline components that emit None on failure (e.g. extractors returning None content) feeding directly into the evaluator; JSON test data with null answers loaded from a file.","solutions":["Filter or replace None values before evaluation: predicted_answers = [a if a is not None else \"\" for a in answers]","Fix the upstream component (e.g. use raise_on_failure=True on extractors/generators) so it never emits None","Validate with all(a is not None for a in predicted_answers) before calling run()"],"exampleFix":"// before\nevaluator.run(ground_truth_answers=gt, predicted_answers=preds)\n// after\ncleaned = [p if p is not None else \"\" for p in preds]\nevaluator.run(ground_truth_answers=gt, predicted_answers=cleaned)","handlingStrategy":"validation","validationCode":"if any(a is None for a in predicted_answers):\n    predicted_answers = [a if a is not None else \"\" for a in predicted_answers]","typeGuard":"def no_none_answers(answers: list) -> bool:\n    return all(a is not None for a in answers)","tryCatchPattern":"try:\n    result = sas_evaluator.run(ground_truth_answers=gt, predicted_answers=preds)\nexcept ValueError:\n    preds = [p or \"\" for p in preds]\n    result = sas_evaluator.run(ground_truth_answers=gt, predicted_answers=preds)","preventionTips":["Sanitize upstream outputs (replace None with \"\") before evaluation","Enable raise_on_failure on upstream generator/extractor components to fail earlier with clearer errors","Add a preprocessing step that validates evaluator inputs"],"tags":["python","validation","null-value"],"backgroundTag":"null-value-not-allowed","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}