{"record":{"id":"5e93c8b11609c888","repo":"deepset-ai/haystack","slug":"aggregate-score-missing-for-metric","errorCode":null,"errorMessage":"Aggregate score missing for {metric}.","messagePattern":"Aggregate score missing for (.+?)\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/evaluation/eval_run_result.py","lineNumber":53,"sourceCode":"            Dictionary containing the results of the evaluators used in the evaluation pipeline. Each key is the name\n            of the metric and its value is dictionary with the following keys:\n                - 'score': The aggregated score for the metric.\n                - 'individual_scores': A list of scores for each input sample.\n        \"\"\"\n        self.run_name = run_name\n        self.inputs = deepcopy(inputs)\n        self.results = deepcopy(results)\n\n        if len(inputs) == 0:\n            raise ValueError(\"No inputs provided.\")\n        if len({len(lst) for lst in inputs.values()}) != 1:\n            raise ValueError(\"Lengths of the inputs should be the same.\")\n\n        expected_len = len(next(iter(inputs.values())))\n\n        for metric, outputs in results.items():\n            if \"score\" not in outputs:\n                raise ValueError(f\"Aggregate score missing for {metric}.\")\n            if \"individual_scores\" not in outputs:\n                raise ValueError(f\"Individual scores missing for {metric}.\")\n\n            if len(outputs[\"individual_scores\"]) != expected_len:\n                raise ValueError(\n                    f\"Length of individual scores for '{metric}' should be the same as the inputs. \"\n                    f\"Got {len(outputs['individual_scores'])} but expected {expected_len}.\"\n                )\n\n    @staticmethod\n    def _write_to_csv(csv_file: str, data: dict[str, list[Any]]) -> str:\n        \"\"\"\n        Write data to a CSV file.\n\n        :param csv_file: Path to the CSV file to write\n        :param data: Dictionary containing the data to write\n        :return: Status message indicating success or failure\n        \"\"\"","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/evaluation/eval_run_result.py#L35-L71","documentation":"EvaluationRunResult validates that every metric entry in the results dict contains an aggregate 'score' key. If a metric's outputs dict lacks 'score', the constructor raises this ValueError immediately, refusing to build the result object. This guarantees all downstream reports (aggregated/detailed/comparative) can rely on an aggregate score existing per metric.","triggerScenarios":"Passing a results dict to EvaluationRunResult(...) where some metric maps to a dict without a 'score' key, e.g. {'metric_name': {'individual_scores': [...]}} missing the aggregate score produced by the metric callable.","commonSituations":"Hand-constructing results from a custom evaluation loop instead of haystack's evaluate() pipeline; a custom metric returning a partial result dict; renaming or dropping the 'score' key when post-processing saved evaluation output.","solutions":["Ensure each metric's results dict contains an aggregate 'score' key, e.g. compute sum/mean of individual_scores.","Use haystack's evaluate()/AggregateOutput format so results are built correctly.","Validate results before constructing: check all(m.get('score') is not None for m in results.values())."],"exampleFix":"// before\nresults = {'exact_match': {'individual_scores': [1, 0]}}\nrun = EvaluationRunResult('run', inputs, results)\n// after\nresults = {'exact_match': {'score': 0.5, 'individual_scores': [1, 0]}}\nrun = EvaluationRunResult('run', inputs, results)","handlingStrategy":"validation","validationCode":"for metric, outputs in results.items():\n    if 'score' not in outputs:\n        raise ValueError(f\"Metric '{metric}' has no aggregate 'score'\")","typeGuard":"def has_score(metric_output: dict) -> bool:\n    return isinstance(metric_output, dict) and 'score' in metric_output","tryCatchPattern":"try:\n    run = EvaluationRunResult(run_name, inputs, results)\nexcept ValueError as e:\n    if 'Aggregate score missing' in str(e):\n        results = {m: {'score': 0.0, **o} for m, o in results.items()}\n        run = EvaluationRunResult(run_name, inputs, results)\n    else:\n        raise","preventionTips":["Always build results via haystack's evaluate() pipeline","Include both 'score' and 'individual_scores' keys in custom metric outputs","Unit-test custom metric result shapes before wiring them in"],"tags":["python","validation","evaluation"],"backgroundTag":"missing-required-field","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}