deepset-ai/haystack · error

The 'other' parameter must have 'run_name', 'inputs', and 'r

Error message

The 'other' parameter must have 'run_name', 'inputs', and 'results' attributes.

What it means

After the isinstance check, comparative_detailed_report verifies the other object exposes run_name, inputs, and results attributes. A subclass or mock missing any of these raises this ValueError, since the comparison needs the other run's inputs and per-metric results.

Source

Thrown at haystack/evaluation/eval_run_result.py:192

        :param other: Results of another evaluation run to compare with.
        :param keep_columns: List of common column names to keep from the inputs of the evaluation runs to compare.
        :param output_format: The output format for the report, "json", "csv", or "df", default to "json".
        :param csv_file: Filepath to save CSV output if `output_format` is "csv", must be provided.

        :returns:
            JSON or DataFrame with a comparison of the detailed scores, in case the output is set to a CSV file,
             a message confirming the successful write or an error message.
        :raises TypeError: If `other` is not an EvaluationRunResult instance, or if the detailed reports are not
            dictionaries.
        :raises ValueError: If the `other` parameter is missing required attributes.
        """

        if not isinstance(other, EvaluationRunResult):
            raise TypeError("Comparative scores can only be computed between EvaluationRunResults.")

        if not hasattr(other, "run_name") or not hasattr(other, "inputs") or not hasattr(other, "results"):
            raise ValueError("The 'other' parameter must have 'run_name', 'inputs', and 'results' attributes.")

        if self.run_name == other.run_name:
            logger.warning(
                "The run names of the two evaluation results are the same ('{run_name}')", run_name=self.run_name
            )

        if self.inputs.keys() != other.inputs.keys():
            logger.warning(
                "The input columns differ between the results; using the input columns of '{run_name}'",
                run_name=self.run_name,
            )

        # got both detailed reports
        detailed_a = self.detailed_report(output_format="json")
        detailed_b = other.detailed_report(output_format="json")

        # ensure both detailed reports are in dictionaries format
        if not isinstance(detailed_a, dict) or not isinstance(detailed_b, dict):

View on GitHub (pinned to e318778c9b)

Solutions

1. Ensure the other object defines all three attributes: run_name, inputs, results.
2. Replace partial mocks with real EvaluationRunResult instances in tests.
3. Update old serialized results to the current attribute schema.

Example fix

// before
class FakeRun:
    run_name = 'b'
run_a.comparative_detailed_report(other=FakeRun())
// after
class FakeRun:
    run_name = 'b'
    inputs = {...}
    results = {...}
run_a.comparative_detailed_report(other=FakeRun())

required = ('run_name', 'inputs', 'results')
if not all(hasattr(other, a) for a in required):
    raise ValueError(f'other must define {required}')

def is_complete_run(obj: object) -> bool:
    return all(hasattr(obj, a) for a in ('run_name', 'inputs', 'results'))

try:
    comparison = run_a.comparative_detailed_report(other=other)
except ValueError as e:
    if "must have 'run_name'" in str(e):
        raise TypeError(f'{type(other).__name__} is not a usable evaluation run') from e
    raise

When it happens

Trigger: Passing an EvaluationRunResult subclass or stub/mock that does not define run_name, inputs, or results — typically a hand-rolled fake in tests.

Common situations: Test doubles that only implement part of the API; a subclass that renamed attributes; pickled objects from an older haystack version with a different attribute layout.

Understand the failure class

Background: "Missing required field" and "field is required" errors: why libraries reject payloads that omit mandatory fields — this error's family across 20 libraries.

Related errors

• Aggregate score missing for {metric}.
• Individual scores missing for {metric}.
• Length of individual scores for '{metric}' should be the sam
• All lists in the JSON must have the same length
• Comparative scores can only be computed between EvaluationRu

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/d069b11da4407d03. Report an issue: GitHub.