deepset-ai/haystack · error

Lengths of the inputs should be the same.

Error message

Lengths of the inputs should be the same.

What it means

EvaluationRunResult.__init__ raises ValueError when the input lists in the inputs dict have different lengths. Inputs and metric outputs are aligned positionally per evaluated item, so ragged input columns make the alignment impossible.

Source

Thrown at haystack/evaluation/eval_run_result.py:47

        :param inputs:
            Dictionary containing the inputs used for the run. Each key is the name of the input and its value is a list
            of input values. The length of the lists should be the same.

        :param results:
            Dictionary containing the results of the evaluators used in the evaluation pipeline. Each key is the name
            of the metric and its value is dictionary with the following keys:
                - 'score': The aggregated score for the metric.
                - 'individual_scores': A list of scores for each input sample.
        """
        self.run_name = run_name
        self.inputs = deepcopy(inputs)
        self.results = deepcopy(results)

        if len(inputs) == 0:
            raise ValueError("No inputs provided.")
        if len({len(lst) for lst in inputs.values()}) != 1:
            raise ValueError("Lengths of the inputs should be the same.")

        expected_len = len(next(iter(inputs.values())))

        for metric, outputs in results.items():
            if "score" not in outputs:
                raise ValueError(f"Aggregate score missing for {metric}.")
            if "individual_scores" not in outputs:
                raise ValueError(f"Individual scores missing for {metric}.")

            if len(outputs["individual_scores"]) != expected_len:
                raise ValueError(
                    f"Length of individual scores for '{metric}' should be the same as the inputs. "
                    f"Got {len(outputs['individual_scores'])} but expected {expected_len}."
                )

    @staticmethod
    def _write_to_csv(csv_file: str, data: dict[str, list[Any]]) -> str:
        """

View on GitHub (pinned to e318778c9b)

Solutions

1. Pad or fix the shorter input lists so every list has the same length before construction
2. Fix the data collection loop so every input key gets exactly one entry per evaluated item
3. Verify lengths programmatically: assert len({len(v) for v in inputs.values()}) == 1

Example fix

// before
inputs = {"queries": [q1, q2, q3], "contexts": [c1, c2]}  # ValueError
// after
assert len(inputs["queries"]) == len(inputs["contexts"])
inputs = {"queries": [q1, q2, q3], "contexts": [c1, c2, c3]}
run = EvaluationRunResult("run1", inputs=inputs, results=results)

lengths = {len(v) for v in inputs.values()}
if len(lengths) > 1:
    raise RuntimeError(f"Input columns have different lengths: { {k: len(v) for k, v in inputs.items()} }")

try:
    run = EvaluationRunResult(run_name, inputs=inputs, results=results)
except ValueError as e:
    if "Lengths of the inputs should be the same" in str(e):
        target = max(len(v) for v in inputs.values())
        inputs = {k: v + [None] * (target - len(v)) for k, v in inputs.items()}  # or fix data
        run = EvaluationRunResult(run_name, inputs=inputs, results=results)
    else:
        raise

When it happens

Trigger: Calling EvaluationRunResult with inputs like {"queries": [q1, q2, q3], "contexts": [c1, c2]} where column lengths differ; building inputs by appending to some keys but not others in a loop.

Common situations: A conditional pipeline branch that skips producing one input for some rows, failed API calls leaving one list shorter, zipping/collecting metrics with different sample counts per column.

Related errors

• No inputs provided.
• Aggregate score missing for {metric}.
• Individual scores missing for {metric}.
• Length of individual scores for '{metric}' should be the sam
• Hook of type '{type(h).__name__}' is registered under hook p

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/16850aee028deb18. Report an issue: GitHub.