deepset-ai/haystack · error

All lists in the JSON must have the same length

Error message

All lists in the JSON must have the same length

What it means

EvaluationRunResult._write_to_csv writes a rectangular table, so every list in the data dict must have the same length. If the lengths differ, no row alignment is possible and a ValueError is raised before any file is written.

Source

Thrown at haystack/evaluation/eval_run_result.py:75

            if len(outputs["individual_scores"]) != expected_len:
                raise ValueError(
                    f"Length of individual scores for '{metric}' should be the same as the inputs. "
                    f"Got {len(outputs['individual_scores'])} but expected {expected_len}."
                )

    @staticmethod
    def _write_to_csv(csv_file: str, data: dict[str, list[Any]]) -> str:
        """
        Write data to a CSV file.

        :param csv_file: Path to the CSV file to write
        :param data: Dictionary containing the data to write
        :return: Status message indicating success or failure
        """
        list_lengths = [len(value) for value in data.values()]

        if len(set(list_lengths)) != 1:
            raise ValueError("All lists in the JSON must have the same length")

        try:
            headers = list(data.keys())
            num_rows = list_lengths[0]
            rows = []

            for i in range(num_rows):
                row = [data[header][i] for header in headers]
                rows.append(row)

            with open(csv_file, "w", newline="") as csvfile:
                writer = csv.writer(csvfile)
                writer.writerow(headers)
                writer.writerows(rows)

            return f"Data successfully written to {csv_file}"
        except PermissionError:
            return f"Error: Permission denied when writing to {csv_file}"

View on GitHub (pinned to e318778c9b)

Solutions

1. Make all columns equal length before requesting CSV output (pad with None or trim).
2. Re-create the EvaluationRunResult with validated, length-matched data.
3. Use output_format='json' or 'df' if ragged data is intentional.

Example fix

// before
data = {'inputs': [1, 2, 3], 'scores': [1, 2]}
_write_to_csv('out.csv', data)
// after
data['scores'] = data['scores'] + [None]
_write_to_csv('out.csv', data)

lengths = {k: len(v) for k, v in data.items()}
if len(set(lengths.values())) != 1:
    raise ValueError(f'Unequal column lengths: {lengths}')

null

try:
    run.detailed_report(output_format='csv', csv_file='out.csv')
except ValueError as e:
    if 'same length' in str(e):
        run.detailed_report(output_format='json')  # fall back to non-rectangular format
    else:
        raise

When it happens

Trigger: Calling aggregated_report/detailed_report(output_format='csv', csv_file=...) when the internal data dict has columns of unequal length — normally caused by constructing EvaluationRunResult with mismatched individual_scores that bypassed checks (e.g. mutated after construction).

Common situations: Manually mutating a result object's results dict after construction; a custom subclass overriding report generation; corrupted evaluation data loaded back from disk.

Related errors

• CSVToDocument: quotechar must be a single character.
• CSVToDocument(row): 'content_column' is required in run() wh
• CSVToDocument(row): could not parse CSV rows for {source}: {
• CSVToDocument(row): content_column='{content_column}' not fo
• CSVToDocument(row): failed to process row {i} for {source}:

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/e1d7134e09ea689d. Report an issue: GitHub.