Invalid output format '{output_format}' provided. Choose fro

Error message

Invalid output format '{output_format}' provided. Choose from 'json', 'csv', or 'df'.

What it means

_handle_output only supports 'json', 'csv', and 'df' output formats. Any other string (including case variants like 'CSV' or 'dataframe') reaches the final fallthrough and raises this ValueError listing the valid options.

Source

Thrown at haystack/evaluation/eval_run_result.py:120

    ) -> Union[str, "DataFrame", dict[str, list[Any]]]:
        """
        Handles output formatting based on `output_format`.

        :returns: DataFrame for 'df', dict for 'json', or confirmation message for 'csv'
        """
        if output_format == "json":
            return data

        if output_format == "df":
            pandas_import.check()
            return DataFrame(data)

        if output_format == "csv":
            if not csv_file:
                raise ValueError("A file path must be provided in 'csv_file' parameter to save the CSV output.")
            return EvaluationRunResult._write_to_csv(csv_file, data)

        raise ValueError(f"Invalid output format '{output_format}' provided. Choose from 'json', 'csv', or 'df'.")

    def aggregated_report(
        self, output_format: Literal["json", "csv", "df"] = "json", csv_file: str | None = None
    ) -> Union[dict[str, list[Any]], "DataFrame", str]:
        """
        Generates a report with aggregated scores for each metric.

        :param output_format: The output format for the report, "json", "csv", or "df", default to "json".
        :param csv_file: Filepath to save CSV output if `output_format` is "csv", must be provided.

        :returns:
            JSON or DataFrame with aggregated scores, in case the output is set to a CSV file, a message confirming the
            successful write or an error message.
        """
        results = {k: v["score"] for k, v in self.results.items()}
        data = {"metrics": list(results.keys()), "score": list(results.values())}
        return self._handle_output(data, output_format, csv_file)

View on GitHub (pinned to e318778c9b)

Solutions

Use exactly one of 'json', 'csv', or 'df' (lowercase).
Normalize input: output_format = output_format.lower() before calling.
Add a whitelist check in calling code to fail fast with a friendly message.

Example fix

// before
run.aggregated_report(output_format='dataframe')
// after
run.aggregated_report(output_format='df')

Defensive patterns

Strategy: validation

Validate before calling

VALID_FORMATS = {'json', 'csv', 'df'}
if output_format not in VALID_FORMATS:
    raise ValueError(f'output_format must be one of {VALID_FORMATS}')

Type guard

from typing import Literal
OutputFormat = Literal['json', 'csv', 'df']
def is_valid_format(f: str) -> bool:
    return f in ('json', 'csv', 'df')

Try / catch

try:
    report = run.aggregated_report(output_format=user_format)
except ValueError as e:
    if 'Invalid output format' in str(e):
        report = run.aggregated_report(output_format='json')
    else:
        raise

Prevention

Use Literal['json','csv','df'] type annotations so type checkers catch bad values
Normalize user-supplied formats with .lower() and map aliases ('dataframe' -> 'df')
Centralize report generation in one helper that validates the format once

When it happens

Trigger: Calling aggregated_report/detailed_report/comparative_detailed_report with output_format set to any value other than 'json', 'csv', or 'df' — e.g. 'dataframe', 'JSON', 'pandas', or a None value.

Common situations: Typo or wrong-cased format string; passing a user-supplied format option through without validation; confusing the format name with the library name ('pandas' instead of 'df').

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/5e00d2682735e15c. Report an issue: GitHub.