LLM evaluator expects all input values to be lists but recei

Error message

LLM evaluator expects all input values to be lists but received {[type(_input) for _input in received.values()]}.

What it means

The LLM evaluator requires every input value to be a list of items (one per evaluation example). If any received value is not a list — e.g. a single string, dict or None — it raises ValueError listing the received Python types.

Source

Thrown at haystack/components/evaluators/llm_evaluator.py:457

            The received input parameters.

        :raises ValueError:
            If not all expected inputs are present in the received inputs
            If the received inputs are not lists or have different lengths
        """
        # Validate that all expected inputs are present in the received inputs
        for param in expected:
            if param not in received:
                msg = f"LLM evaluator expected input parameter '{param}' but received only {received.keys()}."
                raise ValueError(msg)

        # Validate that all received inputs are lists
        if not all(isinstance(_input, list) for _input in received.values()):
            msg = (
                "LLM evaluator expects all input values to be lists but received "
                f"{[type(_input) for _input in received.values()]}."
            )
            raise ValueError(msg)

        # Validate that all received inputs are of the same length
        inputs = received.values()
        length = len(next(iter(inputs)))
        if not all(len(_input) == length for _input in inputs):
            msg = (
                f"LLM evaluator expects all input lists to have the same length but received {inputs} with lengths "
                f"{[len(_input) for _input in inputs]}."
            )
            raise ValueError(msg)

View on GitHub (pinned to e318778c9b)

Solutions

Wrap each input value in a list, even for a single item: run({"questions": [q], "responses": [r]})
Verify the upstream component's output is a list; adapt with a component that wraps scalars if needed
Check pipeline connection types; the error only fires on direct .run() calls since connections are type-checked at connect time

Example fix

// before
evaluator.run({"questions": question, "responses": answer})
// after
evaluator.run({"questions": [question], "responses": [answer]})

Defensive patterns

Strategy: validation

Validate before calling

for key, value in inputs.items():
    if not isinstance(value, list):
        inputs[key] = [value]  # or raise

Type guard

def is_list_of_inputs(inputs: dict) -> bool:
    return all(isinstance(v, list) for v in inputs.values())

Try / catch

try:
    result = evaluator.run(inputs)
except ValueError:
    inputs = {k: (v if isinstance(v, list) else [v]) for k, v in inputs.items()}
    result = evaluator.run(inputs)

Prevention

Always wrap single items in a list before calling the evaluator
Call the evaluator through a Pipeline so connection type checks catch scalars early
Write a test evaluating a single example to keep the list-wrapping path covered

When it happens

Trigger: Calling run()/run_async() with a scalar instead of a list, e.g. run({"questions": "What is AI?", "responses": "..."}) instead of wrapping each value in a list.

Common situations: Calling the evaluator outside a Pipeline (pipelines usually pass lists per input name); batching code that passes one item instead of a batch; upstream component emitting a single value rather than a list.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/83384416ffdaf038. Report an issue: GitHub.