{"record":{"id":"83384416ffdaf038","repo":"deepset-ai/haystack","slug":"llm-evaluator-expects-all-input-values-to-be-lists","errorCode":null,"errorMessage":"LLM evaluator expects all input values to be lists but received {[type(_input) for _input in received.values()]}.","messagePattern":"LLM evaluator expects all input values to be lists but received (.+?)\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/llm_evaluator.py","lineNumber":457,"sourceCode":"            The received input parameters.\n\n        :raises ValueError:\n            If not all expected inputs are present in the received inputs\n            If the received inputs are not lists or have different lengths\n        \"\"\"\n        # Validate that all expected inputs are present in the received inputs\n        for param in expected:\n            if param not in received:\n                msg = f\"LLM evaluator expected input parameter '{param}' but received only {received.keys()}.\"\n                raise ValueError(msg)\n\n        # Validate that all received inputs are lists\n        if not all(isinstance(_input, list) for _input in received.values()):\n            msg = (\n                \"LLM evaluator expects all input values to be lists but received \"\n                f\"{[type(_input) for _input in received.values()]}.\"\n            )\n            raise ValueError(msg)\n\n        # Validate that all received inputs are of the same length\n        inputs = received.values()\n        length = len(next(iter(inputs)))\n        if not all(len(_input) == length for _input in inputs):\n            msg = (\n                f\"LLM evaluator expects all input lists to have the same length but received {inputs} with lengths \"\n                f\"{[len(_input) for _input in inputs]}.\"\n            )\n            raise ValueError(msg)\n","sourceCodeStart":439,"sourceCodeEnd":468,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/llm_evaluator.py#L439-L468","documentation":"The LLM evaluator requires every input value to be a list of items (one per evaluation example). If any received value is not a list — e.g. a single string, dict or None — it raises ValueError listing the received Python types.","triggerScenarios":"Calling run()/run_async() with a scalar instead of a list, e.g. run({\"questions\": \"What is AI?\", \"responses\": \"...\"}) instead of wrapping each value in a list.","commonSituations":"Calling the evaluator outside a Pipeline (pipelines usually pass lists per input name); batching code that passes one item instead of a batch; upstream component emitting a single value rather than a list.","solutions":["Wrap each input value in a list, even for a single item: run({\"questions\": [q], \"responses\": [r]})","Verify the upstream component's output is a list; adapt with a component that wraps scalars if needed","Check pipeline connection types; the error only fires on direct .run() calls since connections are type-checked at connect time"],"exampleFix":"// before\nevaluator.run({\"questions\": question, \"responses\": answer})\n// after\nevaluator.run({\"questions\": [question], \"responses\": [answer]})","handlingStrategy":"validation","validationCode":"for key, value in inputs.items():\n    if not isinstance(value, list):\n        inputs[key] = [value]  # or raise","typeGuard":"def is_list_of_inputs(inputs: dict) -> bool:\n    return all(isinstance(v, list) for v in inputs.values())","tryCatchPattern":"try:\n    result = evaluator.run(inputs)\nexcept ValueError:\n    inputs = {k: (v if isinstance(v, list) else [v]) for k, v in inputs.items()}\n    result = evaluator.run(inputs)","preventionTips":["Always wrap single items in a list before calling the evaluator","Call the evaluator through a Pipeline so connection type checks catch scalars early","Write a test evaluating a single example to keep the list-wrapping path covered"],"tags":["python","validation","type-error"],"backgroundTag":"wrong-input-type","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}