{"record":{"id":"317405a741a15438","repo":"deepset-ai/haystack","slug":"llm-evaluator-expects-all-input-lists-to-have-the","errorCode":null,"errorMessage":"LLM evaluator expects all input lists to have the same length but received {inputs} with lengths {[len(_input) for _input in inputs]}.","messagePattern":"LLM evaluator expects all input lists to have the same length but received (.+?) with lengths (.+?)\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/llm_evaluator.py","lineNumber":467,"sourceCode":"                raise ValueError(msg)\n\n        # Validate that all received inputs are lists\n        if not all(isinstance(_input, list) for _input in received.values()):\n            msg = (\n                \"LLM evaluator expects all input values to be lists but received \"\n                f\"{[type(_input) for _input in received.values()]}.\"\n            )\n            raise ValueError(msg)\n\n        # Validate that all received inputs are of the same length\n        inputs = received.values()\n        length = len(next(iter(inputs)))\n        if not all(len(_input) == length for _input in inputs):\n            msg = (\n                f\"LLM evaluator expects all input lists to have the same length but received {inputs} with lengths \"\n                f\"{[len(_input) for _input in inputs]}.\"\n            )\n            raise ValueError(msg)\n","sourceCodeStart":449,"sourceCodeEnd":468,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/llm_evaluator.py#L449-L468","documentation":"All input lists of the LLM evaluator must have the same length so items can be paired per example. If lengths differ it raises ValueError showing the received values and their lengths.","triggerScenarios":"Calling run()/run_async() with e.g. 10 questions but 8 responses; inputs = {\"questions\": [...10], \"responses\": [...8]} causes the length-equality check in validate_input_parameters to fail.","commonSituations":"Filtering or dropping failed answers from one list but not the other before evaluation; an upstream generator producing fewer responses than prompts; off-by-one slicing of test data.","solutions":["Make all input lists the same length before calling run(), pairing each question with its response","If some answers are missing, pad with placeholders (e.g. \"\") to keep alignment rather than dropping from one list","Log/assert len() equality on all inputs before evaluation"],"exampleFix":"// before\nevaluator.run({\"questions\": questions[:5], \"responses\": responses})\n// after\nevaluator.run({\"questions\": questions[:5], \"responses\": responses[:5]})","handlingStrategy":"validation","validationCode":"lengths = {k: len(v) for k, v in inputs.items()}\nif len(set(lengths.values())) > 1:\n    raise ValueError(f\"Input lengths differ: {lengths}\")","typeGuard":null,"tryCatchPattern":"try:\n    result = evaluator.run(inputs)\nexcept ValueError as e:\n    logger.error(\"Unequal input lengths: %s\", e)\n    raise","preventionTips":["Filter paired lists together using zip so indices stay aligned","Assert len() equality on all evaluator inputs before evaluation","Avoid independent slicing/filtering of questions and answers"],"tags":["python","validation","length-mismatch"],"backgroundTag":"list-length-mismatch","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}