{"record":{"id":"16850aee028deb18","repo":"deepset-ai/haystack","slug":"lengths-of-the-inputs-should-be-the-same","errorCode":null,"errorMessage":"Lengths of the inputs should be the same.","messagePattern":"Lengths of the inputs should be the same\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/evaluation/eval_run_result.py","lineNumber":47,"sourceCode":"\n        :param inputs:\n            Dictionary containing the inputs used for the run. Each key is the name of the input and its value is a list\n            of input values. The length of the lists should be the same.\n\n        :param results:\n            Dictionary containing the results of the evaluators used in the evaluation pipeline. Each key is the name\n            of the metric and its value is dictionary with the following keys:\n                - 'score': The aggregated score for the metric.\n                - 'individual_scores': A list of scores for each input sample.\n        \"\"\"\n        self.run_name = run_name\n        self.inputs = deepcopy(inputs)\n        self.results = deepcopy(results)\n\n        if len(inputs) == 0:\n            raise ValueError(\"No inputs provided.\")\n        if len({len(lst) for lst in inputs.values()}) != 1:\n            raise ValueError(\"Lengths of the inputs should be the same.\")\n\n        expected_len = len(next(iter(inputs.values())))\n\n        for metric, outputs in results.items():\n            if \"score\" not in outputs:\n                raise ValueError(f\"Aggregate score missing for {metric}.\")\n            if \"individual_scores\" not in outputs:\n                raise ValueError(f\"Individual scores missing for {metric}.\")\n\n            if len(outputs[\"individual_scores\"]) != expected_len:\n                raise ValueError(\n                    f\"Length of individual scores for '{metric}' should be the same as the inputs. \"\n                    f\"Got {len(outputs['individual_scores'])} but expected {expected_len}.\"\n                )\n\n    @staticmethod\n    def _write_to_csv(csv_file: str, data: dict[str, list[Any]]) -> str:\n        \"\"\"","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/evaluation/eval_run_result.py#L29-L65","documentation":"EvaluationRunResult.__init__ raises ValueError when the input lists in the inputs dict have different lengths. Inputs and metric outputs are aligned positionally per evaluated item, so ragged input columns make the alignment impossible.","triggerScenarios":"Calling EvaluationRunResult with inputs like {\"queries\": [q1, q2, q3], \"contexts\": [c1, c2]} where column lengths differ; building inputs by appending to some keys but not others in a loop.","commonSituations":"A conditional pipeline branch that skips producing one input for some rows, failed API calls leaving one list shorter, zipping/collecting metrics with different sample counts per column.","solutions":["Pad or fix the shorter input lists so every list has the same length before construction","Fix the data collection loop so every input key gets exactly one entry per evaluated item","Verify lengths programmatically: assert len({len(v) for v in inputs.values()}) == 1"],"exampleFix":"// before\ninputs = {\"queries\": [q1, q2, q3], \"contexts\": [c1, c2]}  # ValueError\n// after\nassert len(inputs[\"queries\"]) == len(inputs[\"contexts\"])\ninputs = {\"queries\": [q1, q2, q3], \"contexts\": [c1, c2, c3]}\nrun = EvaluationRunResult(\"run1\", inputs=inputs, results=results)","handlingStrategy":"validation","validationCode":"lengths = {len(v) for v in inputs.values()}\nif len(lengths) > 1:\n    raise RuntimeError(f\"Input columns have different lengths: { {k: len(v) for k, v in inputs.items()} }\")","typeGuard":null,"tryCatchPattern":"try:\n    run = EvaluationRunResult(run_name, inputs=inputs, results=results)\nexcept ValueError as e:\n    if \"Lengths of the inputs should be the same\" in str(e):\n        target = max(len(v) for v in inputs.values())\n        inputs = {k: v + [None] * (target - len(v)) for k, v in inputs.items()}  # or fix data\n        run = EvaluationRunResult(run_name, inputs=inputs, results=results)\n    else:\n        raise","preventionTips":["Build all input columns in one loop so each evaluated item contributes to every key","Assert equal column lengths right after collecting inputs","Investigate pipeline branches/failed calls that skip producing some inputs"],"tags":["evaluation","validation","data-alignment"],"backgroundTag":"input-length-mismatch","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}