run-llama/llama_index · error · ValueError

At least one item in inputs_list must be provided.

Error message

At least one item in inputs_list must be provided.

What it means

Raised by BatchRunner._validate_and_clean_inputs when inputs_list contains no lists, or every element is None. The method first asserts the list itself is non-empty (an empty list instead fails the bare assert), then scans for the first non-None input to determine the shared length; if none exists there is nothing to evaluate against.

Source

Thrown at llama-index-core/llama_index/core/evaluation/batch_runner.py:132

        *inputs_list: Any,
    ) -> List[Any]:
        """
        Validate and clean input lists.

        Enforce that at least one of the inputs is not None.
        Make sure that all inputs have the same length.
        Make sure that None inputs are replaced with [None] * len(inputs).

        """
        assert len(inputs_list) > 0
        # first, make sure at least one of queries or response_strs is not None
        input_len: Optional[int] = None
        for inputs in inputs_list:
            if inputs is not None:
                input_len = len(inputs)
                break
        if input_len is None:
            raise ValueError("At least one item in inputs_list must be provided.")

        new_inputs_list = []
        for inputs in inputs_list:
            if inputs is None:
                new_inputs_list.append([None] * input_len)
            else:
                if len(inputs) != input_len:
                    raise ValueError("All inputs must have the same length.")
                new_inputs_list.append(inputs)
        return new_inputs_list

    def _validate_nested_eval_kwargs_types(
        self, eval_kwargs_lists: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Ensure eval kwargs are acceptable format.
            either a Dict[str, List] or a Dict[str, Dict[str, List]].

View on GitHub (pinned to afd0fef371)

Solutions

1. Pass a real list for at least one kwarg (and None for the others): e.g. references=[ref] * len(queries).
2. If a kwarg is genuinely unused by all evaluators, omit it entirely from the call instead of passing None.
3. Check that you are not passing an empty list [] — either use a populated list or drop the argument (empty lists fail the preceding assert len(inputs_list) > 0).

Example fix

# before
await runner.aevaluate_responses(
    queries=queries, responses=responses, references=None
)

# after
await runner.aevaluate_responses(
    queries=queries, responses=responses,
    references=[None] * len(queries),  # or omit entirely
)

def check_batch_inputs(queries, responses, **kwargs_lists):
    lists = [queries, responses, *kwargs_lists.values()]
    if all(x is None for x in lists):
        raise ValueError("no evaluation inputs provided")
    return True

def has_at_least_one_list(lists) -> bool:
    return any(x is not None and len(x) > 0 for x in lists)

When it happens

Trigger: Passing eval_kwargs lists where every value is None, e.g. await runner.aevaluate_responses(queries=qs, responses=rs, references=None), or an empty list [] as a kwargs value. Also triggered by aevaluate when all eval kwargs are None.

Common situations: Forwarding optional per-evaluator kwargs (like references) that a given evaluator does not need, set to None instead of a list; programmatically building kwargs dicts where a variable defaults to None; mixing evaluators that take references with ones that do not.

Related errors

• code_execute_fn must be provided for CodeActAgent
• All inputs must have the same length.
• `queries` must be provided
• Unsupported message content type: {type(bank_message.content
• First argument to Readability constructor should be a docume

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/94128aefa471bcef. Report an issue: GitHub.