{"record":{"id":"669ab27d5f110ae9","repo":"run-llama/llama_index","slug":"queries-must-be-provided","errorCode":null,"errorMessage":"`queries` must be provided","messagePattern":"`queries` must be provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/batch_runner.py","lineNumber":336,"sourceCode":"\n    async def aevaluate_queries(\n        self,\n        query_engine: BaseQueryEngine,\n        queries: Optional[List[str]] = None,\n        **eval_kwargs_lists: Dict[str, Any],\n    ) -> Dict[str, List[EvaluationResult]]:\n        \"\"\"\n        Evaluate queries.\n\n        Args:\n            query_engine (BaseQueryEngine): Query engine.\n            queries (Optional[List[str]]): List of query strings. Defaults to None.\n            **eval_kwargs_lists (Dict[str, Any]): Dict of lists of kwargs to\n                pass to evaluator. Defaults to None.\n\n        \"\"\"\n        if queries is None:\n            raise ValueError(\"`queries` must be provided\")\n\n        # gather responses\n        response_jobs = []\n        for query in queries:\n            response_jobs.append(response_worker(self.semaphore, query_engine, query))\n        responses = await self.asyncio_mod.gather(*response_jobs)\n\n        return await self.aevaluate_responses(\n            queries=queries,\n            responses=responses,\n            **eval_kwargs_lists,\n        )\n\n    def evaluate_response_strs(\n        self,\n        queries: Optional[List[str]] = None,\n        response_strs: Optional[List[str]] = None,\n        contexts_list: Optional[List[List[str]]] = None,","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/batch_runner.py#L318-L354","documentation":"Raised by BatchRunner.aevaluate_queries when the queries argument is None. The runner needs an explicit list of query strings to dispatch through the query_engine via response_worker coroutines before evaluation.","triggerScenarios":"Calling await runner.aevaluate_queries(query_engine=engine, queries=None); typically a variable that was supposed to hold a list of questions but was never populated or was conditionally set to None.","commonSituations":"Loading queries from a file/dataset that failed silently and left the variable None; branching code where one path forgets to assign queries; refactoring from aevaluate_responses to aevaluate_queries and dropping the argument.","solutions":["Pass an explicit non-empty list of query strings: queries=[\"What is X?\", ...].","Add a guard before the call: if not queries: raise/log early with context about why the list is empty.","Verify the upstream loader (dataset read, QueryDataset.queries) actually returned items."],"exampleFix":"# before\nqueries = load_queries()  # returned None\nawait runner.aevaluate_queries(engine, queries)\n\n# after\nqueries = load_queries() or []\nif not queries:\n    raise RuntimeError(\"no queries loaded from dataset\")\nawait runner.aevaluate_queries(engine, queries)","handlingStrategy":"validation","validationCode":"if not queries:\n    raise RuntimeError(f\"refusing to evaluate: {len(queries) if queries else 0} queries\")","typeGuard":"def has_queries(queries) -> bool:\n    return isinstance(queries, list) and len(queries) > 0 and all(isinstance(q, str) for q in queries)","tryCatchPattern":null,"preventionTips":["Validate dataset loading returns a non-empty list before entering the evaluation loop.","Fail fast at the loader with a descriptive error instead of letting the runner raise generically."],"tags":["batch-evaluation","input-validation","api-misuse"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}