{"record":{"id":"4ceb1905c61a90f6","repo":"run-llama/llama_index","slug":"query-and-response-must-be-provided","errorCode":null,"errorMessage":"query and response must be provided","messagePattern":"query and response must be provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/answer_relevancy.py","lineNumber":117,"sourceCode":"        if \"eval_template\" in prompts:\n            self._eval_template = prompts[\"eval_template\"]\n        if \"refine_template\" in prompts:\n            self._refine_template = prompts[\"refine_template\"]\n\n    async def aevaluate(\n        self,\n        query: str | None = None,\n        response: str | None = None,\n        contexts: Sequence[str] | None = None,\n        sleep_time_in_seconds: int = 0,\n        **kwargs: Any,\n    ) -> EvaluationResult:\n        \"\"\"Evaluate whether the response is relevant to the query.\"\"\"\n        del kwargs  # Unused\n        del contexts  # Unused\n\n        if query is None or response is None:\n            raise ValueError(\"query and response must be provided\")\n\n        await asyncio.sleep(sleep_time_in_seconds)\n\n        eval_response = await self._llm.apredict(\n            prompt=self._eval_template,\n            query=query,\n            response=response,\n        )\n\n        score, reasoning = self.parser_function(eval_response)\n\n        invalid_result, invalid_reason = False, None\n        if score is None and reasoning is None:\n            if self._raise_error:\n                raise ValueError(\"The response is invalid\")\n            invalid_result = True\n            invalid_reason = \"Unable to parse the output string.\"\n","sourceCodeStart":99,"sourceCodeEnd":135,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/answer_relevancy.py#L99-L135","documentation":"AnswerRelevancyEvaluator.aevaluate() scores how well a response addresses a query and requires both pieces of text. It deliberately ignores contexts/kwargs, so passing only contexts (a common habit from faithfulness/relevancy evaluators) or omitting either string is rejected with this ValueError before any LLM call.","triggerScenarios":"Calling evaluator.aevaluate(query=q) without response; aevaluate(response=r) without query; aevaluate(contexts=[...]) expecting contexts to suffice (they are deleted); positionally passing only one argument.","commonSituations":"Reusing one generic eval loop across multiple evaluator types (FaithfulnessEvaluator needs contexts, AnswerRelevancy needs query+response); refactors that drop the response field; batch-eval notebooks where one row has a missing value.","solutions":["Pass both: await evaluator.aevaluate(query=..., response=...)","In shared eval loops, branch per evaluator type or use getattr on results to source query/response from a QueryResponse pair","Validate your eval dataset rows have non-null query and response fields before running"],"exampleFix":"// before\nresult = await evaluator.aevaluate(\n    query=\"What is LlamaIndex?\", contexts=response.source_nodes  # ValueError\n)\n\n// after\nresult = await evaluator.aevaluate(\n    query=\"What is LlamaIndex?\", response=response.response\n)","handlingStrategy":"validation","validationCode":"if not query or not response:\n    raise ValueError(\"AnswerRelevancyEvaluator needs non-empty query and response\")","typeGuard":"def is_evaluable_pair(query, response) -> bool:\n    return bool(query) and bool(response)","tryCatchPattern":"try:\n    result = await evaluator.aevaluate(query=q, response=r)\nexcept ValueError as e:\n    if \"must be provided\" in str(e):\n        result = None  # skip incomplete eval rows\n    else:\n        raise","preventionTips":["Read each evaluator's aevaluate signature: AnswerRelevancy uses query+response, Faithfulness uses response+contexts","Validate eval datasets for null query/response cells before batch runs","Source query/response from the same (query_str, Response) pair produced by query_engine.aquery"],"tags":["evaluation","api-misuse","validation","llm"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}