{"record":{"id":"70992b81cdfd3e97","repo":"run-llama/llama_index","slug":"query-contexts-and-response-must-be-provided","errorCode":null,"errorMessage":"query, contexts, and response must be provided","messagePattern":"query, contexts, and response must be provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/multi_modal/relevancy.py","lineNumber":125,"sourceCode":"        if \"eval_template\" in prompts:\n            self._eval_template = prompts[\"eval_template\"]\n        if \"refine_template\" in prompts:\n            self._refine_template = prompts[\"refine_template\"]\n\n    def evaluate(\n        self,\n        query: Union[str, None] = None,\n        response: Union[str, None] = None,\n        contexts: Union[Sequence[str], None] = None,\n        image_paths: Union[List[str], None] = None,\n        image_urls: Union[List[str], None] = None,\n        **kwargs: Any,\n    ) -> EvaluationResult:\n        \"\"\"Evaluate whether the multi-modal contexts and response are relevant to the query.\"\"\"\n        del kwargs  # Unused\n\n        if query is None or contexts is None or response is None:\n            raise ValueError(\"query, contexts, and response must be provided\")\n\n        context_str = \"\\n\\n\".join(contexts)\n        evaluation_query_str = f\"Question: {query}\\nResponse: {response}\"\n        fmt_prompt = self._eval_template.format(\n            context_str=context_str, query_str=evaluation_query_str\n        )\n\n        blocks: List[Union[ImageBlock, TextBlock]] = []\n\n        if image_paths:\n            blocks.extend(\n                [ImageBlock(path=Path(image_path)) for image_path in image_paths]\n            )\n        if image_urls:\n            blocks.extend([ImageBlock(url=image_url) for image_url in image_urls])\n\n        blocks.append(TextBlock(text=fmt_prompt))\n","sourceCodeStart":107,"sourceCodeEnd":143,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/multi_modal/relevancy.py#L107-L143","documentation":"Raised by MultiModalRelevancyEvaluator.evaluate when any of query, contexts, or response is None. These three parameters are required to build the evaluation prompt (context_str and evaluation_query_str); the signature marks them Optional only to allow keyword omission, and the guard rejects None before any LLM call.","triggerScenarios":"Calling evaluate(query=..., contexts=None, response=...) or omitting any of the three kwargs; passing an empty pipeline result (e.g. response=None because the RAG step failed or was skipped).","commonSituations":"Wiring an eval loop where some samples lack retrieved contexts or a generated response; mapping over batches where a field is None for failed items; renaming kwargs (answer vs response).","solutions":["Ensure all three values are non-None strings/lists before calling: evaluate(query=q, contexts=ctxs, response=resp)","Filter or skip samples with missing fields in your eval loop before invoking the evaluator","Default missing values explicitly, e.g. response='' or contexts=[] if you intentionally want empty inputs evaluated"],"exampleFix":"# before\nresult = evaluator.evaluate(query=q, contexts=None, response=r)\n\n# after\nif q is None or r is None or ctxs is None:\n    continue  # skip incomplete sample\nresult = evaluator.evaluate(query=q, contexts=ctxs, response=r)","handlingStrategy":"validation","validationCode":"def can_evaluate(query, contexts, response) -> bool:\n    return query is not None and contexts is not None and response is not None\n\nif can_evaluate(q, ctxs, resp):\n    result = evaluator.evaluate(query=q, contexts=ctxs, response=resp)","typeGuard":null,"tryCatchPattern":"try:\n    result = evaluator.evaluate(query=q, contexts=ctxs, response=resp)\nexcept ValueError as e:\n    if \"must be provided\" in str(e):\n        logger.warning(\"skipping incomplete sample\")\n        continue\n    raise","preventionTips":["Validate eval samples (query/contexts/response all non-None) before the evaluator call","Type your dataclasses so fields can't silently be None","In batch loops, log skipped samples instead of crashing the whole run"],"tags":["validation","evaluation","multi-modal","required-args"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}