{"record":{"id":"91d5224c7a6b769e","repo":"run-llama/llama_index","slug":"both-query-and-contexts-must-be-provided","errorCode":null,"errorMessage":"Both query and contexts must be provided","messagePattern":"Both query and contexts must be provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/context_relevancy.py","lineNumber":143,"sourceCode":"        if \"eval_template\" in prompts:\n            self._eval_template = prompts[\"eval_template\"]\n        if \"refine_template\" in prompts:\n            self._refine_template = prompts[\"refine_template\"]\n\n    async def aevaluate(\n        self,\n        query: str | None = None,\n        response: str | None = None,\n        contexts: Sequence[str] | None = None,\n        sleep_time_in_seconds: int = 0,\n        **kwargs: Any,\n    ) -> EvaluationResult:\n        \"\"\"Evaluate whether the contexts is relevant to the query.\"\"\"\n        del kwargs  # Unused\n        del response  # Unused\n\n        if query is None or contexts is None:\n            raise ValueError(\"Both query and contexts must be provided\")\n\n        docs = [Document(text=context) for context in contexts]\n        index = SummaryIndex.from_documents(docs)\n\n        await asyncio.sleep(sleep_time_in_seconds)\n\n        query_engine = index.as_query_engine(\n            llm=self._llm,\n            text_qa_template=self._eval_template,\n            refine_template=self._refine_template,\n        )\n        response_obj = await query_engine.aquery(query)\n        raw_response_txt = str(response_obj)\n\n        score, reasoning = self.parser_function(raw_response_txt)\n\n        invalid_result, invalid_reason = False, None\n        if score is None and reasoning is None:","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/context_relevancy.py#L125-L161","documentation":"Thrown by ContextRelevancyEvaluator.aevaluate when either query or contexts is None (response is deliberately discarded). The evaluator builds a SummaryIndex over the context strings and queries it with the query string, so both are mandatory.","triggerScenarios":"Calling await evaluator.aevaluate(query=None, contexts=[...]) or aevaluate(query=q, contexts=None); commonly via BatchRunner where contexts were not supplied per-item (all-None contexts list is fine item-wise, but a missing list propagates None).","commonSituations":"Evaluating from a response object where source_nodes were never retrieved (contexts=[] vs None confusion); integrating with a custom retriever that returns None on empty; forgetting that BatchRunner needs contexts passed as a kwargs list for this evaluator.","solutions":["Always pass both arguments: await evaluator.aevaluate(query=q, contexts=ctxs) with ctxs a list of strings.","When using BatchRunner, pass contexts=[...] (aligned with queries) so each item gets a real list; use None entries only where genuinely absent.","If contexts can be empty, pass [] or [''] rather than None so evaluation can proceed (or skip that item yourself)."],"exampleFix":"# before\nresult = await evaluator.aevaluate(query=q, contexts=None)\n\n# after\ncontexts = [n.get_content() for n in response.source_nodes] or [\"\"]\nresult = await evaluator.aevaluate(query=q, contexts=contexts)","handlingStrategy":"validation","validationCode":"if query is None or contexts is None:\n    raise ValueError(\"skipping context relevancy: missing query or contexts\") from None","typeGuard":"def evaluable_contexts(query, contexts) -> bool:\n    return bool(query) and contexts is not None","tryCatchPattern":null,"preventionTips":["Always derive contexts from response.source_nodes and pass them alongside the query.","In BatchRunner runs, include an aligned contexts=[...] kwarg list."],"tags":["evaluation","context-relevancy","input-validation"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}