{"record":{"id":"ff6cfe5748fb6504","repo":"run-llama/llama_index","slug":"must-specify-both-response-and-reference","errorCode":null,"errorMessage":"Must specify both response and reference","messagePattern":"Must specify both response and reference","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/semantic_similarity.py","lineNumber":70,"sourceCode":"    def _get_prompts(self) -> PromptDictType:\n        \"\"\"Get prompts.\"\"\"\n        return {}\n\n    def _update_prompts(self, prompts: PromptDictType) -> None:\n        \"\"\"Update prompts.\"\"\"\n\n    async def aevaluate(\n        self,\n        query: Optional[str] = None,\n        response: Optional[str] = None,\n        contexts: Optional[Sequence[str]] = None,\n        reference: Optional[str] = None,\n        **kwargs: Any,\n    ) -> EvaluationResult:\n        del query, contexts, kwargs  # Unused\n\n        if response is None or reference is None:\n            raise ValueError(\"Must specify both response and reference\")\n\n        response_embedding = await self._embed_model.aget_text_embedding(response)\n        reference_embedding = await self._embed_model.aget_text_embedding(reference)\n\n        similarity_score = self._similarity_fn(response_embedding, reference_embedding)\n        passing = similarity_score >= self._similarity_threshold\n        return EvaluationResult(\n            score=similarity_score,\n            passing=passing,\n            feedback=f\"Similarity score: {similarity_score}\",\n        )\n","sourceCodeStart":52,"sourceCodeEnd":82,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/semantic_similarity.py#L52-L82","documentation":"Raised by SemanticSimilarityEvaluator.aevaluate when either response or reference is None. This evaluator embeds the generated response and a ground-truth reference and compares the embeddings; query and contexts are explicitly deleted as unused, so both text arguments are mandatory inputs.","triggerScenarios":"Calling await evaluator.aevaluate(query=..., contexts=...) without response/reference; passing reference=None for LLM-generated answers with no gold answer; evaluating a pipeline whose response was empty (None) due to an upstream failure.","commonSituations":"Building an eval harness that loops over datasets where some rows lack reference answers; an upstream LLM call returning None and the harness forwarding it; reusing a CorrectnessEvaluator-style call signature that passes query+contexts only.","solutions":["Provide both strings: await evaluator.aevaluate(response=resp, reference=ref).","Skip or separately flag dataset rows with missing references before running evaluation.","Guard upstream: if the LLM response is None/empty, short-circuit the eval with a failing result instead of calling aevaluate."],"exampleFix":"# before\nresult = await evaluator.aevaluate(query=q, contexts=ctx)  # response/reference missing\n\n# after\nresult = await evaluator.aevaluate(response=answer, reference=gold_answer)","handlingStrategy":"validation","validationCode":"if not response or not reference:\n    return EvaluationResult(score=0.0, passing=False, feedback=\"missing response/reference\")\nresult = await evaluator.aevaluate(response=response, reference=reference)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Filter dataset rows lacking reference answers before evaluation loops.","Short-circuit when upstream LLM responses are None/empty instead of forwarding them.","Keep a required-fields assertion in batch eval harnesses."],"tags":["validation","evaluation","missing-argument","embeddings"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}