{"record":{"id":"d795587e81d26b01","repo":"run-llama/llama_index","slug":"retrieved-ids-and-expected-ids-must-be-provided","errorCode":null,"errorMessage":"Retrieved ids and expected ids must be provided","messagePattern":"Retrieved ids and expected ids must be provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/retrieval/metrics.py","lineNumber":68,"sourceCode":"            retrieved_texts (Optional[List[str]]): Retrieved texts (not used in the current implementation).\n\n        Raises\n        ------\n            ValueError: If the necessary IDs are not provided.\n\n        Returns\n        -------\n            RetrievalMetricResult: The result with the computed hit rate score.\n\n        \"\"\"\n        # Checking for the required arguments\n        if (\n            retrieved_ids is None\n            or expected_ids is None\n            or not retrieved_ids\n            or not expected_ids\n        ):\n            raise ValueError(\"Retrieved ids and expected ids must be provided\")\n\n        if self.use_granular_hit_rate:\n            # Granular HitRate calculation: Calculate all hits and divide by the number of expected docs\n            expected_set = set(expected_ids)\n            hits = sum(1 for doc_id in retrieved_ids if doc_id in expected_set)\n            score = hits / len(expected_ids) if expected_ids else 0.0\n        else:\n            # Default HitRate calculation: Check if there is a single hit\n            is_hit = any(id in expected_ids for id in retrieved_ids)\n            score = 1.0 if is_hit else 0.0\n\n        return RetrievalMetricResult(score=score)\n\n\nclass MRR(BaseRetrievalMetric):\n    \"\"\"\n    MRR (Mean Reciprocal Rank) metric with two calculation options.\n","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/retrieval/metrics.py#L50-L86","documentation":"HitRate.compute requires both retrieved_ids and expected_ids to be non-None AND non-empty. The guard rejects None or falsy lists (empty lists fail `not retrieved_ids`), so evaluating a retrieval that returned zero docs raises ValueError before scoring.","triggerScenarios":"Calling hit_rate.compute(retrieved_ids=[], expected_ids=['id1']) (retriever returned nothing), or either argument None, or both empty.","commonSituations":"Top-k retrievers returning zero results for out-of-domain queries; eval datasets with empty golden ids; pipeline bugs producing empty retrieved lists; treating empty retrieval as scoreable.","solutions":["Skip metric computation when either list is empty rather than calling compute","Ensure your retriever returns at least one node (check similarity cutoffs / filters that can zero out results)","Validate the dataset: every query must have at least one expected id"],"exampleFix":"# before\nscore = hit_rate.compute(retrieved_ids=retrieved, expected_ids=expected)  # retrieved == []\n\n# after\nif retrieved and expected:\n    score = hit_rate.compute(retrieved_ids=retrieved, expected_ids=expected)\nelse:\n    score = 0.0  # or skip this query","handlingStrategy":"validation","validationCode":"def compute_hit_rate(metric, retrieved_ids, expected_ids):\n    if not retrieved_ids or not expected_ids:\n        return RetrievalMetricResult(score=0.0)  # or None to mark not-computable\n    return metric.compute(retrieved_ids=retrieved_ids, expected_ids=expected_ids)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never call retrieval metrics with empty lists; short-circuit first","Alert when retrievers return zero results — usually a config problem","Validate datasets have expected ids for every query"],"tags":["validation","evaluation","retrieval-metrics","hit-rate","required-args"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}