{"record":{"id":"9ce3399abf27e32c","repo":"run-llama/llama_index","slug":"invalid-metric-name-metric","errorCode":null,"errorMessage":"Invalid metric name: {metric}","messagePattern":"Invalid metric name: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/retrieval/metrics.py","lineNumber":511,"sourceCode":"        )\n\n\nMETRIC_REGISTRY: Dict[str, Type[BaseRetrievalMetric]] = {\n    \"hit_rate\": HitRate,\n    \"mrr\": MRR,\n    \"precision\": Precision,\n    \"recall\": Recall,\n    \"ap\": AveragePrecision,\n    \"ndcg\": NDCG,\n    \"cohere_rerank_relevancy\": CohereRerankRelevancyMetric,\n}\n\n\ndef resolve_metrics(metrics: List[str]) -> List[Type[BaseRetrievalMetric]]:\n    \"\"\"Resolve metrics from list of metric names.\"\"\"\n    for metric in metrics:\n        if metric not in METRIC_REGISTRY:\n            raise ValueError(f\"Invalid metric name: {metric}\")\n\n    return [METRIC_REGISTRY[metric] for metric in metrics]\n","sourceCodeStart":493,"sourceCodeEnd":514,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/retrieval/metrics.py#L493-L514","documentation":"Raised by resolve_metrics() in the retrieval-evaluation metrics registry when a requested metric name is not a key of METRIC_REGISTRY. Valid names are exactly: hit_rate, mrr (plus their classes), precision, recall, ap, ndcg, and cohere_rerank_relevancy; the check is a simple dict membership test before class resolution.","triggerScenarios":"Calling resolve_metrics(['hit-rate']) with a hyphen instead of underscore, passing a metric class instead of its name string, or passing a typo like 'cohire_rerank_relevancy' when building a RetrieverEvaluator.","commonSituations":"Translating metric names from other libraries (sklearn's 'average_precision', BEIR conventions); renaming between llama-index versions; user-supplied config files (YAML/CLI) feeding metric names into an eval harness.","solutions":["Use one of the registered names: 'hit_rate', 'mrr', 'precision', 'recall', 'ap', 'ndcg', 'cohere_rerank_relevancy'.","Validate user-provided metric names against METRIC_REGISTRY keys before calling resolve_metrics, and surface the allowed list in your error message.","Check for hyphen/underscore mismatches and case differences in config-driven metric lists."],"exampleFix":"# before\nmetrics = resolve_metrics(['hit-rate', 'average_precision'])\n\n# after\nfrom llama_index.core.evaluation.retrieval.metrics import METRIC_REGISTRY\nmetrics = resolve_metrics(['hit_rate', 'ap'])\nassert set(user_metrics) <= set(METRIC_REGISTRY), f\"allowed: {sorted(METRIC_REGISTRY)}\"","handlingStrategy":"validation","validationCode":"from llama_index.core.evaluation.retrieval.metrics import METRIC_REGISTRY\n\ninvalid = [m for m in requested_metrics if m not in METRIC_REGISTRY]\nif invalid:\n    raise ValueError(f\"Unknown metrics {invalid}; allowed: {sorted(METRIC_REGISTRY)}\")\nmetrics = resolve_metrics(requested_metrics)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Whitelist metric names at the config boundary against METRIC_REGISTRY keys.","Normalize underscores/hyphens and casing before resolution.","Echo the allowed metric list in user-facing errors."],"tags":["validation","configuration","evaluation","typo"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}