{"record":{"id":"04a79323bf2dab37","repo":"run-llama/llama_index","slug":"metric-key-metric-key-not-in-results-df","errorCode":null,"errorMessage":"Metric key {metric_key} not in results_df","messagePattern":"Metric key (.+?) not in results_df","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/notebook_utils.py","lineNumber":37,"sourceCode":"        import pandas as pd\n    except ImportError:\n        raise ImportError(\n            \"pandas is required for this function. Please install it with `pip install pandas`.\"\n        )\n\n    metric_keys = metric_keys or DEFAULT_METRIC_KEYS\n\n    avg_metrics_dict = defaultdict(list)\n    for name, eval_results in zip(names, results_arr):\n        metric_dicts = []\n        for eval_result in eval_results:\n            metric_dict = eval_result.metric_vals_dict\n            metric_dicts.append(metric_dict)\n        results_df = pd.DataFrame(metric_dicts)\n\n        for metric_key in metric_keys:\n            if metric_key not in results_df.columns:\n                raise ValueError(f\"Metric key {metric_key} not in results_df\")\n            avg_metrics_dict[metric_key].append(results_df[metric_key].mean())\n\n    return pd.DataFrame({\"retrievers\": names, **avg_metrics_dict})\n\n\ndef get_eval_results_df(\n    names: List[str], results_arr: List[EvaluationResult], metric: Optional[str] = None\n) -> Tuple[Any, Any]:\n    \"\"\"\n    Organizes EvaluationResults into a deep dataframe and computes the mean\n    score.\n\n    result:\n        result_df: pd.DataFrame representing all the evaluation results\n        mean_df: pd.DataFrame of average scores groupby names\n    \"\"\"\n    try:\n        import pandas as pd","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/notebook_utils.py#L19-L55","documentation":"While aggregating retrieval metrics, get_retrieval_results_df iterates metric_keys and requires each key to be a column of the per-retriever DataFrame built from RetrievalEvalResult.metric_vals_dict. If a requested key never appears in those dicts, the column is missing and the function raises ValueError naming the offending key.","triggerScenarios":"Passing metric_keys=['mrr'] when results were produced only with a HitRate metric (metric_vals_dict lacks 'mrr'); requesting custom metric names that were never computed; mixing metric sets across retrievers so some columns are absent.","commonSituations":"RetrievalEvalMode/metrics changed between runs; default keys ['hit_rate','mrr'] assumed but only one metric was passed to RetrieverEvaluator(metrics=[...]); copy-pasted metric_keys from another project.","solutions":["Align metric_keys with the metrics you actually computed: RetrieverEvaluator(metrics=['hit_rate'], ...)","Pass metric_keys explicitly derived from results: metric_keys = list(results[0][0].metric_vals_dict.keys())","Compute both default metrics (include mrr) or drop the key you don't have"],"exampleFix":"# before\nevaluator = RetrieverEvaluator(metric=[\"hit_rate\"], ...)\ndf = get_retrieval_results_df(names, results)  # default keys include 'mrr' -> raises\n\n# after\ndf = get_retrieval_results_df(names, results, metric_keys=[\"hit_rate\"])  # or compute mrr too","handlingStrategy":"validation","validationCode":"available_keys = set()\nfor eval_results in results_arr:\n    for er in eval_results:\n        available_keys.update(er.metric_vals_dict.keys())\nsafe_keys = [k for k in metric_keys if k in available_keys] if metric_keys else sorted(available_keys)\ndf = get_retrieval_results_df(names, results_arr, metric_keys=safe_keys)","typeGuard":null,"tryCatchPattern":"try:\n    df = get_retrieval_results_df(names, results, metric_keys=keys)\nexcept ValueError as e:\n    if \"not in results_df\" in str(e):\n        missing = str(e).split('Metric key ')[-1].split(' not')[0]\n        logger.warning(\"metric %s was not computed; rerun RetrieverEvaluator with it\", missing)\n    raise","preventionTips":["Derive metric_keys from the results' own metric_vals_dict instead of hardcoding","Keep RetrieverEvaluator(metrics=[...]) and display metric_keys in one shared constant","Assert computed-vs-requested metric sets match in eval test suites"],"tags":["validation","evaluation","retrieval-metrics","pandas"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}