{"record":{"id":"56b5000943e7a27c","repo":"open-mmlab/mmdetection","slug":"metric-is-not-in-results","errorCode":null,"errorMessage":"{metric} is not in results","messagePattern":"(.+?) is not in results","errorType":"validation","errorClass":"KeyError","httpStatus":null,"severity":"error","filePath":"mmdet/evaluation/metrics/coco_metric.py","lineNumber":450,"sourceCode":"            logger.info(f'Evaluating {metric}...')\n\n            # TODO: May refactor fast_eval_recall to an independent metric?\n            # fast eval recall\n            if metric == 'proposal_fast':\n                ar = self.fast_eval_recall(\n                    preds, self.proposal_nums, self.iou_thrs, logger=logger)\n                log_msg = []\n                for i, num in enumerate(self.proposal_nums):\n                    eval_results[f'AR@{num}'] = ar[i]\n                    log_msg.append(f'\\nAR@{num}\\t{ar[i]:.4f}')\n                log_msg = ''.join(log_msg)\n                logger.info(log_msg)\n                continue\n\n            # evaluate proposal, bbox and segm\n            iou_type = 'bbox' if metric == 'proposal' else metric\n            if metric not in result_files:\n                raise KeyError(f'{metric} is not in results')\n            try:\n                predictions = load(result_files[metric])\n                if iou_type == 'segm':\n                    # Refer to https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py#L331  # noqa\n                    # When evaluating mask AP, if the results contain bbox,\n                    # cocoapi will use the box area instead of the mask area\n                    # for calculating the instance area. Though the overall AP\n                    # is not affected, this leads to different\n                    # small/medium/large mask AP results.\n                    for x in predictions:\n                        x.pop('bbox')\n                coco_dt = self._coco_api.loadRes(predictions)\n\n            except IndexError:\n                logger.error(\n                    'The testing results of the whole dataset is empty.')\n                break\n","sourceCodeStart":432,"sourceCodeEnd":468,"githubUrl":"https://github.com/open-mmlab/mmdetection/blob/cfd5d3a985b0249de009b67d04f37263e11cdf3d/mmdet/evaluation/metrics/coco_metric.py#L432-L468","documentation":"During CocoMetric.compute_metrics, after converting predictions to COCO json files, the code looks up result_files[metric] for each metric being evaluated. If the per-metric result file was never produced (typically because the model returned no results of that type), it raises KeyError '{metric} is not in results'. It means the eval loop requested a metric whose predictions are absent from data_samples.","triggerScenarios":"Setting metric=['bbox','segm'] while the model/detector only outputs detection boxes (no mask head), so no 'segm' result file is generated; or running segm eval on checkpoints trained without a mask branch; also 'proposal' eval when predictions lack proposal fields.","commonSituations":"Reusing a bbox-only config/checkpoint and just adding 'segm' to the evaluator; evaluating an RPN-only model with metric='bbox'; model outputs custom result keys that bypass the bbox2coco/segm2coco mapping.","solutions":["Remove the unsupported metric from the metric list (e.g. drop 'segm' for a bbox-only model)","If you need 'segm', use a config/checkpoint with a mask head (e.g. Mask R-CNN) so predictions contain 'segm' results","Check data_samples contain the expected pred fields before eval (data_sample.pred_instances keys)"],"exampleFix":"# before (bbox-only model)\nval_evaluator = dict(type='CocoMetric', metric=['bbox', 'segm'])\n# after\nval_evaluator = dict(type='CocoMetric', metric='bbox')","handlingStrategy":"validation","validationCode":"requested = {'bbox','segm'} if isinstance(metric, list) else {metric}\navailable = set(next(iter(data_samples)).get('pred_instances', {}).keys()) or set()\n# inspect one sample's keys to see which result types the model emits\nprint('model produces:', available)","typeGuard":"def model_supports(model_output_keys: set, wanted: str) -> bool:\n    mapping = {'bbox': 'bboxes', 'segm': 'masks', 'proposal': 'proposals'}\n    return mapping.get(wanted, 'bboxes') in model_output_keys or wanted == 'proposal_fast'","tryCatchPattern":"try:\n    evaluator.compute_metrics(results)\nexcept KeyError as e:\n    missing = e.args[0].split(' is not in results')[0]\n    print(f'model produced no {missing} predictions; drop it from metric list')","preventionTips":["Match the metric list to the model's heads (no 'segm' without a mask head)","Smoke-test the evaluator on a single batch before launching full eval","After checkpoint changes, re-check that output types still match metric list"],"tags":["mmdetection","coco","metric-mismatch","missing-predictions"],"backgroundTag":"missing-result-key","analyzedSha":"cfd5d3a985b0249de009b67d04f37263e11cdf3d","analyzedAt":"2026-08-27T20:54:20.183Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}