open-mmlab/mmdetection · error · KeyError

{metric} is not in results

Error message

{metric} is not in results

What it means

Thrown by OV-COCOMetric.compute_metrics when the requested metric name (e.g. 'bbox', 'segm', 'proposal') has no corresponding entry in result_files, meaning results for that prediction type were never produced during evaluation. It signals a mismatch between the metrics configured on the metric object and the prediction formats actually accumulated by the model/dataset pipeline.

Source

Thrown at mmdet/evaluation/metrics/ov_coco_metric.py:90

            logger.info(f'Evaluating {metric}...')

            # TODO: May refactor fast_eval_recall to an independent metric?
            # fast eval recall
            if metric == 'proposal_fast':
                ar = self.fast_eval_recall(
                    preds, self.proposal_nums, self.iou_thrs, logger=logger)
                log_msg = []
                for i, num in enumerate(self.proposal_nums):
                    eval_results[f'AR@{num}'] = ar[i]
                    log_msg.append(f'\nAR@{num}\t{ar[i]:.4f}')
                log_msg = ''.join(log_msg)
                logger.info(log_msg)
                continue

            # evaluate proposal, bbox and segm
            iou_type = 'bbox' if metric == 'proposal' else metric
            if metric not in result_files:
                raise KeyError(f'{metric} is not in results')
            try:
                predictions = load(result_files[metric])
                if iou_type == 'segm':
                    # Refer to https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py#L331  # noqa
                    # When evaluating mask AP, if the results contain bbox,
                    # cocoapi will use the box area instead of the mask area
                    # for calculating the instance area. Though the overall AP
                    # is not affected, this leads to different
                    # small/medium/large mask AP results.
                    for x in predictions:
                        x.pop('bbox')
                coco_dt = self._coco_api.loadRes(predictions)

            except IndexError:
                logger.error(
                    'The testing results of the whole dataset is empty.')
                break

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Ensure the model actually produces predictions of the type requested (add mask head for 'segm', or remove 'segm' from metric list)
2. If only exporting predictions, set format_only=True and remove unsupported metric names from metric=[...]
3. Check dataset_meta and GT annotation types: mask metrics require 'masks' or 'segmentations' in GT annotations; add them or drop the metric

Example fix

# before
val_evaluator = dict(type='OVCocoMetric', metric=['bbox', 'segm'])
# after (bbox-only model)
val_evaluator = dict(type='OVCocoMetric', metric=['bbox'])

allowed = set(evaluator.metric) & {'bbox','segm','proposal'}
# verify the dataset/model produce those prediction types before evaluation
assert allowed == set(evaluator.metric), f'metrics without results: {set(evaluator.metric)-allowed}'

def has_metric_result(result_files: dict, metric: str) -> bool:
    return isinstance(result_files, dict) and metric in result_files

try:
    metrics = evaluator.compute_metrics(result_files)
except KeyError as e:
    logging.warning('skipping unavailable metric: %s', e); metrics = {}

When it happens

Trigger: Calling compute_metrics (or engine.train()/test()) with metric=['bbox','segm'] when the model only outputs one prediction type, or when format_only=True skips generating a results file for a requested metric; also when results2json-style conversion omits a key for the configured metric.

Common situations: Configuring OpenVocabCocoMetric with 'segm' for a detection-only (no mask) model; using a test pipeline that drops mask predictions; copy-pasting a COCO instance-seg config for a bbox-only model.

Related errors

• metric item "{metric_item}" is not supported
• metric should be one of 'recall', 'mAP', but got {metric}.
• Invalid text mode "{self.text_mode}".
• The type of frame_range must be int or list.
• results does not contain masks.

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/a2621a524503aeeb. Report an issue: GitHub.