open-mmlab/mmdetection · error · KeyError

metric should be one of 'recall', 'mAP', but got {metric}.

Error message

metric should be one of 'recall', 'mAP', but got {metric}.

What it means

VOCMetric only implements two metrics, 'recall' (proposal recall) and 'mAP'; any other string raises this KeyError in __init__. If metric is a non-str iterable it must have exactly one element, which is then used.

Source

Thrown at mmdet/evaluation/metrics/voc_metric.py:64

    def __init__(self,
                 iou_thrs: Union[float, List[float]] = 0.5,
                 scale_ranges: Optional[List[tuple]] = None,
                 metric: Union[str, List[str]] = 'mAP',
                 proposal_nums: Sequence[int] = (100, 300, 1000),
                 eval_mode: str = '11points',
                 collect_device: str = 'cpu',
                 prefix: Optional[str] = None) -> None:
        super().__init__(collect_device=collect_device, prefix=prefix)
        self.iou_thrs = [iou_thrs] if isinstance(iou_thrs, float) \
            else iou_thrs
        self.scale_ranges = scale_ranges
        # voc evaluation metrics
        if not isinstance(metric, str):
            assert len(metric) == 1
            metric = metric[0]
        allowed_metrics = ['recall', 'mAP']
        if metric not in allowed_metrics:
            raise KeyError(
                f"metric should be one of 'recall', 'mAP', but got {metric}.")
        self.metric = metric
        self.proposal_nums = proposal_nums
        assert eval_mode in ['area', '11points'], \
            'Unrecognized mode, only "area" and "11points" are supported'
        self.eval_mode = eval_mode

    # TODO: data_batch is no longer needed, consider adjusting the
    #  parameter position
    def process(self, data_batch: dict, data_samples: Sequence[dict]) -> None:
        """Process one batch of data samples and predictions. The processed
        results should be stored in ``self.results``, which will be used to
        compute the metrics when all batches have been processed.

        Args:
            data_batch (dict): A batch of data from the dataloader.
            data_samples (Sequence[dict]): A batch of data samples that
                contain annotations and predictions.

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Use exactly one of 'mAP' or 'recall' (case-sensitive), e.g. metric='mAP'
2. If a list is used it must contain exactly one of those strings
3. Choose eval_mode ('area' or '11points') separately per VOC protocol, not via the metric name

Example fix

# before
val_evaluator = dict(type='VOCMetric', metric=['AP50', 'mAP'], eval_mode='11points')
# after
val_evaluator = dict(type='VOCMetric', metric='mAP', eval_mode='11points')

assert metric in ('recall', 'mAP') or (not isinstance(metric, str) and len(metric) == 1 and metric[0] in ('recall', 'mAP'))

def is_valid_voc_metric(metric) -> bool:
    if isinstance(metric, (list, tuple)):
        return len(metric) == 1 and metric[0] in ('recall', 'mAP')
    return metric in ('recall', 'mAP')

When it happens

Trigger: Passing metric='AP50', metric=['mAP','recall'] (list of two, fails the len==1 assert), or metric='map' (lowercase) to VOCMetric.

Common situations: Confusing VOC 11-point/AP50 terminology with the two supported modes; passing a list where only a single metric is allowed; copy-pasting from CocoMetric configs that use lists.

Related errors

• {metric} is not in results
• Pascal VOC2007 uses `11points` as default evaluate mode, but
• Pascal VOC2012 uses `area` as default evaluate mode, but you
• Invalid text mode "{self.text_mode}".
• The type of frame_range must be int or list.

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/dd355cf849079b4f. Report an issue: GitHub.