open-mmlab/mmdetection · error · KeyError

metric should be one of 'bbox', 'segm', 'proposal', 'proposa

Error message

metric should be one of 'bbox', 'segm', 'proposal', 'proposal_fast', but got {metric}.

What it means

CocoMetric.__init__ validates the `metric` argument against the allowed COCO evaluation types: 'bbox', 'segm', 'proposal', 'proposal_fast'. Any other string (or a list containing one) raises this KeyError because mmdetection has no COCO eval implementation for it. This guards against typos and unsupported eval types at construction time, before any training/eval loop starts.

Source

Thrown at mmdet/evaluation/metrics/coco_metric.py:91

                 classwise: bool = False,
                 proposal_nums: Sequence[int] = (100, 300, 1000),
                 iou_thrs: Optional[Union[float, Sequence[float]]] = None,
                 metric_items: Optional[Sequence[str]] = None,
                 format_only: bool = False,
                 outfile_prefix: Optional[str] = None,
                 file_client_args: dict = None,
                 backend_args: dict = None,
                 collect_device: str = 'cpu',
                 prefix: Optional[str] = None,
                 sort_categories: bool = False,
                 use_mp_eval: bool = False) -> None:
        super().__init__(collect_device=collect_device, prefix=prefix)
        # coco evaluation metrics
        self.metrics = metric if isinstance(metric, list) else [metric]
        allowed_metrics = ['bbox', 'segm', 'proposal', 'proposal_fast']
        for metric in self.metrics:
            if metric not in allowed_metrics:
                raise KeyError(
                    "metric should be one of 'bbox', 'segm', 'proposal', "
                    f"'proposal_fast', but got {metric}.")

        # do class wise evaluation, default False
        self.classwise = classwise
        # whether to use multi processing evaluation, default False
        self.use_mp_eval = use_mp_eval

        # proposal_nums used to compute recall or precision.
        self.proposal_nums = list(proposal_nums)

        # iou_thrs used to compute recall or precision.
        if iou_thrs is None:
            iou_thrs = np.linspace(
                .5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
        self.iou_thrs = iou_thrs
        self.metric_items = metric_items
        self.format_only = format_only

View on GitHub (pinned to cfd5d3a985)

Solutions

Set metric to one of 'bbox', 'segm', 'proposal', 'proposal_fast' (or a list of them), e.g. dict(type='CocoMetric', metric=['bbox','segm'])
If you need a different dataset's metrics, use the matching metric class (e.g. CrowdHumanMetric, LVISMetric)
If you only want quick proposal quality, use 'proposal_fast'

Example fix

// before
val_evaluator = dict(type='CocoMetric', metric='AP')
// after
val_evaluator = dict(type='CocoMetric', metric='bbox')

Defensive patterns

Strategy: validation

Validate before calling

ALLOWED = {'bbox','segm','proposal','proposal_fast'}
metrics = cfg['val_evaluator']['metric']
metrics = [metrics] if isinstance(metrics, str) else metrics
bad = [m for m in metrics if m not in ALLOWED]
assert not bad, f'Invalid CocoMetric metric(s): {bad}, allowed: {sorted(ALLOWED)}'

Type guard

from typing import Union, List

def is_valid_coco_metric(m: Union[str, List[str]]) -> bool:
    allowed = {'bbox', 'segm', 'proposal', 'proposal_fast'}
    items = [m] if isinstance(m, str) else m
    return bool(items) and all(x in allowed for x in items)

Prevention

Validate metric names against the class's allowed_metrics before building the evaluator
Use a config linter or unit test that instantiates val_evaluator for every config in the repo
Copy metric names from the official mmdet config of the same task, not from other datasets

When it happens

Trigger: Instantiating CocoMetric(metric='box') / 'bbox-segm' / 'AP' / ['bbox','wrong'], or passing a val_cfg/val_evaluator config in mmdet where the metric name is misspelled or belongs to another dataset's metric class (e.g. CrowdHuman 'MR'/'JI').

Common situations: Copy-pasting an evaluator config from a different task (e.g. a CrowdHuman or LVIS config) into a COCO config; using 'AP' or 'mAP' as a metric name; upgrading configs where older metric aliases existed.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/51716008db42a1da. Report an issue: GitHub.