open-mmlab/mmdetection · error · KeyError

metric {metric} is not supported.

Error message

metric {metric} is not supported.

What it means

ReIDMetrics supports only its allowed_metrics set (currently ['mAP', 'CMC'], plus 'tCMC' in some versions); any other metric name raises this KeyError. The check runs per item after the type check on the metric argument.

Source

Thrown at mmdet/evaluation/metrics/reid_metric.py:46

    allowed_metrics = ['mAP', 'CMC']
    default_prefix: Optional[str] = 'reid-metric'

    def __init__(self,
                 metric: Union[str, Sequence[str]] = 'mAP',
                 metric_options: Optional[dict] = None,
                 collect_device: str = 'cpu',
                 prefix: Optional[str] = None) -> None:
        super().__init__(collect_device, prefix)

        if isinstance(metric, list):
            metrics = metric
        elif isinstance(metric, str):
            metrics = [metric]
        else:
            raise TypeError('metric must be a list or a str.')
        for metric in metrics:
            if metric not in self.allowed_metrics:
                raise KeyError(f'metric {metric} is not supported.')
        self.metrics = metrics

        self.metric_options = metric_options or dict(
            rank_list=[1, 5, 10, 20], max_rank=20)
        for rank in self.metric_options['rank_list']:
            assert 1 <= rank <= self.metric_options['max_rank']

    def process(self, data_batch: dict, data_samples: Sequence[dict]) -> None:
        """Process one batch of data samples and predictions.

        The processed results should be stored in ``self.results``, which will
        be used to compute the metrics when all batches have been processed.

        Args:
            data_batch (dict): A batch of data from the dataloader.
            data_samples (Sequence[dict]): A batch of data samples that
                contain annotations and predictions.
        """

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Use only supported names: 'mAP' and 'CMC' (and 'tCMC' where available) with exact casing
2. Check ReIDMetrics.allowed_metrics in your installed mmdet version for the definitive list
3. Upgrade mmdet if you need an additional metric that exists only in newer versions

Example fix

# before
val_evaluator = dict(type='ReIDMetrics', metric=['mAP', 'R1'])
# after
val_evaluator = dict(type='ReIDMetrics', metric=['mAP', 'CMC'])

from mmdet.evaluation.metrics.reid_metric import ReIDMetrics
bad = set(metric_list) - set(ReIDMetrics.allowed_metrics)
assert not bad, f'unsupported ReID metrics: {bad}'

def is_supported_reid_metric(metric) -> bool:
    return metric in getattr(ReIDMetrics, 'allowed_metrics', {'mAP', 'CMC'})

try:
    m = ReIDMetrics(metric=metric)
except KeyError as e:
    raise ValueError(f'use only {ReIDMetrics.allowed_metrics}: {e}') from e

When it happens

Trigger: Passing metric=['top1', 'recall@k', 'R1'] or a misspelled 'MAP'/'map' (case-sensitive) to ReIDMetrics.

Common situations: Porting metric names from other ReID codebases (torchreid uses 'rank-1', 'mAP'); assuming case-insensitive names; using metric names from newer mmdet versions on older installs.

Related errors

• metric must be a list or a str.
• Since cross entropy is not set, the num_classes will be igno
• Invalid text mode "{self.text_mode}".
• The type of frame_range must be int or list.
• results does not contain masks.

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/09a9ddb114c8e2ce. Report an issue: GitHub.