open-mmlab/mmdetection · error · KeyError

metric item "{metric_item}" is not supported

Error message

metric item "{metric_item}" is not supported

What it means

YouTubeVisMetric.compute_metrics validates optional metric_items against the coco_metric_names table (mAP, mAP_50, mAP_75, mAP_s, mAP_m, mAP_l, AR@100, AR_l@100, ...). Unknown or misspelled items raise this KeyError when evaluation runs.

Source

Thrown at mmdet/evaluation/metrics/youtube_vis_metric.py:163

        coco_metric_names = {
            'mAP': 0,
            'mAP_50': 1,
            'mAP_75': 2,
            'mAP_s': 3,
            'mAP_m': 4,
            'mAP_l': 5,
            'AR@1': 6,
            'AR@10': 7,
            'AR@100': 8,
            'AR_s@100': 9,
            'AR_m@100': 10,
            'AR_l@100': 11
        }
        metric_items = self.metric_items
        if metric_items is not None:
            for metric_item in metric_items:
                if metric_item not in coco_metric_names:
                    raise KeyError(
                        f'metric item "{metric_item}" is not supported')

        if metric_items is None:
            metric_items = [
                'mAP', 'mAP_50', 'mAP_75', 'mAP_s', 'mAP_m', 'mAP_l'
            ]
        for metric_item in metric_items:
            key = f'{metric}_{metric_item}'
            val = float(
                f'{ytvisEval.stats[coco_metric_names[metric_item]]:.3f}')
            eval_results[key] = val

        return eval_results

    def format_gts(self, gts: Tuple[List]) -> dict:
        """Gather all ground-truth from self.results."""
        self.categories = [
            dict(id=id + 1, name=name)

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Use only names present in the metric's coco_metric_names dict (mAP, mAP_50, mAP_75, mAP_s, mAP_m, mAP_l, AR@100, AR_l@100, etc.)
2. Omit metric_items to get the default set (mAP, mAP_50, mAP_75, mAP_s, mAP_m, mAP_l)
3. Inspect the dict at the raise site in your installed version for exact keys

Example fix

# before
val_evaluator = dict(type='YouTubeVisMetric', metric='youtube_vis_ap', metric_items=['AP50'])
# after
val_evaluator = dict(type='YouTubeVisMetric', metric='youtube_vis_ap', metric_items=['mAP_50'])

SUPPORTED = {'mAP','mAP_50','mAP_75','mAP_s','mAP_m','mAP_l','AR@100','AR@300','AR@1000','AR_s@1000','AR_m@1000','AR_l@1000'}
assert metric_items is None or set(metric_items) <= SUPPORTED

def valid_vis_items(items) -> bool:
    S = {'mAP','mAP_50','mAP_75','mAP_s','mAP_m','mAP_l','AR@100','AR@300','AR@1000','AR_s@1000','AR_m@1000','AR_l@1000'}
    return items is None or set(items) <= S

try:
    evaluator.compute_metrics(results)
except KeyError as e:
    logging.error('invalid metric_items: %s', e); raise

When it happens

Trigger: Passing metric_items=['AP50','AP','AR'] or any name not in the table; evaluation-time failure even though construction succeeded.

Common situations: Reusing metric_items lists written for CocoMetric AR@1000 variants that the YouTube-VIS table omits; hand-writing item names from papers.

Related errors

• metric item "{metric_item}" is not supported
• Not supported YouTubeVIS datasetversion: {dataset_version}
• {metric} is not in results
• metric should be one of 'recall', 'mAP', but got {metric}.
• metric should be 'youtube_vis_ap', but got {metric}.

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/0b9afe3f613c87ff. Report an issue: GitHub.