open-mmlab/mmdetection · error · KeyError

metric item "{metric_item}" is not supported

Error message

metric item "{metric_item}" is not supported

What it means

CocoMetric.compute_metrics validates user-supplied `metric_items` (specific summary items like 'mAP', 'AP50', 'AR@100') against the fixed coco_metric_names mapping. An item not in that mapping cannot be extracted from the COCOeval stats array, so a KeyError is raised. metric_items only filters which numbers are logged, not which evals run.

Source

Thrown at mmdet/evaluation/metrics/coco_metric.py:498

            coco_metric_names = {
                'mAP': 0,
                'mAP_50': 1,
                'mAP_75': 2,
                'mAP_s': 3,
                'mAP_m': 4,
                'mAP_l': 5,
                'AR@100': 6,
                'AR@300': 7,
                'AR@1000': 8,
                'AR_s@1000': 9,
                'AR_m@1000': 10,
                'AR_l@1000': 11
            }
            metric_items = self.metric_items
            if metric_items is not None:
                for metric_item in metric_items:
                    if metric_item not in coco_metric_names:
                        raise KeyError(
                            f'metric item "{metric_item}" is not supported')

            if metric == 'proposal':
                coco_eval.params.useCats = 0
                coco_eval.evaluate()
                coco_eval.accumulate()
                coco_eval.summarize()
                if metric_items is None:
                    metric_items = [
                        'AR@100', 'AR@300', 'AR@1000', 'AR_s@1000',
                        'AR_m@1000', 'AR_l@1000'
                    ]

                for item in metric_items:
                    val = float(
                        f'{coco_eval.stats[coco_metric_names[item]]:.3f}')
                    eval_results[item] = val
            else:

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Use only supported items: 'mAP','mAP_50','mAP_75','mAP_s','mAP_m','mAP_l','AR@100','AR@300','AR@1000','AR_s@1000','AR_m@1000','AR_l@1000'
2. Match the exact spelling/case from coco_metric_names in the source
3. For custom metrics, subclass CocoMetric and extend compute_metrics instead of metric_items

Example fix

# before
metric_items=['mAP','F1']
# after
metric_items=['mAP','mAP_50','AR@100']

COCO_ITEMS = {'mAP','mAP_50','mAP_75','mAP_s','mAP_m','mAP_l',
               'AR@100','AR@300','AR@1000','AR_s@1000','AR_m@1000','AR_l@1000'}
items = cfg['val_evaluator'].get('metric_items')
if items:
    bad = [i for i in items if i not in COCO_ITEMS]
    assert not bad, f'unsupported metric_items: {bad}'

def is_supported_metric_item(item: str) -> bool:
    return item in {'mAP','mAP_50','mAP_75','mAP_s','mAP_m','mAP_l',
                    'AR@100','AR@300','AR@1000','AR_s@1000','AR_m@1000','AR_l@1000'}

When it happens

Trigger: Passing CocoMetric(metric='bbox', metric_items=['mAP75','APs','F1']) where 'F1' is not a coco_metric_names key; using lowercase names like 'ap50' instead of 'AP50'; requesting combined names like 'AR@1000' spelled differently from the mapping ('AR_l@1000').

Common situations: Trying to log extra derived metrics (F1, recall at custom IoU) that COCOeval does not compute; typo/case mismatch in metric item names copied from a paper or another codebase.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• metric should be one of 'bbox', 'segm', 'proposal', 'proposa
• {metric} is not in results
• metric should be one of 'MR', 'AP', 'JI',but got {metric}.
• LoadImageFromFile is not found in the test pipeline
• Visualization needs the "visualizer" termdefined in the conf

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/947bb0afe79c81f0. Report an issue: GitHub.