open-mmlab/mmdetection · error · KeyError

metric should be 'youtube_vis_ap', but got {metric}.

Error message

metric should be 'youtube_vis_ap', but got {metric}.

What it means

YouTubeVisMetric only supports the single metric name 'youtube_vis_ap'; any other entry in the metric list raises this KeyError during construction. The argument is normalized to a list, then each item validated against allowed_metrics.

Source

Thrown at mmdet/evaluation/metrics/youtube_vis_metric.py:60

    """

    default_prefix: Optional[str] = 'youtube_vis'

    def __init__(self,
                 metric: Union[str, List[str]] = 'youtube_vis_ap',
                 metric_items: Optional[Sequence[str]] = None,
                 outfile_prefix: Optional[str] = None,
                 collect_device: str = 'cpu',
                 prefix: Optional[str] = None,
                 format_only: bool = False) -> None:
        super().__init__(collect_device=collect_device, prefix=prefix)
        # vis evaluation metrics
        self.metrics = metric if isinstance(metric, list) else [metric]
        self.format_only = format_only
        allowed_metrics = ['youtube_vis_ap']
        for metric in self.metrics:
            if metric not in allowed_metrics:
                raise KeyError(
                    f"metric should be 'youtube_vis_ap', but got {metric}.")

        self.metric_items = metric_items
        self.outfile_prefix = outfile_prefix
        self.per_video_res = []
        self.categories = []
        self._vis_meta_info = defaultdict(list)  # record video and image infos

    def process_video(self, data_samples):

        video_length = len(data_samples)
        for frame_id in range(video_length):
            result = dict()
            img_data_sample = data_samples[frame_id].to_dict()
            pred = img_data_sample['pred_track_instances']
            video_id = img_data_sample['video_id']

            result['img_id'] = img_data_sample['img_id']

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Set metric='youtube_vis_ap' exactly (or ['youtube_vis_ap'])
2. If only exporting predictions, use format_only=True with outfile_prefix and keep metric='youtube_vis_ap'

Example fix

# before
val_evaluator = dict(type='YouTubeVisMetric', metric='mAP')
# after
val_evaluator = dict(type='YouTubeVisMetric', metric='youtube_vis_ap')

assert all(m == 'youtube_vis_ap' for m in (metric if isinstance(metric, list) else [metric]))

def is_valid_ytvis_metric(metric) -> bool:
    items = metric if isinstance(metric, list) else [metric]
    return all(m == 'youtube_vis_ap' for m in items)

When it happens

Trigger: Passing metric='AP', metric=['mAP'], or metric='youtube_vis' to YouTubeVisMetric; also setting format_only=True but still listing an invalid metric name.

Common situations: Copy-pasting CocoMetric-style metric lists ('bbox','segm') into a video instance segmentation config; typos in the long metric name.

Related errors

• Invalid text mode "{self.text_mode}".
• The type of frame_range must be int or list.
• results does not contain masks.
• Not supported YouTubeVIS datasetversion: {dataset_version}
• {metric} is not in results

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/3ac886ca3ff53a78. Report an issue: GitHub.