open-mmlab/mmdetection · error · KeyError

metric {metric} is not supported.

Error message

metric {metric} is not supported.

What it means

MOTChallengeMetric.__init__ raises KeyError when a requested metric name is not in self.allowed_metrics (the MOT trackeval-supported set such as 'mot Challenge' metrics: mota, motp, idf1, hota, etc., built earlier in __init__). This is a value-validation error for the metric list contents.

Source

Thrown at mmdet/evaluation/metrics/mot_challenge_metric.py:107

                 collect_device: str = 'cpu',
                 prefix: Optional[str] = None) -> None:
        super().__init__(collect_device=collect_device, prefix=prefix)
        if trackeval is None:
            raise RuntimeError(
                'trackeval is not installed,'
                'please install it by: pip install'
                'git+https://github.com/JonathonLuiten/TrackEval.git'
                'trackeval need low version numpy, please install it'
                'by: pip install -U numpy==1.23.5')
        if isinstance(metric, list):
            metrics = metric
        elif isinstance(metric, str):
            metrics = [metric]
        else:
            raise TypeError('metric must be a list or a str.')
        for metric in metrics:
            if metric not in self.allowed_metrics:
                raise KeyError(f'metric {metric} is not supported.')
        self.metrics = metrics
        self.format_only = format_only
        if self.format_only:
            assert outfile_prefix is not None, 'outfile_prefix must be not'
            'None when format_only is True, otherwise the result files will'
            'be saved to a temp directory which will be cleaned up at the end.'
        self.use_postprocess = use_postprocess
        self.postprocess_tracklet_cfg = postprocess_tracklet_cfg.copy()
        self.postprocess_tracklet_methods = [
            TASK_UTILS.build(cfg) for cfg in self.postprocess_tracklet_cfg
        ]
        assert benchmark in self.allowed_benchmarks
        self.benchmark = benchmark
        self.track_iou_thr = track_iou_thr
        self.tmp_dir = tempfile.TemporaryDirectory()
        self.tmp_dir.name = get_tmpdir()
        self.seq_info = defaultdict(
            lambda: dict(seq_length=-1, gt_tracks=[], pred_tracks=[]))

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Use only MOT-supported metric names, e.g. metric=['mota', 'motp', 'idf1', 'hota', 'recall', 'precision'] — check allowed_metrics in mot_challenge_metric.py for the exact set
2. Fix case/typos: metric names are lowercase ('mota', not 'MOTA')
3. Remove detection-only metrics like 'bbox'/'segm' from the MOT evaluator config, or switch to CocoMetric if you meant detection evaluation

Example fix

// before
val_evaluator=dict(type='MOTChallengeMetric', metric=['bbox'])
// after
val_evaluator=dict(type='MOTChallengeMetric', metric=['mota', 'idf1'])

from mmdet.evaluation.metrics.mot_challenge_metric import MOTChallengeMetric
allowed = MOTChallengeMetric.allowed_metrics if hasattr(MOTChallengeMetric, 'allowed_metrics') else \
          {'mota', 'motp', 'idf1', 'hota', 'recall', 'precision'}
metrics = metric if isinstance(metric, list) else [metric]
bad = [m for m in metrics if m not in allowed]
assert not bad, f'Unsupported MOT metrics: {bad}; allowed: {sorted(allowed)}'

def is_valid_mot_metric_names(metric, allowed: set) -> bool:
    vals = metric if isinstance(metric, (list, tuple)) else [metric]
    return all(isinstance(m, str) and m in allowed for m in vals)

try:
    m = MOTChallengeMetric(metric=metric)
except KeyError as e:
    raise ValueError(f'Unsupported MOT metric in {metric}: {e}') from e

When it happens

Trigger: Passing metric=['map'] or metric='accuracy' to MOTChallengeMetric — names valid for detection metrics but not supported by the MOT evaluator's allowed_metrics set.

Common situations: Reusing a CocoMetric config block (metric='bbox') as the val_evaluator for a MOT config; typos/case errors like 'MOTA' vs 'mota'; assuming detection metrics are available in tracking evaluation.

Related errors

• metric must be a list or a str.
• metric should be one of 'bbox', 'segm', 'proposal', 'proposa
• trackeval is not installed,please install it by: pip install
• module must be a str or a list.
• LoadImageFromFile is not found in the test pipeline

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/36e867043663d7c0. Report an issue: GitHub.