open-mmlab/mmdetection · warning

Pascal VOC2007 uses `11points` as default evaluate mode, but

Error message

Pascal VOC2007 uses `11points` as default evaluate mode, but you are using {self.eval_mode}.

What it means

VOCMetric.compute_metrics() warns that VOC2007's standard evaluation protocol uses 11-point interpolation mAP ('11points'), but the metric is configured with a different eval mode. Scores will not match published VOC2007 benchmarks.

Source

Thrown at mmdet/evaluation/metrics/voc_metric.py:128

        """Compute the metrics from processed results.

        Args:
            results (list): The processed results of each batch.

        Returns:
            dict: The computed metrics. The keys are the names of the metrics,
            and the values are corresponding results.
        """
        logger: MMLogger = MMLogger.get_current_instance()
        gts, preds = zip(*results)
        eval_results = OrderedDict()
        if self.metric == 'mAP':
            assert isinstance(self.iou_thrs, list)
            dataset_type = self.dataset_meta.get('dataset_type')
            if dataset_type in ['VOC2007', 'VOC2012']:
                dataset_name = 'voc'
                if dataset_type == 'VOC2007' and self.eval_mode != '11points':
                    warnings.warn('Pascal VOC2007 uses `11points` as default '
                                  'evaluate mode, but you are using '
                                  f'{self.eval_mode}.')
                elif dataset_type == 'VOC2012' and self.eval_mode != 'area':
                    warnings.warn('Pascal VOC2012 uses `area` as default '
                                  'evaluate mode, but you are using '
                                  f'{self.eval_mode}.')
            else:
                dataset_name = self.dataset_meta['classes']

            mean_aps = []
            for iou_thr in self.iou_thrs:
                logger.info(f'\n{"-" * 15}iou_thr: {iou_thr}{"-" * 15}')
                # Follow the official implementation,
                # http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCdevkit_18-May-2011.tar
                # we should use the legacy coordinate system in mmdet 1.x,
                # which means w, h should be computed as 'x2 - x1 + 1` and
                # `y2 - y1 + 1`
                mean_ap, _ = eval_map(

View on GitHub (pinned to cfd5d3a985)

Solutions

1. Set eval_mode='11points' in the VOCMetric val_evaluator config for VOC2007
2. Or intentionally accept the non-standard mode if comparing against area-mode baselines
3. Double-check dataset_type in the dataset config matches the actual year of data

Example fix

# before
val_evaluator = dict(type='VOCMetric', metric='mAP', eval_mode='area')
# after
val_evaluator = dict(type='VOCMetric', metric='mAP', eval_mode='11points')  # VOC2007 default

eval_mode = '11points' if dataset_meta['dataset_type'] == 'VOC2007' else eval_mode
assert not (dataset_meta['dataset_type'] == 'VOC2007' and eval_mode != '11points')

When it happens

Trigger: Setting metric=['mAP'] with eval_mode not equal to '11points' while dataset_meta['dataset_type'] == 'VOC2007' — e.g. passing eval_mode='area' in the VOCMetric config.

Common situations: Copy-pasting a VOC2012 config for VOC2007 data, or manually setting iou_thrs/eval_mode without matching the dataset year.

Related errors

• Pascal VOC2012 uses `area` as default evaluate mode, but you
• metric should be one of 'recall', 'mAP', but got {metric}.
• Invalid text mode "{self.text_mode}".
• The type of frame_range must be int or list.
• results does not contain masks.

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/b573d397e011f0c5. Report an issue: GitHub.