What causes "In the image with ID {} segment with ID {} has unknown category_id {}." in open-mmlab/mmdetection?

Verify all category_id values in predictions exist in the COCO panoptic categories used by the metric

open-mmlab/mmdetection · error · KeyError

In the image with ID {} segment with ID {} has unknown categ

Error message

In the image with ID {} segment with ID {} has unknown category_id {}.

What it means

Every prediction segment's category_id in segments_info must exist in the dataset categories dict used for PQ. An unknown category_id raises KeyError with the image and segment id.

Source

Thrown at mmdet/evaluation/functional/panoptic_utils.py:98

        gt_segms = {el['id']: el for el in gt_ann['segments_info']}
        pred_segms = {el['id']: el for el in pred_ann['segments_info']}

        # predicted segments area calculation + prediction sanity checks
        pred_labels_set = set(el['id'] for el in pred_ann['segments_info'])
        labels, labels_cnt = np.unique(pan_pred, return_counts=True)
        for label, label_cnt in zip(labels, labels_cnt):
            if label not in pred_segms:
                if label == VOID:
                    continue
                raise KeyError(
                    'In the image with ID {} segment with ID {} is '
                    'presented in PNG and not presented in JSON.'.format(
                        gt_ann['image_id'], label))
            pred_segms[label]['area'] = label_cnt
            pred_labels_set.remove(label)
            if pred_segms[label]['category_id'] not in categories:
                raise KeyError(
                    'In the image with ID {} segment with ID {} has '
                    'unknown category_id {}.'.format(
                        gt_ann['image_id'], label,
                        pred_segms[label]['category_id']))
        if len(pred_labels_set) != 0:
            raise KeyError(
                'In the image with ID {} the following segment IDs {} '
                'are presented in JSON and not presented in PNG.'.format(
                    gt_ann['image_id'], list(pred_labels_set)))

        # confusion matrix calculation
        pan_gt_pred = pan_gt.astype(np.uint64) * OFFSET + pan_pred.astype(
            np.uint64)
        gt_pred_map = {}
        labels, labels_cnt = np.unique(pan_gt_pred, return_counts=True)
        for label, intersection in zip(labels, labels_cnt):
            gt_id = label // OFFSET
            pred_id = label % OFFSET

View on GitHub (pinned to cfd5d3a985)

Solutions

Verify all category_id values in predictions exist in the COCO panoptic categories used by the metric
Fix label remapping (e.g. CLASSES order, offset for stuff classes) in postprocessing
Re-inspect dataset meta['classes'] / categories file passed to pq_compute_single_core

Example fix

# before
segments_info = [{'id': 1, 'category_id': 999, ...}]
# after
assert seg['category_id'] in categories for seg in segments_info  # remap before dump

Defensive patterns

Strategy: validation

Validate before calling

valid_cats = set(categories)  # or category ids
assert all(s['category_id'] in valid_cats for s in segments_info), 'unknown category_id in predictions'

Try / catch

try:
    pq_compute_single_core(...)
except KeyError as e:
    if 'unknown category_id' in str(e):
        raise ValueError('Remap prediction category ids to the eval dataset') from e
    raise

Prevention

Remap model class indices to dataset category ids in postprocess
Validate category ids against the categories dict before eval

When it happens

Trigger: Predictions containing category ids outside the dataset's thing/stuff category map (e.g. id from a different label-space or off-by-one after remapping).

Common situations: Model trained on a different number of classes than the eval dataset; category remapping bugs in converters; using continuous ids when eval expects disjoint thing/stuff ids.

Related errors

AI-assisted analysis of open-mmlab/mmdetection@cfd5d3a985 (2026-08-27). Data as JSON: /api/errors/3ffd97b4eabd4397. Report an issue: GitHub.