{"record":{"id":"5b36e27d638a6fae","repo":"roboflow/supervision","slug":"evaluating-predictions-with-segmentation-is-not-su","errorCode":null,"errorMessage":"Evaluating predictions with segmentation is not supported.","messagePattern":"Evaluating predictions with segmentation is not supported\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/supervision/metrics/mean_average_precision.py","lineNumber":542,"sourceCode":"        if len(predictions) == 0:\n            predictions_dataset.dataset[\"annotations\"] = []\n            return predictions_dataset\n\n        ids = [pred[\"image_id\"] for pred in predictions]\n\n        # Make sure the image ids from predictions exist in the current dataset.\n        # A plain ``assert`` would be stripped under ``python -O``, so validate\n        # this public-input contract with an explicit exception instead.\n        if not set(ids) <= set(self.get_image_ids()):\n            raise ValueError(\"Results do not correspond to current coco set\")\n\n        # Check if the predictions contain any unsupported keys\n        if \"caption\" in predictions[0]:\n            raise NotImplementedError(\n                \"Evaluating predictions with caption is not supported.\"\n            )\n        elif \"segmentation\" in predictions[0]:\n            raise NotImplementedError(\n                \"Evaluating predictions with segmentation is not supported.\"\n            )\n        elif \"keypoints\" in predictions[0]:\n            raise NotImplementedError(\n                \"Evaluating predictions with keypoints is not supported.\"\n            )\n\n        elif \"bbox\" in predictions[0] and not predictions[0][\"bbox\"] == []:\n            predictions_dataset.dataset[\"categories\"] = copy.deepcopy(\n                self.dataset[\"categories\"]\n            )\n\n            # Prepare fields for every prediction of the given image\n            for idx, pred in enumerate(predictions):\n                x, y, w, h = pred[\"bbox\"]\n                x1, x2, y1, y2 = [x, x + w, y, y + h]\n\n                # Make segmentation from bounding box coordinates","sourceCodeStart":524,"sourceCodeEnd":560,"githubUrl":"https://github.com/roboflow/supervision/blob/7f254d9784d4c37e0f03cd89ddee164c8db099c0/src/supervision/metrics/mean_average_precision.py#L524-L560","documentation":"EvaluationDataset.load_predictions() only implements box-detection results. This NotImplementedError fires when the first prediction dict contains a 'segmentation' key — the COCO segmentation result format (RLE or polygon masks). The library's COCO backend does not evaluate segmentation predictions in this path; mask-based evaluation must instead go through MeanAveragePrecision with metric_target=MetricTarget.MASKS and sv.Detections carrying .mask.","triggerScenarios":"Loading COCO instance-segmentation results files (with 'segmentation' RLE/polygons) into load_predictions; converting a panoptic/segmentation model's output to COCO dict format and feeding it here; mixing detection and segmentation entries where the first entry has a segmentation key.","commonSituations":"Evaluating YOLO-seg / Mask R-CNN outputs saved in official COCO results format; assuming pycocotools-style segm evaluation is available; reusing detection evaluation scripts unchanged for segmentation models.","solutions":["For mask evaluation, use MeanAveragePrecision(metric_target=MetricTarget.MASKS) with Detections whose .mask is populated (segmentation connectors like from_ultralytics on YOLO-seg populate it)","Strip 'segmentation' keys only if you truly want box-only evaluation of those results","Use pycocotools' COCOeval with iouType='segm' for official RLE-based COCO segmentation scoring","Route result files by task before evaluation: check keys of the first entry"],"exampleFix":"# before\nseg_results = [{'image_id': 1, 'category_id': 2,\n                'segmentation': [[...polygon...]], 'score': 0.9}]\ncoco_det = coco_gt.load_predictions(seg_results)\n\n# after (mask mAP via the Detections API)\nmap_ = sv.MeanAveragePrecision(metric_target=sv.MetricTarget.MASKS)\nmap_.update(sv.Detections(xyxy=boxes, mask=masks, class_id=ids,\n                          confidence=confs),\n            sv.Detections(xyxy=gt_boxes, mask=gt_masks, class_id=gt_ids))\nresult = map_.compute()","handlingStrategy":"validation","validationCode":"if predictions and 'segmentation' in predictions[0]:\n    raise TypeError('use MeanAveragePrecision with MetricTarget.MASKS for masks, '\n                    'or pycocotools COCOeval segm; this API is box-only')","typeGuard":"def unsupported_keys(preds: list) -> set:\n    \"\"\"Return COCO task keys this evaluator cannot handle.\"\"\"\n    return {'caption', 'segmentation', 'keypoints'} & (set(preds[0]) if preds else set())","tryCatchPattern":"try:\n    dataset.load_predictions(results)\nexcept NotImplementedError as e:\n    if 'segmentation' in str(e):\n        results = [{k: v for k, v in r.items() if k != 'segmentation'}\n                   for r in results]  # only if box-only eval is intended\n        coco_det = dataset.load_predictions(results)\n    else:\n        raise","preventionTips":["For mask mAP use MetricTarget.MASKS with Detections.mask","Use pycocotools COCOeval(iouType='segm') for official RLE scoring","Split multi-task result files before evaluation"],"tags":["metrics","mean-average-precision","coco","segmentation","not-implemented"],"backgroundTag":null,"analyzedSha":"7f254d9784d4c37e0f03cd89ddee164c8db099c0","analyzedAt":"2026-08-15T05:13:01.950Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}