{"record":{"id":"7865ace7ce659ea8","repo":"facebookresearch/detectron2","slug":"unknown-segmentation-type-type-segmentation","errorCode":null,"errorMessage":"Unknown segmentation type {type(segmentation)}!","messagePattern":"Unknown segmentation type (.+?)!","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"detectron2/data/datasets/coco.py","lineNumber":387,"sourceCode":"                bbox = bbox.tolist()\n            if len(bbox) not in [4, 5]:\n                raise ValueError(f\"bbox has to has length 4 or 5. Got {bbox}.\")\n            from_bbox_mode = annotation[\"bbox_mode\"]\n            to_bbox_mode = BoxMode.XYWH_ABS if len(bbox) == 4 else BoxMode.XYWHA_ABS\n            bbox = BoxMode.convert(bbox, from_bbox_mode, to_bbox_mode)\n\n            # COCO requirement: instance area\n            if \"segmentation\" in annotation:\n                # Computing areas for instances by counting the pixels\n                segmentation = annotation[\"segmentation\"]\n                # TODO: check segmentation type: RLE, BinaryMask or Polygon\n                if isinstance(segmentation, list):\n                    polygons = PolygonMasks([segmentation])\n                    area = polygons.area()[0].item()\n                elif isinstance(segmentation, dict):  # RLE\n                    area = mask_util.area(segmentation).item()\n                else:\n                    raise TypeError(f\"Unknown segmentation type {type(segmentation)}!\")\n            else:\n                # Computing areas using bounding boxes\n                if to_bbox_mode == BoxMode.XYWH_ABS:\n                    bbox_xy = BoxMode.convert(bbox, to_bbox_mode, BoxMode.XYXY_ABS)\n                    area = Boxes([bbox_xy]).area()[0].item()\n                else:\n                    area = RotatedBoxes([bbox]).area()[0].item()\n\n            if \"keypoints\" in annotation:\n                keypoints = annotation[\"keypoints\"]  # list[int]\n                for idx, v in enumerate(keypoints):\n                    if idx % 3 != 2:\n                        # COCO's segmentation coordinates are floating points in [0, H or W],\n                        # but keypoint coordinates are integers in [0, H-1 or W-1]\n                        # For COCO format consistency we substract 0.5\n                        # https://github.com/facebookresearch/detectron2/pull/175#issuecomment-551202163\n                        keypoints[idx] = v - 0.5\n                if \"num_keypoints\" in annotation:","sourceCodeStart":369,"sourceCodeEnd":405,"githubUrl":"https://github.com/facebookresearch/detectron2/blob/a2f4a8771ab77e8411c26b27f24f9489a28a2453/detectron2/data/datasets/coco.py#L369-L405","documentation":"When converting annotations to COCO dict format and computing area from segmentation, only list (polygon) and dict (RLE) segmentations are supported. Any other type (e.g. a numpy array mask, bytes, torch.Tensor, None, str) raises TypeError.","triggerScenarios":"Calling convert_to_coco_dict with annotation['segmentation'] as a binary mask ndarray or torch tensor; encoding masks yourself as encoded-RLE bytes instead of the dict form {'size':..., 'counts':...}.","commonSituations":"See trigger scenarios.","solutions":["Convert masks to RLE dict with pycocotools: mask_util.encode(np.asfortranarray(mask.astype(np.uint8))) and keep the dict","Or convert mask to polygons via cv2.findContours and store as list[list[float]]","Pass segmentation=None/omit when only boxes are available"],"exampleFix":"# before\nann['segmentation'] = mask  # np.ndarray HxW\n# after\nfrom pycocotools import mask as mask_util\nrle = mask_util.encode(np.asfortranarray(mask.astype(np.uint8)))\nrle['counts'] = rle['counts'].decode('utf-8')\nann['segmentation'] = rle","handlingStrategy":"type-guard","validationCode":"seg = annotation.get('segmentation')\nassert seg is None or isinstance(seg, (list, dict)), \\\n    f'segmentation must be polygon list or RLE dict, got {type(seg)}'","typeGuard":"def is_coco_segmentation(s) -> bool:\n    return s is None or isinstance(s, (list, dict))","tryCatchPattern":null,"preventionTips":["Encode binary masks to RLE dicts with pycocotools before export","Remember counts must be a str for JSON serialization","Keep Tensor/ndarray masks out of annotation dicts destined for COCO"],"tags":["detectron2","coco-export","segmentation","rle","typeerror"],"backgroundTag":"unsupported-segmentation-format","analyzedSha":"a2f4a8771ab77e8411c26b27f24f9489a28a2453","analyzedAt":"2026-08-27T12:08:21.260Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}