{"record":{"id":"7112f4f551d3fb8c","repo":"roboflow/supervision","slug":"results-do-not-correspond-to-current-coco-set","errorCode":null,"errorMessage":"Results do not correspond to current coco set","messagePattern":"Results do not correspond to current coco set","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/supervision/metrics/mean_average_precision.py","lineNumber":534,"sourceCode":"        # Create an empty EvaluationDataset object for the predictions\n        predictions_dataset = EvaluationDataset.empty()\n        predictions_dataset.dataset[\"images\"] = list(self.dataset[\"images\"])\n\n        if not isinstance(predictions, list):\n            raise ValueError(\"results must be a list\")\n\n        # Handle empty predictions\n        if len(predictions) == 0:\n            predictions_dataset.dataset[\"annotations\"] = []\n            return predictions_dataset\n\n        ids = [pred[\"image_id\"] for pred in predictions]\n\n        # Make sure the image ids from predictions exist in the current dataset.\n        # A plain ``assert`` would be stripped under ``python -O``, so validate\n        # this public-input contract with an explicit exception instead.\n        if not set(ids) <= set(self.get_image_ids()):\n            raise ValueError(\"Results do not correspond to current coco set\")\n\n        # Check if the predictions contain any unsupported keys\n        if \"caption\" in predictions[0]:\n            raise NotImplementedError(\n                \"Evaluating predictions with caption is not supported.\"\n            )\n        elif \"segmentation\" in predictions[0]:\n            raise NotImplementedError(\n                \"Evaluating predictions with segmentation is not supported.\"\n            )\n        elif \"keypoints\" in predictions[0]:\n            raise NotImplementedError(\n                \"Evaluating predictions with keypoints is not supported.\"\n            )\n\n        elif \"bbox\" in predictions[0] and not predictions[0][\"bbox\"] == []:\n            predictions_dataset.dataset[\"categories\"] = copy.deepcopy(\n                self.dataset[\"categories\"]","sourceCodeStart":516,"sourceCodeEnd":552,"githubUrl":"https://github.com/roboflow/supervision/blob/7f254d9784d4c37e0f03cd89ddee164c8db099c0/src/supervision/metrics/mean_average_precision.py#L516-L552","documentation":"In EvaluationDataset.load_predictions(), every prediction dict carries an image_id that must reference an image known to the target dataset (set(ids) <= set(self.get_image_ids())). This ValueError fires when one or more prediction image_ids do not exist in the COCO ground-truth set being evaluated against. It protects the evaluator from silently scoring predictions against images it has no ground truth for.","triggerScenarios":"Evaluating predictions from a different split (val predictions vs test ground truths); image ids that are strings in one place and ints in the other (COCO ids are ints); category/image id remapping after subsetting the dataset; predictions generated for all images but ground truth filtered to a sample; off-by-one or custom image_id schemes in a private dataset converted to COCO format.","commonSituations":"Mixing train/val/test artifacts; converting a custom dataset where filenames were hashed to ids differently on each export; type mismatches (str '000000123' vs int 123) after JSON round-trips; evaluating predictions produced by a checkpoint on data preprocessed with a different id mapping.","solutions":["Verify alignment first: set(p['image_id'] for p in preds) <= set(dataset.get_image_ids())","Re-generate predictions on the exact image set of the ground-truth dataset, or subset predictions to known ids","Normalize id types (cast to int) on both sides before evaluation","If the ground truth legitimately lacks images, filter predictions: preds = [p for p in preds if p['image_id'] in known_ids] — but prefer fixing the pipeline instead"],"exampleFix":"# before\nresults = json.load(open('val_preds.json'))       # ids from val split\ncoco_det = coco_gt.load_predictions(results)       # gt is test split -> error\n\n# after\nknown = set(coco_gt.get_image_ids())\nresults = [r for r in results if int(r['image_id']) in known]\nassert results, 'no predictions left after filtering'\ncoco_det = coco_gt.load_predictions(results)","handlingStrategy":"validation","validationCode":"known = set(dataset.get_image_ids())\nunknown = {p['image_id'] for p in predictions} - known\nif unknown:\n    raise ValueError(f'predictions reference unknown image ids: {sorted(unknown)[:5]}')\ndataset.load_predictions(predictions)","typeGuard":"def predictions_match_dataset(preds: list, dataset) -> bool:\n    \"\"\"True when every prediction image_id exists in the dataset.\"\"\"\n    return {p['image_id'] for p in preds} <= set(dataset.get_image_ids())","tryCatchPattern":"try:\n    coco_det = dataset.load_predictions(results)\nexcept ValueError as e:\n    if 'do not correspond' in str(e):\n        known = set(dataset.get_image_ids())\n        results = [r for r in results if r['image_id'] in known]\n        coco_det = dataset.load_predictions(results)\n    else:\n        raise","preventionTips":["Generate predictions and ground truth from the same image manifest","Normalize image_id types (int) on both sides after any JSON round-trip","Check id-set containment before evaluation"],"tags":["metrics","mean-average-precision","coco","data-alignment","validation"],"backgroundTag":null,"analyzedSha":"7f254d9784d4c37e0f03cd89ddee164c8db099c0","analyzedAt":"2026-08-15T05:13:01.950Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}