roboflow/supervision · error · ValueError

Results do not correspond to current coco set

Error message

Results do not correspond to current coco set

What it means

In EvaluationDataset.load_predictions(), every prediction dict carries an image_id that must reference an image known to the target dataset (set(ids) <= set(self.get_image_ids())). This ValueError fires when one or more prediction image_ids do not exist in the COCO ground-truth set being evaluated against. It protects the evaluator from silently scoring predictions against images it has no ground truth for.

Source

Thrown at src/supervision/metrics/mean_average_precision.py:534

        # Create an empty EvaluationDataset object for the predictions
        predictions_dataset = EvaluationDataset.empty()
        predictions_dataset.dataset["images"] = list(self.dataset["images"])

        if not isinstance(predictions, list):
            raise ValueError("results must be a list")

        # Handle empty predictions
        if len(predictions) == 0:
            predictions_dataset.dataset["annotations"] = []
            return predictions_dataset

        ids = [pred["image_id"] for pred in predictions]

        # Make sure the image ids from predictions exist in the current dataset.
        # A plain ``assert`` would be stripped under ``python -O``, so validate
        # this public-input contract with an explicit exception instead.
        if not set(ids) <= set(self.get_image_ids()):
            raise ValueError("Results do not correspond to current coco set")

        # Check if the predictions contain any unsupported keys
        if "caption" in predictions[0]:
            raise NotImplementedError(
                "Evaluating predictions with caption is not supported."
            )
        elif "segmentation" in predictions[0]:
            raise NotImplementedError(
                "Evaluating predictions with segmentation is not supported."
            )
        elif "keypoints" in predictions[0]:
            raise NotImplementedError(
                "Evaluating predictions with keypoints is not supported."
            )

        elif "bbox" in predictions[0] and not predictions[0]["bbox"] == []:
            predictions_dataset.dataset["categories"] = copy.deepcopy(
                self.dataset["categories"]

View on GitHub (pinned to 7f254d9784)

Solutions

1. Verify alignment first: set(p['image_id'] for p in preds) <= set(dataset.get_image_ids())
2. Re-generate predictions on the exact image set of the ground-truth dataset, or subset predictions to known ids
3. Normalize id types (cast to int) on both sides before evaluation
4. If the ground truth legitimately lacks images, filter predictions: preds = [p for p in preds if p['image_id'] in known_ids] — but prefer fixing the pipeline instead

Example fix

# before
results = json.load(open('val_preds.json'))       # ids from val split
coco_det = coco_gt.load_predictions(results)       # gt is test split -> error

# after
known = set(coco_gt.get_image_ids())
results = [r for r in results if int(r['image_id']) in known]
assert results, 'no predictions left after filtering'
coco_det = coco_gt.load_predictions(results)

known = set(dataset.get_image_ids())
unknown = {p['image_id'] for p in predictions} - known
if unknown:
    raise ValueError(f'predictions reference unknown image ids: {sorted(unknown)[:5]}')
dataset.load_predictions(predictions)

def predictions_match_dataset(preds: list, dataset) -> bool:
    """True when every prediction image_id exists in the dataset."""
    return {p['image_id'] for p in preds} <= set(dataset.get_image_ids())

try:
    coco_det = dataset.load_predictions(results)
except ValueError as e:
    if 'do not correspond' in str(e):
        known = set(dataset.get_image_ids())
        results = [r for r in results if r['image_id'] in known]
        coco_det = dataset.load_predictions(results)
    else:
        raise

When it happens

Trigger: Evaluating predictions from a different split (val predictions vs test ground truths); image ids that are strings in one place and ints in the other (COCO ids are ints); category/image id remapping after subsetting the dataset; predictions generated for all images but ground truth filtered to a sample; off-by-one or custom image_id schemes in a private dataset converted to COCO format.

Common situations: Mixing train/val/test artifacts; converting a custom dataset where filenames were hashed to ids differently on each export; type mismatches (str '000000123' vs int 123) after JSON round-trips; evaluating predictions produced by a checkpoint on data preprocessed with a different id mapping.

Related errors

• results must be a list
• Invalid metric target: {self._metric_target}
• Evaluating predictions with caption is not supported.
• coco_targets must be provided
• coco_predictions must be provided

AI-assisted analysis of roboflow/supervision@7f254d9784 (2026-08-15). Data as JSON: /api/errors/7112f4f551d3fb8c. Report an issue: GitHub.