roboflow/supervision · error · ValueError

results must be a list

Error message

results must be a list

What it means

EvaluationDataset.load_predictions() (the COCO-format evaluation path used by MeanAveragePrecision's COCO backend) requires its predictions argument to already be a list of COCO-style result dicts. Unlike the Detections-facing update() path, it does not auto-wrap single objects; this ValueError fires when predictions is not a list instance. The comment-free contract: pass a list (possibly empty) of dicts with keys like image_id, category_id, bbox, score.

Source

Thrown at src/supervision/metrics/mean_average_precision.py:521

            return []
        return [self.anns[idx] for idx in ids]

    def load_predictions(self, predictions: list[_TypeCocoDict]) -> EvaluationDataset:
        """
        Load prediction result into an EvaluationDataset object.

        Args:
            predictions: prediction result.

        Returns:
            EvaluationDataset object representing the predictions.
        """
        # Create an empty EvaluationDataset object for the predictions
        predictions_dataset = EvaluationDataset.empty()
        predictions_dataset.dataset["images"] = list(self.dataset["images"])

        if not isinstance(predictions, list):
            raise ValueError("results must be a list")

        # Handle empty predictions
        if len(predictions) == 0:
            predictions_dataset.dataset["annotations"] = []
            return predictions_dataset

        ids = [pred["image_id"] for pred in predictions]

        # Make sure the image ids from predictions exist in the current dataset.
        # A plain ``assert`` would be stripped under ``python -O``, so validate
        # this public-input contract with an explicit exception instead.
        if not set(ids) <= set(self.get_image_ids()):
            raise ValueError("Results do not correspond to current coco set")

        # Check if the predictions contain any unsupported keys
        if "caption" in predictions[0]:
            raise NotImplementedError(
                "Evaluating predictions with caption is not supported."

View on GitHub (pinned to 7f254d9784)

Solutions

1. Pass a list: load_predictions(list(predictions))
2. If you have a single prediction dict, wrap it: load_predictions([pred])
3. For COCO JSON files, load and pass the top-level array: results = json.load(f); load_predictions(results)
4. Prefer the public MeanAveragePrecision.update(preds, targets) with Detections, which normalizes input shapes itself

Example fix

# before
preds = {'image_id': 1, 'category_id': 2, 'bbox': [...], 'score': 0.9}
dataset.load_predictions(preds)  # dict, not list

# after
preds = [{'image_id': 1, 'category_id': 2, 'bbox': [...], 'score': 0.9}]
dataset.load_predictions(preds)

if not isinstance(predictions, list):
    predictions = [predictions] if isinstance(predictions, dict) else list(predictions)
dataset.load_predictions(predictions)

from typing import Any, List, Dict

def is_prediction_list(value: Any) -> bool:
    """True when value is a list (possibly of COCO result dicts)."""
    return isinstance(value, list)

try:
    coco_det = dataset.load_predictions(results)
except ValueError as e:
    if 'must be a list' in str(e) and isinstance(results, dict):
        coco_det = dataset.load_predictions([results])
    else:
        raise

When it happens

Trigger: Calling EvaluationDataset.load_predictions(single_dict) or load_predictions(tuple(generator)) or a numpy array of dicts; passing a COCO-results JSON object (a dict of lists) instead of the list itself; calling the COCO-evaluator API directly instead of going through MeanAveragePrecision.update().

Common situations: json.load of a COCO results file yields a list but users sometimes wrap or transform it; using pandas itertuples/tuple outputs; migrating code that mixed torchmetrics/supervision COCO APIs; calling internal evaluation APIs while integrating custom pipelines.

Related errors

• Results do not correspond to current coco set
• Invalid metric target: {self._metric_target}
• Evaluating predictions with caption is not supported.
• coco_targets must be provided
• coco_predictions must be provided

AI-assisted analysis of roboflow/supervision@7f254d9784 (2026-08-15). Data as JSON: /api/errors/d90f413bfa6bb5d3. Report an issue: GitHub.