roboflow/supervision · error · ValueError

MeanAverageRecall metric requires `confidence` on prediction

Error message

MeanAverageRecall metric requires `confidence` on predictions.

What it means

MeanAverageRecall ranks predictions by confidence when computing recall across IoU thresholds, so predictions must carry a confidence array. This ValueError fires in compute() when, for an image that has both non-empty targets and non-empty predictions, predictions.confidence is None. Targets never need confidence; only the prediction side is checked.

Source

Thrown at src/supervision/metrics/mean_average_recall.py:429

                        "predictions and targets."
                    )
                if len(predictions) == 0:
                    target_class_ids = np.asarray(targets.class_id, dtype=np.int32)
                    if len(target_class_ids) == 0:
                        continue
                    stats.append(
                        (
                            np.zeros((0, iou_thresholds.size), dtype=bool),
                            np.zeros((0, iou_thresholds.size), dtype=bool),
                            np.zeros((0,), dtype=int),
                            np.zeros((0,), dtype=int),
                            target_class_ids,
                        )
                    )

                else:
                    if predictions.confidence is None:
                        raise ValueError(
                            "MeanAverageRecall metric requires `confidence` on "
                            "predictions."
                        )
                    prediction_class_ids = np.asarray(
                        predictions.class_id, dtype=np.int32
                    )
                    target_class_ids = np.asarray(targets.class_id, dtype=np.int32)
                    prediction_confidence = np.asarray(
                        predictions.confidence, dtype=np.float32
                    )
                    if self._metric_target == MetricTarget.BOXES:
                        # BOXES target never yields CompactMask; narrow for mypy.
                        iou = box_iou_batch(
                            cast(npt.NDArray[np.number], target_contents),
                            cast(npt.NDArray[np.number], prediction_contents),
                        )
                    elif self._metric_target == MetricTarget.MASKS:
                        iou = mask_iou_batch(target_contents, prediction_contents)

View on GitHub (pinned to 7f254d9784)

Solutions

1. Attach a confidence array when constructing prediction Detections (even a constant ones array makes ranking well-defined)
2. Use a model connector that preserves scores (e.g. from_ultralytics keeps confidence)
3. If predictions genuinely have no scores, set confidence=np.ones(len(detections), dtype=np.float32) on both sides deliberately
4. Pre-check before update(): if len(preds)>0 and preds.confidence is None, raise your own descriptive error or fill defaults

Example fix

# before
preds = sv.Detections(xyxy=boxes, class_id=ids)  # no confidence
mar.update(preds, targets)

# after
preds = sv.Detections(xyxy=boxes, class_id=ids,
                      confidence=np.array(scores, dtype=np.float32))
mar.update(preds, targets)

import numpy as np
from supervision.detection.core import Detections

def confidence_ready(preds: Detections) -> bool:
    """Non-empty predictions need confidence for MAR ranking."""
    return len(preds) == 0 or preds.confidence is not None

import numpy as np
from supervision.detection.core import Detections

def with_confidence(dets: Detections) -> Detections:
    """Return Detections guaranteed to carry confidence (default 1.0)."""
    if dets.confidence is None and len(dets) > 0:
        dets.confidence = np.ones(len(dets), dtype=np.float32)
    return dets

When it happens

Trigger: Building prediction sv.Detections from ground-truth-style data or manual boxes without confidence=; using a connector that discards scores; copying target Detections to fake predictions in a sanity test; note the guard is inside the len(predictions)>0 branch, so it triggers exactly when there is something to rank.

Common situations: Hand-crafted unit tests or visualizations converted into evaluation without scores; trackers (ByteTrack) whose output Detections may lack confidence unless re-attached; deterministic rule-based detectors that emit boxes with no score; assuming MAR works like a rank-free overlap metric.

Related errors

• The number of predictions ({len(predictions)}) and targets (
• MeanAverageRecall metric requires `class_id` on both predict
• Invalid metric target: {self._metric_target}
• MeanAverageRecall with `MetricTarget.MASKS` requires detecti
• The number of predictions ({len(predictions)}) and targets (

AI-assisted analysis of roboflow/supervision@7f254d9784 (2026-08-15). Data as JSON: /api/errors/e0491c9e2ec56401. Report an issue: GitHub.