roboflow/supervision · error · ValueError

F1Score metric requires `confidence` on predictions.

Error message

F1Score metric requires `confidence` on predictions.

What it means

Raised by F1Score.update() when an image has both predictions and targets, but the predictions carry no confidence array. Confidence is used to rank predictions when computing precision-recall curves and picking the operating point for F1. Without it the metric cannot order detections, so it refuses rather than returning misleading numbers.

Source

Thrown at src/supervision/metrics/f1_score.py:237

                if len(predictions) == 0:
                    target_class_ids = np.asarray(targets.class_id, dtype=np.int32)[
                        target_size_mask
                    ]
                    if len(target_class_ids) == 0:
                        continue
                    stats.append(
                        (
                            np.zeros((0, iou_thresholds.size), dtype=bool),
                            np.zeros((0, iou_thresholds.size), dtype=bool),
                            np.zeros((0,), dtype=np.float32),
                            np.zeros((0,), dtype=int),
                            target_class_ids,
                        )
                    )

                else:
                    if predictions.confidence is None:
                        raise ValueError(
                            "F1Score metric requires `confidence` on predictions."
                        )
                    prediction_class_ids = np.asarray(
                        predictions.class_id, dtype=np.int32
                    )
                    target_class_ids = np.asarray(targets.class_id, dtype=np.int32)
                    prediction_confidence = np.asarray(
                        predictions.confidence, dtype=np.float32
                    )
                    if self._metric_target == MetricTarget.BOXES:
                        # BOXES target never yields CompactMask; narrow for mypy.
                        iou = box_iou_batch(
                            cast(npt.NDArray[np.number], target_contents),
                            cast(npt.NDArray[np.number], prediction_contents),
                        )
                    elif self._metric_target == MetricTarget.MASKS:
                        iou = mask_iou_batch(target_contents, prediction_contents)
                    elif self._metric_target == MetricTarget.ORIENTED_BOUNDING_BOXES:

View on GitHub (pinned to 7f254d9784)

Solutions

1. Attach a confidence array to predictions: Detections(..., confidence=np.full(len(xyxy), 1.0, dtype=np.float32)) when no real score exists
2. If scores come from your model, propagate them instead of dropping them in post-processing
3. Filter out scoreless detections before evaluation when a dummy 1.0 confidence would distort results

Example fix

# before
preds = sv.Detections(xyxy=boxes, class_id=ids)  # no confidence
f1.update(targets=targets, predictions=preds)

# after
preds = sv.Detections(
    xyxy=boxes,
    class_id=ids,
    confidence=np.full(len(boxes), 1.0, dtype=np.float32),
)
f1.update(targets=targets, predictions=preds)

if len(predictions) > 0 and len(targets) > 0 and predictions.confidence is None:
    predictions = sv.Detections(
        xyxy=predictions.xyxy,
        class_id=predictions.class_id,
        confidence=np.ones(len(predictions), dtype=np.float32),
    )
f1.update(targets=targets, predictions=predictions)

def has_confidence(dets: sv.Detections) -> bool:
    """True when confidence is populated."""
    return dets.confidence is not None

try:
    f1.update(targets=targets, predictions=predictions)
except ValueError as e:
    if 'confidence on predictions' in str(e):
        predictions.confidence = np.ones(len(predictions), dtype=np.float32)
    else:
        raise

When it happens

Trigger: Calling F1Score().update() where len(predictions) > 0, len(targets) > 0, predictions.class_id is set, but predictions.confidence is None.

Common situations: Detections built from non-probabilistic sources (manual annotation, geometric detection) that omit scores; tracker outputs stripped of confidence; fixtures copied from examples that only set xyxy and class_id.

Related errors

• MeanAverageRecall metric requires `confidence` on prediction
• F1Score metric requires `class_id` and `confidence` on predi
• F1Score metric requires `class_id` on both predictions and t
• No edges defined for class_id={class_id}.
• 2D boolean mask row count {mask.shape[0]} does not match obj

AI-assisted analysis of roboflow/supervision@7f254d9784 (2026-08-15). Data as JSON: /api/errors/a4d43905a431df09. Report an issue: GitHub.