roboflow/supervision · error · ValueError

Confusion matrix must have shape (..., 3), got {confusion_ma

Error message

Confusion matrix must have shape (..., 3), got {confusion_matrix.shape}

What it means

This ValueError comes from the internal broadcastable helper _recall_from_confusion_matrix, which computes recall = TP/(TP+FN) from an array whose last axis must hold exactly [TP, FP, FN]. It fires when the supplied confusion-matrix-like array's last dimension is not 3. End users normally never touch this helper; it is exercised inside MeanAverageRecall.compute(), so seeing it usually means the private API was called directly with a wrongly shaped array, or upstream code built an invalid stats structure.

Source

Thrown at src/supervision/metrics/mean_average_recall.py:653

        result_confusion_matrix: npt.NDArray[np.float64] = confusion_matrix
        return result_confusion_matrix

    @staticmethod
    def _compute_recall(
        confusion_matrix: npt.NDArray[np.float64],
    ) -> npt.NDArray[np.float64]:
        """
        Broadcastable function, computing the recall from the confusion matrix.

        Args:
            confusion_matrix: shape (N, ..., 3), where the last dimension
                contains the true positives, false positives, and false negatives.

        Returns:
            shape (N, ...), containing the recall for each element.
        """
        if not confusion_matrix.shape[-1] == 3:
            raise ValueError(
                f"Confusion matrix must have shape (..., 3), got "
                f"{confusion_matrix.shape}"
            )
        true_positives = confusion_matrix[..., 0]
        false_negatives = confusion_matrix[..., 2]

        denominator = true_positives + false_negatives
        recall = np.divide(
            true_positives,
            denominator,
            out=np.zeros_like(denominator, dtype=np.float64),
            where=denominator != 0,
        )

        result_recall: npt.NDArray[np.float64] = recall
        return result_recall

    def _detections_content(

View on GitHub (pinned to 7f254d9784)

Solutions

1. Reshape your data to (..., 3) with columns [true_positives, false_positives, false_negatives] before calling the helper
2. If integrating external confusion matrices, convert: stack TP, FP, FN along the last axis with np.stack([tp, fp, fn], axis=-1)
3. Do not call the private helper directly; use the public update()/compute() API which builds correctly shaped arrays
4. If reached via public compute() on unmodified supervision, report it as a bug with a reproducer

Example fix

# before
recall = mar._recall_from_confusion_matrix(np.stack([tp, fp], axis=-1))  # (N,2)

# after
cm = np.stack([tp, fp, fn], axis=-1)  # shape (N, 3): [TP, FP, FN]
recall = mar._recall_from_confusion_matrix(cm)

import numpy as np

def valid_confusion_matrix(cm: np.ndarray) -> bool:
    """True when last axis holds [TP, FP, FN]."""
    return cm.ndim >= 1 and cm.shape[-1] == 3

import numpy as np

def is_tpfpn_array(arr: object) -> bool:
    """Narrow an object to a (..., 3) TP/FP/FN numpy array."""
    return isinstance(arr, np.ndarray) and arr.ndim >= 1 and arr.shape[-1] == 3

try:
    recall = helper(cm)
except ValueError as e:
    raise ValueError(f'reshape {cm.shape} to (..., 3) as [TP, FP, FN]') from e

When it happens

Trigger: Calling MeanAverageRecall._recall_from_confusion_matrix (private) with an array whose last axis has != 3 elements, e.g. shape (N,4) from a 2x2 confusion matrix or (N,2) TP/FP-only arrays; internal misuse in compute() would indicate a supervision bug or corrupted stats accumulation (e.g. custom fork modified the stats tuples).

Common situations: Reusing code written for binary classification 2x2 matrices; passing precision-oriented [TP, FP] pairs; contributing to/forking supervision and changing the stats tuple layout; feeding precomputed arrays from another library (torchmetrics, sklearn) without reshaping.

Related errors

• Value must be a np.ndarray or a list
• xyxyxyxy must have shape (N, 4, 2); got {corners.shape}
• Unsupported metric target for IoU calculation
• Bounding boxes must be shaped (N, 4)
• Areas must be shaped (N,)

AI-assisted analysis of roboflow/supervision@7f254d9784 (2026-08-15). Data as JSON: /api/errors/17c079c03ce166d7. Report an issue: GitHub.