TheAlgorithms/Python · error · ValueError

Predicted probabilities must sum to approximately 1.

Error message

Predicted probabilities must sum to approximately 1.

What it means

Raised by categorical_cross_entropy when rows of y_pred do not sum to approximately 1 within the epsilon tolerance (np.isclose with rtol=atol=epsilon). Cross-entropy against a one-hot target is only a proper probability loss when predictions form a distribution per row, so non-normalized outputs are rejected.

Source

Thrown at machine_learning/loss_functions.py:145

    >>> categorical_cross_entropy(true_labels, pred_probs)
    Traceback (most recent call last):
        ...
    ValueError: y_true must be one-hot encoded.
    >>> true_labels = np.array([[1, 0, 0], [0, 1, 0]])
    >>> pred_probs = np.array([[0.9, 0.1, 0.1], [0.2, 0.7, 0.1]])
    >>> categorical_cross_entropy(true_labels, pred_probs)
    Traceback (most recent call last):
        ...
    ValueError: Predicted probabilities must sum to approximately 1.
    """
    if y_true.shape != y_pred.shape:
        raise ValueError("Input arrays must have the same shape.")

    if np.any((y_true != 0) & (y_true != 1)) or np.any(y_true.sum(axis=1) != 1):
        raise ValueError("y_true must be one-hot encoded.")

    if not np.all(np.isclose(np.sum(y_pred, axis=1), 1, rtol=epsilon, atol=epsilon)):
        raise ValueError("Predicted probabilities must sum to approximately 1.")

    y_pred = np.clip(y_pred, epsilon, 1)  # Clip predictions to avoid log(0)
    return -np.sum(y_true * np.log(y_pred))


def categorical_focal_cross_entropy(
    y_true: np.ndarray,
    y_pred: np.ndarray,
    alpha: np.ndarray = None,
    gamma: float = 2.0,
    epsilon: float = 1e-15,
) -> float:
    """
    Calculate the mean categorical focal cross-entropy (CFCE) loss between true
    labels and predicted probabilities for multi-class classification.

    CFCE loss is a generalization of binary focal cross-entropy for multi-class
    classification. It addresses class imbalance by focusing on hard examples.

View on GitHub (pinned to f5988cc097)

Solutions

1. Apply softmax to logits before calling: y_pred = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True).
2. Check row sums in your eval harness: np.allclose(y_pred.sum(axis=1), 1).
3. Ensure the model's final layer is softmax (not linear/sigmoid) for this loss.

Example fix

# before
logits = model(x)  # unnormalized
categorical_cross_entropy(y_true, logits)

# after
logits = model(x)
probs = np.exp(logits) / np.exp(logits).sum(axis=1, keepdims=True)
categorical_cross_entropy(y_true, probs)

def softmax_rows(logits: np.ndarray) -> np.ndarray:
    e = np.exp(logits - logits.max(axis=1, keepdims=True))
    return e / e.sum(axis=1, keepdims=True)

probs = softmax_rows(np.asarray(y_pred))
assert np.allclose(probs.sum(axis=1), 1.0)
loss = categorical_cross_entropy(y_true, probs)

def is_row_stochastic(y_pred: np.ndarray) -> bool:
    return y_pred.ndim == 2 and np.all(np.isclose(y_pred.sum(axis=1), 1.0))

try:
    categorical_cross_entropy(y_true, y_pred)
except ValueError as e:
    if "sum to approximately 1" in str(e):
        e_ = np.exp(y_pred - y_pred.max(axis=1, keepdims=True))
        return categorical_cross_entropy(y_true, e_ / e_.sum(axis=1, keepdims=True))
    raise

When it happens

Trigger: Passing raw logits (unnormalized model outputs) as y_pred, e.g. [[0.9, 0.1, 0.1], [0.2, 0.7, 0.1]] whose first row sums to 1.1 (the exact doctest example), or forgetting a softmax layer.

Common situations: Taking the layer before the softmax from a neural net, applying sigmoid instead of softmax for multiclass output, or manually dividing by the wrong normalizer.

Related errors

• Input arrays must have the same shape.
• Input arrays must have the same length.
• y_true must be one-hot encoded.
• Shape of y_true and y_pred must be the same.
• Length of alpha must match the number of classes.

AI-assisted analysis of TheAlgorithms/Python@f5988cc097 (2026-08-14). Data as JSON: /api/errors/f4de4823f614620d. Report an issue: GitHub.