oobabooga/textgen · error · ValueError

`n_sigma` must be a non-negative float, but is {n_sigma}

Error message

`n_sigma` must be a non-negative float, but is {n_sigma}

What it means

Error "`n_sigma` must be a non-negative float, but is {n_sigma}" thrown in oobabooga/textgen.

Source

Thrown at modules/sampler_hijack.py:209

            sorted_indices_to_remove[..., : self.min_tokens_to_keep] = 0

        indices_to_remove = sorted_indices_to_remove.scatter(1, sorted_indices, sorted_indices_to_remove)
        scores = scores.masked_fill(indices_to_remove, self.filter_value)
        return scores


class TopNSigmaLogitsWarper(LogitsProcessor):
    def __init__(self, n_sigma: float = 2.0, filter_value: float = -float("Inf"), min_tokens_to_keep: int = 1):
        """
        Initialize Top-nσ Sampling logits warper.

        Args:
            n_sigma: The threshold multiplier for standard deviation
            filter_value: Value to assign to filtered logits
            min_tokens_to_keep: Minimum number of tokens to keep
        """
        if n_sigma < 0:
            raise ValueError(f"`n_sigma` must be a non-negative float, but is {n_sigma}")
        self.n_sigma = n_sigma
        self.filter_value = filter_value
        self.min_tokens_to_keep = min_tokens_to_keep

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        # Calculate max of logits
        max_logit = torch.max(scores, dim=-1, keepdim=True)[0]

        # Calculate standard deviation only on finite values
        finite_mask = torch.isfinite(scores)
        finite_scores = scores.masked_fill(~finite_mask, 0.0)
        std_logit = torch.std(finite_scores, dim=-1, keepdim=True)

        # Create mask where tokens with logits >= max_logit - n_sigma * std_logit are kept
        threshold = max_logit - self.n_sigma * std_logit
        indices_to_remove = scores < threshold

        if self.min_tokens_to_keep > 1:

View on GitHub (pinned to ed888c71f2)

Solutions

1. Set n_sigma to a non-negative float (>= 0); use 0 to disable the n_sigma sampler.

When it happens

Trigger: Raised during sampler validation when `n_sigma` is a negative number. Triggers when the n_sigma sampling parameter is set below 0; use 0 to disable or a non-negative float.

Common situations: See trigger scenarios.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/b257bc8e3e3c8738. Report an issue: GitHub.