oobabooga/textgen · error · ValueError

`penalty` has to be strictly positive, but is {penalty}

Error message

`penalty` has to be strictly positive, but is {penalty}

What it means

Error "`penalty` has to be strictly positive, but is {penalty}" thrown in oobabooga/textgen.

Source

Thrown at modules/sampler_hijack.py:486

        indices_to_remove = sorted_indices_to_remove.unsqueeze(0).scatter(1, sorted_indices.unsqueeze(0), sorted_indices_to_remove.unsqueeze(0))
        scores = scores.masked_fill(indices_to_remove, self.filter_value)
        return scores


class SpyLogitsWarper(LogitsProcessor):
    def __init__(self):
        pass

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        global global_scores
        global_scores = scores
        return scores


class RepetitionPenaltyLogitsProcessorWithRange(LogitsProcessor):
    def __init__(self, penalty: float, _range: int):
        if not (penalty > 0):
            raise ValueError(f"`penalty` has to be strictly positive, but is {penalty}")
        self.penalty = penalty
        self._range = _range

    def apply_repetition_penalty(self, input_ids_row, scores_row):
        unique_ids = torch.unique(input_ids_row)
        score = torch.gather(scores_row, 0, unique_ids)

        # Apply multiplicative repetition penalty
        score = torch.where(score < 0, score * self.penalty, score / self.penalty)
        scores_row.scatter_(0, unique_ids, score)
        return scores_row

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        input_ids = input_ids[:, -self._range:]
        for input_ids_row, scores_row in zip(input_ids, scores):
            scores_row = self.apply_repetition_penalty(input_ids_row, scores_row)

        return scores

View on GitHub (pinned to ed888c71f2)

Solutions

1. Set the penalty to a value strictly greater than 0; typical values are around 1.0-1.2.

When it happens

Trigger: Raised during sampler validation when a repetition/frequency `penalty` value is not strictly positive (<= 0). Triggers when penalty parameters are set to 0 or negative in generation settings.

Common situations: See trigger scenarios.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/8cbb73ad3cae7ca7. Report an issue: GitHub.