unslothai/unsloth · error · ValueError

`penalty` has to be a positive float, but is {penalty}

Error message

`penalty` has to be a positive float, but is {penalty}

What it means

Raised inside the monkey-patch RepetitionPenaltyLogitsProcessorPatch.__init__ (inference.py:2321), installed once per process over transformers' RepetitionPenaltyLogitsProcessor with a 64-token sliding window (from the OuteTTS notebook). Unlike upstream transformers (which accepts a float >= 1.0 conceptually and lets 0 slip through), this patch strictly requires a true float > 0 — isinstance(penalty, float) — so an int (e.g. 1) or a non-positive value raises ValueError. It fires the first time a generation with repetition_penalty runs after patching.

Source

Thrown at studio/backend/core/inference/inference.py:2321

    @classmethod
    def _patch_repetition_penalty_processor(cls):
        """Monkey-patch transformers' RepetitionPenaltyLogitsProcessor with a
        64-token sliding-window variant (from the OuteTTS notebook).
        Applied once per process.
        """
        if cls._repetition_penalty_patched:
            return
        cls._repetition_penalty_patched = True

        from transformers import LogitsProcessor
        import transformers.generation.utils as generation_utils

        class RepetitionPenaltyLogitsProcessorPatch(LogitsProcessor):
            def __init__(self, penalty: float):
                self.penalty_last_n = 64
                if not isinstance(penalty, float) or penalty <= 0:
                    raise ValueError(f"`penalty` has to be a positive float, but is {penalty}")
                self.penalty = penalty

            @torch.no_grad()
            def __call__(
                self, input_ids: torch.LongTensor, scores: torch.FloatTensor
            ) -> torch.FloatTensor:
                if self.penalty_last_n == 0 or self.penalty == 1.0:
                    return scores
                batch_size, seq_len = input_ids.shape
                vocab_size = scores.shape[-1]
                for b in range(batch_size):
                    start_index = max(0, seq_len - self.penalty_last_n)
                    window_indices = input_ids[b, start_index:]
                    if window_indices.numel() == 0:
                        continue
                    for token_id in set(window_indices.tolist()):
                        if token_id >= vocab_size:
                            continue

View on GitHub (pinned to 203007d190)

Solutions

1. Pass an explicit positive float: repetition_penalty=1.0 (int 1 fails isinstance(penalty, float)).
2. Coerce at the API boundary: float(repetition_penalty) before it reaches generation.
3. Validate range early: require 0 < repetition_penalty, and treat 1.0 as 'no penalty' (the patch short-circuits on penalty == 1.0).
4. If a numpy scalar is involved, wrap with float() — np.float64 is not a Python float under this check.

Example fix

# before
generate(text = "hi", repetition_penalty = 1)  # int -> ValueError

# after
penalty = float(raw_config.get("repetition_penalty", 1.0))
if penalty <= 0:
    raise HTTPException(400, "repetition_penalty must be positive")
generate(text = "hi", repetition_penalty = penalty)

def valid_penalty(p) -> float:
    p = float(p)
    if p <= 0:
        raise ValueError("repetition_penalty must be > 0")
    return p

def is_valid_penalty(p) -> bool:
    return isinstance(p, float) and p > 0

try:
    gen = engine.generate(text, repetition_penalty = penalty)
except ValueError as e:
    if "penalty" in str(e):
        return HTTPException(400, "repetition_penalty must be a positive float")
    raise

When it happens

Trigger: Passing repetition_penalty as an int (repetition_penalty=1 or 2) or a value <= 0 to audio/text generation once the patch class is instantiated; passing a numpy float or Decimal also fails the isinstance check.

Common situations: Config/JSON supplies repetition_penalty: 1 (int) instead of 1.0; a default of 0 used as 'unset'; copy-pasting upstream transformers code that tolerates ints; sliders in the UI yielding ints.

Related errors

• load error: {p.get('error')}
• model load did not reach ready within {timeout_s}s
• load error: {p.get('error')}
• Audio must be {max_minutes} {unit} or shorter.
• Could not decode the audio.

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/500023777b714fda. Report an issue: GitHub.