sgl-project/sglang · error · ValueError

frequency_penalty must be in [-2, 2], got {self.frequency_pe

Error message

frequency_penalty must be in [-2, 2], got {self.frequency_penalty}.

What it means

SamplingParams.verify() requires frequency_penalty to lie in [-2, 2] (OpenAI-compatible range). Out-of-range values raise this ValueError during normalize()/verify(). frequency_penalty penalizes tokens proportionally to how often they have appeared.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:170

            self.top_k = TOP_K_ALL  # whole vocabulary

    def verify(self, vocab_size):
        if self.beam_width is not None and self.beam_width < 1:
            raise ValueError(f"beam_width must be at least 1, got {self.beam_width}.")
        if not math.isfinite(self.temperature) or self.temperature < 0.0:
            raise ValueError(
                f"temperature must be a non-negative finite number, got {self.temperature}."
            )
        if not 0.0 < self.top_p <= 1.0:
            raise ValueError(f"top_p must be in (0, 1], got {self.top_p}.")
        if not 0.0 <= self.min_p <= 1.0:
            raise ValueError(f"min_p must be in [0, 1], got {self.min_p}.")
        if self.top_k < 1 or self.top_k == -1:
            raise ValueError(
                f"top_k must be -1 (disable) or at least 1, got {self.top_k}."
            )
        if not -2.0 <= self.frequency_penalty <= 2.0:
            raise ValueError(
                "frequency_penalty must be in [-2, 2], got "
                f"{self.frequency_penalty}."
            )
        if not -2.0 <= self.presence_penalty <= 2.0:
            raise ValueError(
                "presence_penalty must be in [-2, 2], got " f"{self.presence_penalty}."
            )
        if not 0.0 < self.repetition_penalty <= 2.0:
            raise ValueError(
                "repetition_penalty must be in (0, 2] (1.0 = no penalty), "
                f"got {self.repetition_penalty}."
            )
        if not 0 <= self.min_new_tokens:
            raise ValueError(
                f"min_new_tokens must be in [0, max_new_tokens], got "
                f"{self.min_new_tokens}."
            )
        if self.max_new_tokens is not None:

View on GitHub (pinned to 0132848349)

Solutions

  1. Clamp frequency_penalty to [-2, 2] before building params
  2. Validate incoming API requests and reject/clamp penalty fields
  3. Re-check tuned values from experiments that exceeded the range

Example fix

# before
params = SamplingParams(frequency_penalty=penalty)
# after
params = SamplingParams(frequency_penalty=max(-2.0, min(2.0, penalty)))
Defensive patterns

Strategy: validation

Validate before calling

frequency_penalty = max(-2.0, min(2.0, frequency_penalty))
params = SamplingParams(frequency_penalty=frequency_penalty)

Type guard

def valid_frequency_penalty(p):
    return isinstance(p,(int,float)) and -2.0 <= p <= 2.0

Try / catch

try:
    llm.generate(prompts, SamplingParams(frequency_penalty=p))
except ValueError as e:
    if 'frequency_penalty' in str(e):
        p = 0.0
    else:
        raise

Prevention

When it happens

Trigger: Passing SamplingParams(frequency_penalty=3.0), a large negative value, or a percent-scaled number; verify() runs via normalize() per request.

Common situations: Proxying OpenAI-API requests with unchecked penalty values; tuning scripts sweeping penalties beyond 2; confusing the 0-1 scale with the -2..2 scale.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5c194263cb8fd0b4. Report an issue: GitHub.