sgl-project/sglang · error · ValueError

min_new_tokens must be in [0, max_new_tokens], got {self.min

Error message

min_new_tokens must be in [0, max_new_tokens], got {self.min_new_tokens}.

What it means

SamplingParams.verify() requires min_new_tokens to be >= 0 (and later <= max_new_tokens when max_new_tokens is set). A negative min_new_tokens raises this ValueError during normalize()/verify(). min_new_tokens forces the model to generate at least N tokens by suppressing EOS until the count is reached.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:190

                f"{self.frequency_penalty}."
            )
        if not -2.0 <= self.presence_penalty <= 2.0:
            raise ValueError(
                "presence_penalty must be in [-2, 2], got " f"{self.presence_penalty}."
            )
        if not 0.0 < self.repetition_penalty <= 2.0:
            raise ValueError(
                "repetition_penalty must be in (0, 2] (1.0 = no penalty), "
                f"got {self.repetition_penalty}."
            )
        if not 0 <= self.min_new_tokens:
            raise ValueError(
                f"min_new_tokens must be in [0, max_new_tokens], got "
                f"{self.min_new_tokens}."
            )
        if self.max_new_tokens is not None:
            if self.max_new_tokens < 0:
                raise ValueError(
                    f"max_new_tokens must be at least 0, got {self.max_new_tokens}."
                )
            if not self.min_new_tokens <= self.max_new_tokens:
                raise ValueError(
                    f"min_new_tokens must be in [0, max_new_tokens({self.max_new_tokens})], got "
                    f"{self.min_new_tokens}."
                )
        if self.logit_bias is not None:
            for token_id in self.logit_bias:
                if not 0 <= int(token_id) < vocab_size:
                    raise ValueError(
                        f"logit_bias must has keys in [0, {vocab_size - 1}], got "
                        f"{token_id}."
                    )

        grammars = [
            self.json_schema,
            self.regex,

View on GitHub (pinned to 0132848349)

Solutions

  1. Clamp min_new_tokens to >= 0 (use 0 to disable)
  2. Ensure min_new_tokens <= max_new_tokens when max_new_tokens is set
  3. Compute min_new_tokens defensively: max(0, min(min_new, max_new or min_new))

Example fix

# before
params = SamplingParams(min_new_tokens=min_len, max_new_tokens=max_len)  # min_len may be negative
# after
params = SamplingParams(min_new_tokens=max(0, min_len), max_new_tokens=max(max_len, 0))
Defensive patterns

Strategy: validation

Validate before calling

min_new_tokens = max(0, min_new_tokens)
if max_new_tokens is not None:
    max_new_tokens = max(0, max_new_tokens)
    min_new_tokens = min(min_new_tokens, max_new_tokens)
params = SamplingParams(min_new_tokens=min_new_tokens, max_new_tokens=max_new_tokens)

Type guard

def valid_min_new_tokens(mn, mx=None):
    return isinstance(mn, int) and mn >= 0 and (mx is None or mn <= mx)

Try / catch

try:
    llm.generate(prompts, SamplingParams(min_new_tokens=mn, max_new_tokens=mx))
except ValueError as e:
    if 'min_new_tokens' in str(e):
        mn = 0
    else:
        raise

Prevention

When it happens

Trigger: Passing SamplingParams(min_new_tokens=-1); also passing min_new_tokens greater than max_new_tokens triggers the companion check further down. verify() runs via normalize() per request.

Common situations: Computing min_new_tokens as target_length - current_length and going negative when the prompt already exceeds the target; config defaults of -1 used as 'unset'; off-by-one arithmetic in token budgeting.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/681dc2d1cf364a71. Report an issue: GitHub.