sgl-project/sglang · error · ValueError

max_new_tokens must be at least 0, got {self.max_new_tokens}

Error message

max_new_tokens must be at least 0, got {self.max_new_tokens}.

What it means

SamplingParams.verify() validates that max_new_tokens, when set, is non-negative. It is raised during normalize() (which calls verify) before the request is scheduled, because a negative generation budget is meaningless and would corrupt scheduling math.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:194

                "presence_penalty must be in [-2, 2], got " f"{self.presence_penalty}."
            )
        if not 0.0 < self.repetition_penalty <= 2.0:
            raise ValueError(
                "repetition_penalty must be in (0, 2] (1.0 = no penalty), "
                f"got {self.repetition_penalty}."
            )
        if not 0 <= self.min_new_tokens:
            raise ValueError(
                f"min_new_tokens must be in [0, max_new_tokens], got "
                f"{self.min_new_tokens}."
            )
        if self.max_new_tokens is not None:
            if self.max_new_tokens < 0:
                raise ValueError(
                    f"max_new_tokens must be at least 0, got {self.max_new_tokens}."
                )
            if not self.min_new_tokens <= self.max_new_tokens:
                raise ValueError(
                    f"min_new_tokens must be in [0, max_new_tokens({self.max_new_tokens})], got "
                    f"{self.min_new_tokens}."
                )
        if self.logit_bias is not None:
            for token_id in self.logit_bias:
                if not 0 <= int(token_id) < vocab_size:
                    raise ValueError(
                        f"logit_bias must has keys in [0, {vocab_size - 1}], got "
                        f"{token_id}."
                    )

        grammars = [
            self.json_schema,
            self.regex,
            self.ebnf,
            self.structural_tag,
        ]  # since mutually exclusive, only one can be set
        if sum(x is not None for x in grammars) > 1:

View on GitHub (pinned to 0132848349)

Solutions

Set max_new_tokens to None if you mean 'no limit', or a non-negative integer.
Fix the arithmetic that produced a negative value (e.g. clamp max(0, budget) or skip the request when the prompt exceeds the cap).
Validate inputs before constructing SamplingParams.

Example fix

# before
sp = SamplingParams(max_new_tokens=context_len - len(prompt_ids))  # can be -1
# after
sp = SamplingParams(max_new_tokens=max(0, context_len - len(prompt_ids)))
# or, for no limit:
sp = SamplingParams(max_new_tokens=None)

Defensive patterns

Strategy: validation

Validate before calling

if max_new_tokens is not None and max_new_tokens < 0:
    max_new_tokens = None  # or clamp to 0 / raise your own error
sp = SamplingParams(max_new_tokens=max_new_tokens)

Prevention

Never use -1 to mean unlimited; use None.
Clamp computed budgets with max(0, budget) and skip when the prompt overflows the context.

When it happens

Trigger: Passing SamplingParams(max_new_tokens=-1) (or a negative value) to a generate/launch request; often happens when max_new_tokens is computed as a subtraction like context_len - prompt_len that goes negative, or when a typo/CLI arg passes a negative number.

Common situations: Dynamic token-budget computation underflows (prompt longer than the cap), off-by-one or negated arithmetic in a wrapper, or passing -1 meaning 'unlimited' (vLLM-style) into SGLang, which expects None for unlimited.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b50e3a65e8d5dcc1. Report an issue: GitHub.