sgl-project/sglang · error · ValueError

top_p must be in (0, 1], got {self.top_p}.

Error message

top_p must be in (0, 1], got {self.top_p}.

What it means

SamplingParams.verify() requires top_p to lie in the exclusive-inclusive interval (0, 1] for nucleus sampling. Values of 0, negative numbers, or anything above 1 raise this ValueError during normalize()/verify(). top_p=0 would select an empty candidate set, hence it is explicitly disallowed even though 0 < 1 numerically.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:162

        self.structural_tag = self.structural_tag or None

        # Process some special cases
        if 0 <= self.temperature < _SAMPLING_EPS:
            # top_k = 1 means greedy sampling
            self.temperature = 1.0
            self.top_k = 1
        if self.top_k == -1:
            self.top_k = TOP_K_ALL  # whole vocabulary

    def verify(self, vocab_size):
        if self.beam_width is not None and self.beam_width < 1:
            raise ValueError(f"beam_width must be at least 1, got {self.beam_width}.")
        if not math.isfinite(self.temperature) or self.temperature < 0.0:
            raise ValueError(
                f"temperature must be a non-negative finite number, got {self.temperature}."
            )
        if not 0.0 < self.top_p <= 1.0:
            raise ValueError(f"top_p must be in (0, 1], got {self.top_p}.")
        if not 0.0 <= self.min_p <= 1.0:
            raise ValueError(f"min_p must be in [0, 1], got {self.min_p}.")
        if self.top_k < 1 or self.top_k == -1:
            raise ValueError(
                f"top_k must be -1 (disable) or at least 1, got {self.top_k}."
            )
        if not -2.0 <= self.frequency_penalty <= 2.0:
            raise ValueError(
                "frequency_penalty must be in [-2, 2], got "
                f"{self.frequency_penalty}."
            )
        if not -2.0 <= self.presence_penalty <= 2.0:
            raise ValueError(
                "presence_penalty must be in [-2, 2], got " f"{self.presence_penalty}."
            )
        if not 0.0 < self.repetition_penalty <= 2.0:
            raise ValueError(
                "repetition_penalty must be in (0, 2] (1.0 = no penalty), "

View on GitHub (pinned to 0132848349)

Solutions

  1. For greedy decoding use top_k=1 and/or temperature=0 instead of top_p=0
  2. Set top_p=1.0 to disable nucleus filtering
  3. If you need very aggressive truncation use a tiny positive value like top_p=0.01

Example fix

# before
params = SamplingParams(top_p=0)   # invalid
# after
params = SamplingParams(top_p=1.0, temperature=0.0)  # greedy decoding
Defensive patterns

Strategy: validation

Validate before calling

top_p = top_p if isinstance(top_p,(int,float)) and 0.0 < top_p <= 1.0 else 1.0
params = SamplingParams(top_p=top_p)

Type guard

def valid_top_p(p):
    return isinstance(p,(int,float)) and 0.0 < p <= 1.0

Try / catch

try:
    llm.generate(prompts, SamplingParams(top_p=p))
except ValueError as e:
    if 'top_p' in str(e):
        p = 1.0
    else:
        raise

Prevention

When it happens

Trigger: Passing SamplingParams(top_p=0) (e.g. intending greedy), top_p=1.5, or a negative value; verify() is called via normalize() when the request is prepared.

Common situations: Porting configs from other engines where top_p=0 disables nucleus sampling; sliders/UIs allowing 0; multiplying top_p by a factor that pushes it above 1.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/55ae46397c983fc7. Report an issue: GitHub.