sgl-project/sglang · error · ValueError

top_k must be -1 (disable) or at least 1, got {self.top_k}.

Error message

top_k must be -1 (disable) or at least 1, got {self.top_k}.

What it means

SamplingParams.verify() requires top_k to be either -1 (meaning 'consider the whole vocabulary', disable top-k filtering) or an integer >= 1. Values like 0, -2, or -100 raise this ValueError during normalize()/verify(). Note that 0 is invalid even though some APIs treat 0 as disabled.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:166

            # top_k = 1 means greedy sampling
            self.temperature = 1.0
            self.top_k = 1
        if self.top_k == -1:
            self.top_k = TOP_K_ALL  # whole vocabulary

    def verify(self, vocab_size):
        if self.beam_width is not None and self.beam_width < 1:
            raise ValueError(f"beam_width must be at least 1, got {self.beam_width}.")
        if not math.isfinite(self.temperature) or self.temperature < 0.0:
            raise ValueError(
                f"temperature must be a non-negative finite number, got {self.temperature}."
            )
        if not 0.0 < self.top_p <= 1.0:
            raise ValueError(f"top_p must be in (0, 1], got {self.top_p}.")
        if not 0.0 <= self.min_p <= 1.0:
            raise ValueError(f"min_p must be in [0, 1], got {self.min_p}.")
        if self.top_k < 1 or self.top_k == -1:
            raise ValueError(
                f"top_k must be -1 (disable) or at least 1, got {self.top_k}."
            )
        if not -2.0 <= self.frequency_penalty <= 2.0:
            raise ValueError(
                "frequency_penalty must be in [-2, 2], got "
                f"{self.frequency_penalty}."
            )
        if not -2.0 <= self.presence_penalty <= 2.0:
            raise ValueError(
                "presence_penalty must be in [-2, 2], got " f"{self.presence_penalty}."
            )
        if not 0.0 < self.repetition_penalty <= 2.0:
            raise ValueError(
                "repetition_penalty must be in (0, 2] (1.0 = no penalty), "
                f"got {self.repetition_penalty}."
            )
        if not 0 <= self.min_new_tokens:
            raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

  1. Use top_k=-1 to disable top-k filtering
  2. Use top_k=1 for greedy decoding
  3. Ensure integer values >= 1 otherwise (e.g. top_k=50)

Example fix

# before
params = SamplingParams(top_k=0)
# after
params = SamplingParams(top_k=-1)  # disabled
Defensive patterns

Strategy: validation

Validate before calling

if not isinstance(top_k, int) or top_k < 1:
    top_k = -1
params = SamplingParams(top_k=top_k)

Type guard

def valid_top_k(k):
    return isinstance(k, int) and (k == -1 or k >= 1)

Try / catch

try:
    llm.generate(prompts, SamplingParams(top_k=k))
except ValueError as e:
    if 'top_k' in str(e):
        k = -1
    else:
        raise

Prevention

When it happens

Trigger: Passing SamplingParams(top_k=0) intending to disable top-k; any top_k < 1 other than exactly -1. verify() is invoked via normalize() when preparing the request.

Common situations: Porting configs from engines where top_k=0 means disabled; negative 'sentinel' values other than -1; UIs with 0 defaults; note the code above already maps top_k=0 -> 1 in a branch that sets top_k=1, so hitting this usually means an explicit out-of-range value.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f09bf5354e09a4e4. Report an issue: GitHub.