sgl-project/sglang · error · ValueError

temperature must be a non-negative finite number, got {self.

Error message

temperature must be a non-negative finite number, got {self.temperature}.

What it means

SamplingParams.verify() requires temperature to be a finite, non-negative number. NaN, +inf/-inf, or any negative value raises this ValueError during normalize()/verify() before scheduling. Temperature is applied as a softmax divisor, so infinite or negative values are mathematically invalid.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:158

        # An empty grammar constraint means "unset", not "constrain to nothing".
        self.json_schema = self.json_schema or None
        self.regex = self.regex or None
        self.ebnf = self.ebnf or None
        self.structural_tag = self.structural_tag or None

        # Process some special cases
        if 0 <= self.temperature < _SAMPLING_EPS:
            # top_k = 1 means greedy sampling
            self.temperature = 1.0
            self.top_k = 1
        if self.top_k == -1:
            self.top_k = TOP_K_ALL  # whole vocabulary

    def verify(self, vocab_size):
        if self.beam_width is not None and self.beam_width < 1:
            raise ValueError(f"beam_width must be at least 1, got {self.beam_width}.")
        if not math.isfinite(self.temperature) or self.temperature < 0.0:
            raise ValueError(
                f"temperature must be a non-negative finite number, got {self.temperature}."
            )
        if not 0.0 < self.top_p <= 1.0:
            raise ValueError(f"top_p must be in (0, 1], got {self.top_p}.")
        if not 0.0 <= self.min_p <= 1.0:
            raise ValueError(f"min_p must be in [0, 1], got {self.min_p}.")
        if self.top_k < 1 or self.top_k == -1:
            raise ValueError(
                f"top_k must be -1 (disable) or at least 1, got {self.top_k}."
            )
        if not -2.0 <= self.frequency_penalty <= 2.0:
            raise ValueError(
                "frequency_penalty must be in [-2, 2], got "
                f"{self.frequency_penalty}."
            )
        if not -2.0 <= self.presence_penalty <= 2.0:
            raise ValueError(
                "presence_penalty must be in [-2, 2], got " f"{self.presence_penalty}."

View on GitHub (pinned to 0132848349)

Solutions

Use temperature=0.0 for greedy decoding (or 1.0 for no scaling)
Fix the upstream computation producing NaN/inf (guard divisions, clamp values)
Clamp temperature before constructing params: max(0.0, min(t, 10.0))

Example fix

# before
params = SamplingParams(temperature=temperature)  # temperature may be NaN
# after
if not math.isfinite(temperature) or temperature < 0:
    temperature = 0.0
params = SamplingParams(temperature=temperature)

Defensive patterns

Strategy: validation

Validate before calling

import math
temperature = temperature if (isinstance(temperature,(int,float)) and math.isfinite(temperature) and temperature >= 0) else 0.0
params = SamplingParams(temperature=temperature)

Type guard

import math
def valid_temperature(t):
    return isinstance(t,(int,float)) and math.isfinite(t) and t >= 0

Try / catch

try:
    llm.generate(prompts, SamplingParams(temperature=t))
except ValueError as e:
    if 'temperature' in str(e):
        llm.generate(prompts, SamplingParams(temperature=0.0))
    else:
        raise

Prevention

Guard divisions in dynamic temperature schedules to avoid NaN
Use temperature=0.0 for greedy decoding, never -1 or NaN as 'disabled'

When it happens

Trigger: Passing SamplingParams(temperature=float('nan')), float('inf'), a negative number, or a string like 'nan' that gets cast; verify() runs via normalize() on every request.

Common situations: Computing temperature dynamically (e.g. log-scaled or decayed) and hitting NaN from a division by zero; JSON configs with "NaN"/"Infinity" literals; copying temperature=-1 from another framework meaning 'disabled'.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/57a1042a02080126. Report an issue: GitHub.