vllm-project/vllm · error · SamplingParamsError

{parameter} must be a finite number, got {value}

Error message

{parameter} must be a finite number, got {value}

What it means

`validate_resolved_sampling_params` (rust/src/text/src/lower/sampling.rs:59-75) applies a finiteness check to `temperature` and `repetition_penalty`; NaN or +/-inf produces `NotFinite`. Other float params are caught by their range checks instead. This mirrors Python vLLM's sampling-param validation and is a request-validation error.

Source

Thrown at rust/src/text/src/lower/sampling.rs:9

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error;
use vllm_engine_core_client::protocol::sampling::EngineCoreSamplingParams;

#[derive(Debug, Error, PartialEq)]
pub enum SamplingParamsError {
    #[error("{parameter} must be a finite number, got {value}")]
    NotFinite { parameter: &'static str, value: f32 },
    #[error("{parameter} must be in {expected}, got {value}")]
    OutOfRange {
        parameter: &'static str,
        value: f32,
        expected: &'static str,
    },
}

fn validate_frequency_penalty(value: f32) -> Result<(), SamplingParamsError> {
    validate_closed_range("frequency_penalty", value, -2.0, 2.0, "[-2, 2]")
}

fn validate_presence_penalty(value: f32) -> Result<(), SamplingParamsError> {
    validate_closed_range("presence_penalty", value, -2.0, 2.0, "[-2, 2]")
}

fn validate_temperature(value: f32) -> Result<(), SamplingParamsError> {

View on GitHub (pinned to c794754062)

Solutions

1. Sanitize floats before sending: reject or default NaN/inf values
2. Compute temperature with clamping, e.g. f32::clamp(0.0, 2.0) after arithmetic

Example fix

// before
let temp = raw_score / count; // may be NaN
req.temperature = Some(temp);

// after
let temp = (raw_score / count).clamp(0.0, 2.0);
req.temperature = if temp.is_finite() { Some(temp) } else { None };

for (name, v) in [("temperature", temperature), ("repetition_penalty", repetition_penalty)] {
    if let Some(v) = v {
        assert!(v.is_finite(), "{name} must be finite, got {v}");
    }
}

fn is_not_finite(e: &Error) -> bool {
    matches!(e, Error::SamplingParams(SamplingParamsError::NotFinite { .. }))
}

match err {
    Error::SamplingParams(SamplingParamsError::NotFinite { parameter, .. }) =>
        bad_request(format!("{parameter} was NaN/inf; send a finite value or omit it")),
    _ => /* ... */
}

When it happens

Trigger: Sending `temperature: NaN` / `Infinity` or `repetition_penalty: inf` (e.g. JSON `NaN`/`Infinity` literals or Rust f32::NAN) in a request; defaults are applied first, so unset fields never trigger it.

Common situations: Client-side math producing NaN (0.0/0.0, log(0) scaling) and passing it straight into temperature; JSON encoders that emit non-standard NaN/Infinity tokens; porting code from runtimes where NaN silently means 'default'.

Related errors

• `min_tokens` must be less than or equal to `max_tokens`, got
• `thinking_token_budget` must be a non-negative integer or -1
• {parameter} must be in {expected}, got {value}
• max_logprobs must be non-negative or -1, got {}
• data parallel size must be at least 1

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/64889a5a472aed43. Report an issue: GitHub.