sgl-project/sglang · error · ValueError

max_new_tokens must be non-negative, got {max_new_tokens}

Error message

max_new_tokens must be non-negative, got {max_new_tokens}

What it means

Raised by _build_video_cfg when max_new_tokens is negative. max_new_tokens is carved out of the seq budget for generation; a negative value is treated as a caller bug and rejected before the config dict is built.

Source

Thrown at python/sglang/srt/multimodal/processors/dots_note_omni.py:51

logger = logging.getLogger(__name__)

_VIDEO_TOKEN_RE = re.compile(r"(<image_\d+>|<audio_\d+>)")
_EXPANDED_VIDEO_MEDIA_RE = re.compile(
    r"<\|sglang_dots_video_(?P<video>\d+)_(?P<modality>image|audio)_(?P<item>\d+)\|>"
)


def _build_video_cfg(
    *,
    seq: int,
    audio_cap: float,
    audio_sr: int,
    max_new_tokens: int,
) -> dict[str, Any]:
    if seq <= 0:
        raise ValueError(f"seq must be positive, got {seq}")
    if max_new_tokens < 0:
        raise ValueError(f"max_new_tokens must be non-negative, got {max_new_tokens}")
    if max_new_tokens >= seq:
        raise ValueError(
            "max_new_tokens must leave room for input: "
            f"max_new_tokens={max_new_tokens}, seq={seq}"
        )
    if audio_cap < 0:
        raise ValueError(f"audio_cap must be non-negative, got {audio_cap}")
    if audio_sr <= 0:
        raise ValueError(f"audio_sr must be positive, got {audio_sr}")

    return {
        "process_audio": audio_cap > 0,
        "seq_length": seq - max_new_tokens,
        "reserve_interleave": True,
        "audio_token_ratio_cap": float(audio_cap),
        "audio_sample_rate": int(audio_sr),
        "video_jpeg_quality": int(os.environ.get("XHS_VIDEO_JPEG_QUALITY", "85")),
    }

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure sampling_params.max_new_tokens is >= 0 (use 0 or omit it if you don't want reserved generation budget)
2. Compute max_new_tokens as max(0, budget - prompt_len) on the client
3. Validate sampling params before submitting the request

Example fix

// before
sampling_params = {"max_new_tokens": budget - prompt_tokens}
// after
sampling_params = {"max_new_tokens": max(0, budget - prompt_tokens)}

mnt = (sampling_params or {}).get('max_new_tokens') or 0
if mnt < 0:
    raise ValueError('max_new_tokens must be >= 0')

def valid_max_new_tokens(sp: dict) -> bool:
    mnt = sp.get('max_new_tokens') or 0
    return isinstance(mnt, int) and mnt >= 0

When it happens

Trigger: Passing a negative max_new_tokens via sampling_params in a video request, e.g. sampling_params={"max_new_tokens": -5}. Note the processor reads it with `sampling_params.get("max_new_tokens") or 0`, so explicit negatives (not None/0) reach the check.

Common situations: Client code computes max_new_tokens as a difference (e.g. requested_len - prompt_len) that goes negative for long prompts and forwards it anyway.

Related errors

• Dots note omni video preprocessing requires one request's sa
• seq must be positive, got {seq}
• max_new_tokens must leave room for input: max_new_tokens={ma
• audio_cap must be non-negative, got {audio_cap}
• audio_sr must be positive, got {audio_sr}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7431dfff57864220. Report an issue: GitHub.