sgl-project/sglang · error · ValueError

max_new_tokens must leave room for input: max_new_tokens={ma

Error message

max_new_tokens must leave room for input: max_new_tokens={max_new_tokens}, seq={seq}

What it means

Raised by _build_video_cfg when max_new_tokens >= seq. The processor reserves max_new_tokens of the sequence budget for generation, so it must be strictly less than seq to leave room for the video input tokens (seq_length = seq - max_new_tokens).

Source

Thrown at python/sglang/srt/multimodal/processors/dots_note_omni.py:53

_VIDEO_TOKEN_RE = re.compile(r"(<image_\d+>|<audio_\d+>)")
_EXPANDED_VIDEO_MEDIA_RE = re.compile(
    r"<\|sglang_dots_video_(?P<video>\d+)_(?P<modality>image|audio)_(?P<item>\d+)\|>"
)


def _build_video_cfg(
    *,
    seq: int,
    audio_cap: float,
    audio_sr: int,
    max_new_tokens: int,
) -> dict[str, Any]:
    if seq <= 0:
        raise ValueError(f"seq must be positive, got {seq}")
    if max_new_tokens < 0:
        raise ValueError(f"max_new_tokens must be non-negative, got {max_new_tokens}")
    if max_new_tokens >= seq:
        raise ValueError(
            "max_new_tokens must leave room for input: "
            f"max_new_tokens={max_new_tokens}, seq={seq}"
        )
    if audio_cap < 0:
        raise ValueError(f"audio_cap must be non-negative, got {audio_cap}")
    if audio_sr <= 0:
        raise ValueError(f"audio_sr must be positive, got {audio_sr}")

    return {
        "process_audio": audio_cap > 0,
        "seq_length": seq - max_new_tokens,
        "reserve_interleave": True,
        "audio_token_ratio_cap": float(audio_cap),
        "audio_sample_rate": int(audio_sr),
        "video_jpeg_quality": int(os.environ.get("XHS_VIDEO_JPEG_QUALITY", "85")),
    }

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce max_new_tokens so it is strictly less than seq
2. Or raise video_config.seq (bounded by the model's context length) so seq > max_new_tokens
3. Verify seq_length = seq - max_new_tokens is large enough for the expected video token count

Example fix

// before
video_config = {"seq": 4096}
sampling_params = {"max_new_tokens": 4096}
// after
video_config = {"seq": 4096}
sampling_params = {"max_new_tokens": 2048}

seq = cfg.get('seq', 131072)
mnt = (sampling_params or {}).get('max_new_tokens') or 0
assert mnt < seq, f'max_new_tokens ({mnt}) must be < seq ({seq})'

def budget_ok(cfg: dict, sp: dict) -> bool:
    seq = cfg.get('seq', 131072)
    mnt = sp.get('max_new_tokens') or 0
    return 0 <= mnt < seq

When it happens

Trigger: Passing sampling_params.max_new_tokens equal to or larger than video_config.seq (default 131072), e.g. max_new_tokens=200000 with seq=131072, or seq=4096 with max_new_tokens=4096.

Common situations: Users set a large max_new_tokens (or a whole context budget) while leaving seq at the default, or shrink seq for memory reasons without adjusting max_new_tokens.

Related errors

• QwenImage RoPE text cache overflow before denoising: require
• seq must be positive, got {seq}
• max_new_tokens must be non-negative, got {max_new_tokens}
• audio_cap must be non-negative, got {audio_cap}
• audio_sr must be positive, got {audio_sr}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f3990ad4bde4fbcc. Report an issue: GitHub.