{"record":{"id":"f3990ad4bde4fbcc","repo":"sgl-project/sglang","slug":"max-new-tokens-must-leave-room-for-input-max-new","errorCode":null,"errorMessage":"max_new_tokens must leave room for input: max_new_tokens={max_new_tokens}, seq={seq}","messagePattern":"max_new_tokens must leave room for input: max_new_tokens=(.+?), seq=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/processors/dots_note_omni.py","lineNumber":53,"sourceCode":"_VIDEO_TOKEN_RE = re.compile(r\"(<image_\\d+>|<audio_\\d+>)\")\n_EXPANDED_VIDEO_MEDIA_RE = re.compile(\n    r\"<\\|sglang_dots_video_(?P<video>\\d+)_(?P<modality>image|audio)_(?P<item>\\d+)\\|>\"\n)\n\n\ndef _build_video_cfg(\n    *,\n    seq: int,\n    audio_cap: float,\n    audio_sr: int,\n    max_new_tokens: int,\n) -> dict[str, Any]:\n    if seq <= 0:\n        raise ValueError(f\"seq must be positive, got {seq}\")\n    if max_new_tokens < 0:\n        raise ValueError(f\"max_new_tokens must be non-negative, got {max_new_tokens}\")\n    if max_new_tokens >= seq:\n        raise ValueError(\n            \"max_new_tokens must leave room for input: \"\n            f\"max_new_tokens={max_new_tokens}, seq={seq}\"\n        )\n    if audio_cap < 0:\n        raise ValueError(f\"audio_cap must be non-negative, got {audio_cap}\")\n    if audio_sr <= 0:\n        raise ValueError(f\"audio_sr must be positive, got {audio_sr}\")\n\n    return {\n        \"process_audio\": audio_cap > 0,\n        \"seq_length\": seq - max_new_tokens,\n        \"reserve_interleave\": True,\n        \"audio_token_ratio_cap\": float(audio_cap),\n        \"audio_sample_rate\": int(audio_sr),\n        \"video_jpeg_quality\": int(os.environ.get(\"XHS_VIDEO_JPEG_QUALITY\", \"85\")),\n    }\n\n","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/processors/dots_note_omni.py#L35-L71","documentation":"Raised by _build_video_cfg when max_new_tokens >= seq. The processor reserves max_new_tokens of the sequence budget for generation, so it must be strictly less than seq to leave room for the video input tokens (seq_length = seq - max_new_tokens).","triggerScenarios":"Passing sampling_params.max_new_tokens equal to or larger than video_config.seq (default 131072), e.g. max_new_tokens=200000 with seq=131072, or seq=4096 with max_new_tokens=4096.","commonSituations":"Users set a large max_new_tokens (or a whole context budget) while leaving seq at the default, or shrink seq for memory reasons without adjusting max_new_tokens.","solutions":["Reduce max_new_tokens so it is strictly less than seq","Or raise video_config.seq (bounded by the model's context length) so seq > max_new_tokens","Verify seq_length = seq - max_new_tokens is large enough for the expected video token count"],"exampleFix":"// before\nvideo_config = {\"seq\": 4096}\nsampling_params = {\"max_new_tokens\": 4096}\n// after\nvideo_config = {\"seq\": 4096}\nsampling_params = {\"max_new_tokens\": 2048}","handlingStrategy":"validation","validationCode":"seq = cfg.get('seq', 131072)\nmnt = (sampling_params or {}).get('max_new_tokens') or 0\nassert mnt < seq, f'max_new_tokens ({mnt}) must be < seq ({seq})'","typeGuard":"def budget_ok(cfg: dict, sp: dict) -> bool:\n    seq = cfg.get('seq', 131072)\n    mnt = sp.get('max_new_tokens') or 0\n    return 0 <= mnt < seq","tryCatchPattern":null,"preventionTips":["Never set max_new_tokens equal to the full context/seq budget","Keep a margin: seq - max_new_tokens >= expected media token count"],"tags":["multimodal","sequence-length","valueerror"],"backgroundTag":"context-length-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}