{"record":{"id":"1f8492361f594716","repo":"sgl-project/sglang","slug":"model-vocab-size-vocab-size-exceeds-mm-pad-shi","errorCode":null,"errorMessage":"Model vocab_size ({vocab_size}) exceeds MM_PAD_SHIFT_VALUE ({MM_PAD_SHIFT_VALUE}). MM padValues may overlap with valid token IDs. Please increase MM_PAD_SHIFT_VALUE in schedule_batch.py.","messagePattern":"Model vocab_size \\((.+?)\\) exceeds MM_PAD_SHIFT_VALUE \\((.+?)\\)\\. MM padValues may overlap with valid token IDs\\. Please increase MM_PAD_SHIFT_VALUE in schedule_batch\\.py\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/managers/schedule_batch.py","lineNumber":209,"sourceCode":"\n\ndef get_batch_return_hidden_states_mode(reqs: List[Req]) -> CaptureHiddenMode:\n    mode = CaptureHiddenMode.NULL\n    for req in reqs:\n        mode = max(mode, req.return_hidden_states_mode)\n    return mode\n\n\ndef need_return_hidden_states(\n    return_hidden_states: Union[List[ReturnHiddenStatesMode], ReturnHiddenStatesMode],\n) -> bool:\n    return get_request_return_hidden_states_mode(return_hidden_states).need_capture()\n\n\n@lru_cache(maxsize=1)\ndef sanity_check_mm_pad_shift_value(vocab_size: int) -> None:\n    if vocab_size > MM_PAD_SHIFT_VALUE:\n        raise ValueError(\n            f\"Model vocab_size ({vocab_size}) exceeds MM_PAD_SHIFT_VALUE ({MM_PAD_SHIFT_VALUE}). \"\n            f\"MM pad_values may overlap with valid token IDs. \"\n            f\"Please increase MM_PAD_SHIFT_VALUE in schedule_batch.py.\"\n        )\n\n\ndef split_cached_prefix_by_tier(\n    prefix_len: int, host_hit_len: int, storage_hit_len: int\n) -> tuple[int, int, int]:\n    \"\"\"Split a request's cached prefix into (device, host, storage) tokens.\n\n    prefix_len is len(prefix_indices) AFTER host load-back, so it contains the\n    host-loaded portion; host_hit_len in turn contains the storage-prefetched\n    portion (storage is clamped to it to handle edge cases).\n    \"\"\"\n    storage = min(host_hit_len, storage_hit_len)\n    host = host_hit_len - storage\n    device = max(0, prefix_len - host_hit_len)","sourceCodeStart":191,"sourceCodeEnd":227,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/schedule_batch.py#L191-L227","documentation":"MM pad token offsets are computed as token_id + MM_PAD_SHIFT_VALUE; if the model's vocab_size exceeds that shift constant, shifted pad values could collide with real token IDs and silently corrupt tokenization, so a startup sanity check fails.","triggerScenarios":"Loading a model with a very large vocabulary on a build where MM_PAD_SHIFT_VALUE is smaller than vocab_size.","commonSituations":"New large-vocab models (e.g. 250k+ tokenizer) on older SGLang; locally lowering MM_PAD_SHIFT_VALUE for experiments.","solutions":["Upgrade SGLang to a release where MM_PAD_SHIFT_VALUE was raised for the model","Edit MM_PAD_SHIFT_VALUE in python/sglang/srt/managers/schedule_batch.py to exceed vocab_size","Serve that model on a deployment without MM pad shifting if applicable"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from sglang.srt.managers.schedule_batch import sanity_check_mm_pad_shift_value\nsanity_check_mm_pad_shift_value(model_config.vocab_size)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Upgrade sglang before deploying very-large-vocab VLMs","Never lower MM_PAD_SHIFT_VALUE below vocab_size"],"tags":["sglang","multimodal","vocab-size","constant-mismatch","startup"],"backgroundTag":"constant-out-of-range-for-model","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}