sgl-project/sglang · critical · ValueError

Model vocab_size ({vocab_size}) exceeds MM_PAD_SHIFT_VALUE (

Error message

Model vocab_size ({vocab_size}) exceeds MM_PAD_SHIFT_VALUE ({MM_PAD_SHIFT_VALUE}). MM padValues may overlap with valid token IDs. Please increase MM_PAD_SHIFT_VALUE in schedule_batch.py.

What it means

MM pad token offsets are computed as token_id + MM_PAD_SHIFT_VALUE; if the model's vocab_size exceeds that shift constant, shifted pad values could collide with real token IDs and silently corrupt tokenization, so a startup sanity check fails.

Source

Thrown at python/sglang/srt/managers/schedule_batch.py:209

def get_batch_return_hidden_states_mode(reqs: List[Req]) -> CaptureHiddenMode:
    mode = CaptureHiddenMode.NULL
    for req in reqs:
        mode = max(mode, req.return_hidden_states_mode)
    return mode


def need_return_hidden_states(
    return_hidden_states: Union[List[ReturnHiddenStatesMode], ReturnHiddenStatesMode],
) -> bool:
    return get_request_return_hidden_states_mode(return_hidden_states).need_capture()


@lru_cache(maxsize=1)
def sanity_check_mm_pad_shift_value(vocab_size: int) -> None:
    if vocab_size > MM_PAD_SHIFT_VALUE:
        raise ValueError(
            f"Model vocab_size ({vocab_size}) exceeds MM_PAD_SHIFT_VALUE ({MM_PAD_SHIFT_VALUE}). "
            f"MM pad_values may overlap with valid token IDs. "
            f"Please increase MM_PAD_SHIFT_VALUE in schedule_batch.py."
        )


def split_cached_prefix_by_tier(
    prefix_len: int, host_hit_len: int, storage_hit_len: int
) -> tuple[int, int, int]:
    """Split a request's cached prefix into (device, host, storage) tokens.

    prefix_len is len(prefix_indices) AFTER host load-back, so it contains the
    host-loaded portion; host_hit_len in turn contains the storage-prefetched
    portion (storage is clamped to it to handle edge cases).
    """
    storage = min(host_hit_len, storage_hit_len)
    host = host_hit_len - storage
    device = max(0, prefix_len - host_hit_len)

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade SGLang to a release where MM_PAD_SHIFT_VALUE was raised for the model
2. Edit MM_PAD_SHIFT_VALUE in python/sglang/srt/managers/schedule_batch.py to exceed vocab_size
3. Serve that model on a deployment without MM pad shifting if applicable

from sglang.srt.managers.schedule_batch import sanity_check_mm_pad_shift_value
sanity_check_mm_pad_shift_value(model_config.vocab_size)

When it happens

Trigger: Loading a model with a very large vocabulary on a build where MM_PAD_SHIFT_VALUE is smaller than vocab_size.

Common situations: New large-vocab models (e.g. 250k+ tokenizer) on older SGLang; locally lowering MM_PAD_SHIFT_VALUE for experiments.

Related errors

• SGLANG_RUST_SERVER=1: no native Rust MM pipeline for model_t
• {selection_error}{component_suffix}
• No compatible attention backend is available{component_suffi
• SGLANG_RUST_SERVER is not supported with the offline Engine
• Multi-node weight cache daemons (nnodes > 1) require --dist-

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1f8492361f594716. Report an issue: GitHub.