sgl-project/sglang · error · ValueError

padding_side must be 'left' or 'right', got {padding_side}

Error message

padding_side must be 'left' or 'right', got {padding_side}

What it means

LTX-2's pack_text_embeds computes a masked mean over text hidden states and only supports 'left' or 'right' token padding. Any other padding_side string raises this ValueError before masking is applied.

Source

Thrown at python/sglang/multimodal_gen/configs/pipeline_configs/ltx_2.py:74

            A small positive value for numerical stability when performing normalization.

    Returns:
        `torch.Tensor` of shape `(batch_size, seq_len, hidden_dim * num_layers)`:
            Normed and flattened text encoder hidden states.
    """
    batch_size, seq_len, hidden_dim, num_layers = text_hidden_states.shape
    original_dtype = text_hidden_states.dtype
    device = text_hidden_states.device

    # Create padding mask
    token_indices = torch.arange(seq_len, device=device).unsqueeze(0)
    if padding_side == "right":
        mask = token_indices < sequence_lengths[:, None]
    elif padding_side == "left":
        start_indices = seq_len - sequence_lengths[:, None]
        mask = token_indices >= start_indices
    else:
        raise ValueError(f"padding_side must be 'left' or 'right', got {padding_side}")
    mask = mask[:, :, None, None]  # [batch_size, seq_len, 1, 1]

    masked_text_hidden_states = text_hidden_states.masked_fill(~mask, 0.0)
    num_valid_positions = (sequence_lengths * hidden_dim).view(batch_size, 1, 1, 1)
    masked_mean = masked_text_hidden_states.sum(dim=(1, 2), keepdim=True) / (
        num_valid_positions + eps
    )

    x_min = text_hidden_states.masked_fill(~mask, float("inf")).amin(
        dim=(1, 2), keepdim=True
    )
    x_max = text_hidden_states.masked_fill(~mask, float("-inf")).amax(
        dim=(1, 2), keepdim=True
    )

    normalized_hidden_states = (text_hidden_states - masked_mean) / (
        x_max - x_min + eps
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Pass exactly "left" or "right" — LTX-2's Gemma path uses "left"
2. Check tokenizer.padding_side / tokenizer.init_kwargs before forwarding it; normalize to a supported value
3. Default to "left" when the upstream value is None or unexpected

Example fix

# before
side = tokenizer.padding_side  # may be None or unexpected
embeds = pack_text_embeds(hs, seq_lens, padding_side=side)

# after
side = tokenizer.padding_side if tokenizer.padding_side in ("left", "right") else "left"
embeds = pack_text_embeds(hs, seq_lens, padding_side=side)

padding_side = padding_side if padding_side in ("left", "right") else "left"

def is_valid_padding_side(s) -> bool:
    return s in ("left", "right")

except ValueError as e:
    if "padding_side" in str(e):
        embeds = pack_text_embeds(hs, seq_lens, padding_side="left")

When it happens

Trigger: Calling pack_text_embeds(hidden_states, sequence_lengths, padding_side=...) (directly or via _gemma_postprocess_func/_pack_text_embeds) with padding_side of e.g. "both", "LONGEST", "", or None.

Common situations: Forwarding a tokenizer's padding_side value without checking (some tokenizers/configs use other values or None); typos or case differences; custom postprocess functions hardcoding a nonstandard side.

Related errors

• num_inference_steps must be positive, got {steps}
• Unknown serve backend {name!r}. Available values: {available
• k_cache can only be None when only_qv=True
• q can only be None when only_qv=True
• No kernels registered for op {op!r}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d7c53db53f22104. Report an issue: GitHub.