sgl-project/sglang · error · ValueError

Invalid STA_param

Error message

Invalid STA_param

What it means

STA forward parses the layer index from self.prefix (e.g. '.double_blocks.0.attn.impl' -> int(parts[-3])) and indexes attn_metadata.STA_param with it. If STA_param is None or its length <= layer_idx, per-layer parameters are unavailable and forward raises 'Invalid STA_param'.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/sliding_tile_attn.py:220

        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        attn_metadata: SlidingTileAttentionMetadata,
    ) -> torch.Tensor:
        if self.mask_strategy is None:
            raise ValueError("mask_strategy cannot be None for SlidingTileAttention")
        if self.mask_strategy[0] is None:
            raise ValueError("mask_strategy[0] cannot be None for SlidingTileAttention")

        timestep = attn_metadata.current_timestep
        forward_context: ForwardContext = get_forward_context()
        forward_batch = forward_context.forward_batch
        if forward_batch is None:
            raise ValueError("forward_batch cannot be None")
        # pattern:'.double_blocks.0.attn.impl' or '.single_blocks.0.attn.impl'
        layer_idx = int(self.prefix.split(".")[-3])
        if attn_metadata.STA_param is None or len(attn_metadata.STA_param) <= layer_idx:
            raise ValueError("Invalid STA_param")
        STA_param = attn_metadata.STA_param[layer_idx]

        text_length = q.shape[1] - self.img_seq_length
        has_text = text_length > 0

        query = q.transpose(1, 2).contiguous()
        key = k.transpose(1, 2).contiguous()
        value = v.transpose(1, 2).contiguous()

        head_num = query.size(1)
        sp_group = get_sp_group()
        current_rank = sp_group.rank_in_group
        start_head = current_rank * head_num

        # searching or tuning mode
        if len(STA_param) < head_num * sp_group.world_size:
            sparse_attn_hidden_states_all = []
            full_mask_window = STA_param[-1]

View on GitHub (pinned to 0132848349)

Solutions

1. Build SlidingTileAttentionMetadata with STA_param sized to cover every STA layer index of your model
2. Verify self.prefix matches '.double_blocks.{i}.attn.impl' / '.single_blocks.{i}.attn.impl' so layer_idx parses correctly
3. Skip STA (dense fallback) for layers beyond the STA_param range if that is intended

Example fix

# before
meta = SlidingTileAttentionMetadata(..., STA_param=None)
out = impl.forward(q, k, v, meta)  # Invalid STA_param
# after
meta = SlidingTileAttentionMetadata(..., STA_param=sta_params_for_all_layers)  # len > max layer_idx
out = impl.forward(q, k, v, meta)

layer_idx = int(self.prefix.split(".")[-3])
assert attn_metadata.STA_param is not None and len(attn_metadata.STA_param) > layer_idx, "STA_param too short for layer"

def sta_param_ready(meta, layer_idx: int) -> bool:
    return meta.STA_param is not None and len(meta.STA_param) > layer_idx

When it happens

Trigger: Calling forward with attn_metadata.STA_param unset or shorter than layer_idx+1, or with a prefix whose parsed index exceeds the list length.

Common situations: Model variants with more double/single blocks than the STA_param list covers; a prefix not matching the '.blocks.{i}.attn.impl' pattern so parsing yields a wrong index; metadata reused from a different model.

Related errors

• {name} must have dtype torch.int32
• {name} must be on the same device as block sparse tensors
• {name} must live on CUDA
• kv-canary: SWA slot {slot} is outside full_to_swa_index_mapp
• Image index {index} out of range

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d4b368ec3d11536. Report an issue: GitHub.