sgl-project/sglang · error · ValueError

mask_strategy[0] cannot be None for SlidingTileAttention

Error message

mask_strategy[0] cannot be None for SlidingTileAttention

What it means

Even when a mask strategy exists, its first timestep entry (mask_strategy[0]) is required by STA forward to derive per-timestep masks; if the loaded JSON has null at index 0 the backend raises ValueError.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/sliding_tile_attn.py:210

    def postprocess_output(
        self,
        output: torch.Tensor,
        attn_metadata: SlidingTileAttentionMetadata,
    ) -> torch.Tensor:
        return self.untile(output)

    def forward(
        self,
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        attn_metadata: SlidingTileAttentionMetadata,
    ) -> torch.Tensor:
        if self.mask_strategy is None:
            raise ValueError("mask_strategy cannot be None for SlidingTileAttention")
        if self.mask_strategy[0] is None:
            raise ValueError("mask_strategy[0] cannot be None for SlidingTileAttention")

        timestep = attn_metadata.current_timestep
        forward_context: ForwardContext = get_forward_context()
        forward_batch = forward_context.forward_batch
        if forward_batch is None:
            raise ValueError("forward_batch cannot be None")
        # pattern:'.double_blocks.0.attn.impl' or '.single_blocks.0.attn.impl'
        layer_idx = int(self.prefix.split(".")[-3])
        if attn_metadata.STA_param is None or len(attn_metadata.STA_param) <= layer_idx:
            raise ValueError("Invalid STA_param")
        STA_param = attn_metadata.STA_param[layer_idx]

        text_length = q.shape[1] - self.img_seq_length
        has_text = text_length > 0

        query = q.transpose(1, 2).contiguous()
        key = k.transpose(1, 2).contiguous()
        value = v.transpose(1, 2).contiguous()

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate the mask strategy file so entry 0 is a valid mask list
2. Use the official mask strategy shipped with the model checkpoint
3. Validate the JSON right after load (reject null first entry) and fall back to another backend if invalid

Example fix

# before
# mask.json: {"0": null, "1": [...]} -> ValueError at forward
# after
ms = json.load(open(mask_strategy_file_path))
assert ms[0] is not None, 'mask_strategy[0] must exist'
backend.mask_strategy = dict_to_3d_list(ms)

strategy = json.load(open(mask_strategy_file_path))
assert strategy and strategy[0] is not None, "mask_strategy[0] missing in mask JSON"

def valid_mask_strategy(ms) -> bool:
    return ms is not None and len(ms) > 0 and ms[0] is not None

When it happens

Trigger: Calling STA forward with a mask_strategy JSON whose first entry is null/missing.

Common situations: Reusing another model's mask_strategy.json; hand-edited or partially generated strategy files; template JSON with null placeholders.

Related errors

• mask_strategy cannot be None for SlidingTileAttention
• v_cache must be provided
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True
• `dt_bias` must have {HV * K} elements (got {dt_bias.numel()}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/eb163cc0245f101d. Report an issue: GitHub.