sgl-project/sglang · error · ValueError

mask_strategy cannot be None for SlidingTileAttention

Error message

mask_strategy cannot be None for SlidingTileAttention

What it means

STA forward needs a loaded timestep-indexed mask strategy (loaded in __init__ from the mask JSON). If self.mask_strategy is None the tile masks cannot be selected for the current timestep, so forward refuses to run.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/sliding_tile_attn.py:208

        )
        return self.tile(qkv)

    def postprocess_output(
        self,
        output: torch.Tensor,
        attn_metadata: SlidingTileAttentionMetadata,
    ) -> torch.Tensor:
        return self.untile(output)

    def forward(
        self,
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        attn_metadata: SlidingTileAttentionMetadata,
    ) -> torch.Tensor:
        if self.mask_strategy is None:
            raise ValueError("mask_strategy cannot be None for SlidingTileAttention")
        if self.mask_strategy[0] is None:
            raise ValueError("mask_strategy[0] cannot be None for SlidingTileAttention")

        timestep = attn_metadata.current_timestep
        forward_context: ForwardContext = get_forward_context()
        forward_batch = forward_context.forward_batch
        if forward_batch is None:
            raise ValueError("forward_batch cannot be None")
        # pattern:'.double_blocks.0.attn.impl' or '.single_blocks.0.attn.impl'
        layer_idx = int(self.prefix.split(".")[-3])
        if attn_metadata.STA_param is None or len(attn_metadata.STA_param) <= layer_idx:
            raise ValueError("Invalid STA_param")
        STA_param = attn_metadata.STA_param[layer_idx]

        text_length = q.shape[1] - self.img_seq_length
        has_text = text_length > 0

        query = q.transpose(1, 2).contiguous()

View on GitHub (pinned to 0132848349)

Solutions

Ensure the backend is constructed normally so mask_strategy loads from the JSON
If it is None, re-run the load: self.mask_strategy = dict_to_3d_list(json.load(open(path)))
Guard at init: fail fast if the mask file cannot be loaded rather than at forward time

Example fix

# before
out = sta_backend.forward(q, k, v, meta)  # mask_strategy is None
# after
if sta_backend.mask_strategy is None:
    sta_backend.mask_strategy = dict_to_3d_list(json.load(open(mask_strategy_path)))
out = sta_backend.forward(q, k, v, meta)

Defensive patterns

Strategy: validation

Validate before calling

assert backend.mask_strategy is not None, "mask_strategy not loaded; check mask file path in __init__"

Type guard

def sta_ready(b) -> bool:
    return getattr(b, "mask_strategy", None) is not None

Try / catch

try:
    out = b.forward(q, k, v, meta)
except ValueError as e:
    if "mask_strategy" in str(e):
        b.mask_strategy = dict_to_3d_list(json.load(open(mask_path)))
        out = b.forward(q, k, v, meta)

Prevention

Ensure backends are constructed via their real __init__ (no mocking that drops state)
Re-validate state after deepcopy/serialization

When it happens

Trigger: Calling forward on a SlidingTileAttentionBackend whose mask_strategy attribute is None — instance bypassed __init__, or the attribute was cleared/reset.

Common situations: Mocking/deepcopying backends in tests that drop the attribute; partial init after a swallowed config-load error; serialization frameworks skipping large list attributes.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c85599949f53c7b5. Report an issue: GitHub.