sgl-project/sglang · error · ValueError

forward_batch cannot be None

Error message

forward_batch cannot be None

What it means

STA forward reads the live ForwardContext.forward_batch (the current request batch) for batch/timestep inputs; if the context's forward_batch is None it raises ValueError.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/sliding_tile_attn.py:216

        return self.untile(output)

    def forward(
        self,
        q: torch.Tensor,
        k: torch.Tensor,
        v: torch.Tensor,
        attn_metadata: SlidingTileAttentionMetadata,
    ) -> torch.Tensor:
        if self.mask_strategy is None:
            raise ValueError("mask_strategy cannot be None for SlidingTileAttention")
        if self.mask_strategy[0] is None:
            raise ValueError("mask_strategy[0] cannot be None for SlidingTileAttention")

        timestep = attn_metadata.current_timestep
        forward_context: ForwardContext = get_forward_context()
        forward_batch = forward_context.forward_batch
        if forward_batch is None:
            raise ValueError("forward_batch cannot be None")
        # pattern:'.double_blocks.0.attn.impl' or '.single_blocks.0.attn.impl'
        layer_idx = int(self.prefix.split(".")[-3])
        if attn_metadata.STA_param is None or len(attn_metadata.STA_param) <= layer_idx:
            raise ValueError("Invalid STA_param")
        STA_param = attn_metadata.STA_param[layer_idx]

        text_length = q.shape[1] - self.img_seq_length
        has_text = text_length > 0

        query = q.transpose(1, 2).contiguous()
        key = k.transpose(1, 2).contiguous()
        value = v.transpose(1, 2).contiguous()

        head_num = query.size(1)
        sp_group = get_sp_group()
        current_rank = sp_group.rank_in_group
        start_head = current_rank * head_num

View on GitHub (pinned to 0132848349)

Solutions

1. Run the layer inside a normal model forward where ForwardContext has a real forward_batch
2. In tests, install a dummy ForwardContext with a ForwardBatch before calling forward
3. Refactor tests to go through the model runner rather than the raw backend

Example fix

# before
out = sta_impl.forward(q, k, v, meta)  # forward_batch is None
# after
ctx = ForwardContext(forward_batch=make_dummy_forward_batch(q.shape[0]))
set_forward_context(ctx)
out = sta_impl.forward(q, k, v, meta)

ctx = get_forward_context()
assert ctx is not None and ctx.forward_batch is not None, "STA forward must run inside a real forward pass"

def in_forward_pass() -> bool:
    ctx = get_forward_context()
    return ctx is not None and ctx.forward_batch is not None

When it happens

Trigger: Calling sta_backend.forward(...) outside a real model forward pass — e.g. unit tests, benchmarks, or warmup — where get_forward_context() returns a context without a bound forward_batch.

Common situations: Standalone kernel microbenchmarks constructing metadata manually; running layers before the scheduler attached a batch; context cleared between engine steps.

Related errors

• f"seq_len {item} not supported for STA"
• st attn not supported
• SGLANG_DIFFUSION_ATTENTION_CONFIG is not set
• mask_strategy cannot be None for SlidingTileAttention
• mask_strategy[0] cannot be None for SlidingTileAttention

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6aa153bec80af53b. Report an issue: GitHub.