sgl-project/sglang · error · ValueError

block_token_tags must cover the rank-local packed sequence (

Error message

block_token_tags must cover the rank-local packed sequence ({local_seq_len}), got {block_token_tags.shape[0]}.

What it means

When block_token_tags are supplied directly (not derived from global token_tags), they must cover exactly the rank-local packed sequence length local_seq_len after sequence-parallel sharding. A mismatch means tags were not sliced to this rank's chunk.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:2540

            local_embedding_layout=kwargs.get("local_embedding_layout"),
        )
        self.release_mps_non_layer_weights(*_MPS_EMBED_WEIGHT_PREFIXES)
        # request-step AdaLN input shared by all blocks
        adaln_input = (
            t_emb
            if self.adaln_t_table is not None
            else nn.functional.silu(t_emb).to(_BF16_DTYPE)
        )
        inverse_indices = inverse_indices.to(device)
        block_inverse = inverse_indices[row_start:row_stop]
        if block_token_tags is None:
            assert token_tags is not None
            token_tags = token_tags.to(device)
            block_token_tags = token_tags[row_start:row_stop].clamp(min=0)
        else:
            block_token_tags = block_token_tags.to(device)
            if block_token_tags.shape[0] != local_seq_len:
                raise ValueError(
                    "block_token_tags must cover the rank-local packed sequence "
                    f"({local_seq_len}), got {block_token_tags.shape[0]}."
                )
        block_combined = kwargs.get("block_combined_indices")
        if block_combined is None:
            block_combined = torch.add(
                block_token_tags,
                block_inverse,
                alpha=MINIMAX_H3_ADALN_MODALITY_NUM,
            )

        hidden = decoder_input
        cu_seqlens = cu_seqlens.to(device)
        block_adaln_params = None
        adaln_cache_plan_index = None
        if self.adaln_cache is not None:
            adaln_cache_plan_index = self.adaln_cache.lookup(
                unique_timesteps.view(-1).to(device)

View on GitHub (pinned to 0132848349)

Solutions

1. Pass global token_tags instead and let forward slice rows itself (the token_tags branch in the source)
2. If supplying block_token_tags, slice to the rank-local chunk length: tags[row_start:row_stop] matching local_seq_len
3. Recompute local_seq_len = seq_len // (ulysses_ws * ring_ws) and verify tag length equals it

Example fix

// before
model(..., block_token_tags=all_tags)  # full length under SP
// after
model(..., token_tags=all_tags)         # let the model slice per-rank

local_len = x.shape[1] // (ulysses_ws * ring_ws)
assert block_token_tags is None or block_token_tags.shape[0] == local_len

When it happens

Trigger: Passing full-sequence (global) token_tags as block_token_tags under SP>1, or slicing with the wrong row_start/row_stop after changing the SP degree.

Common situations: Custom pipelines that precompute block tags without accounting for ring/Ulysses sharding; SP config changed between tag generation and forward.

Related errors

• token_tags must cover the full packed sequence ({seq_len}),
• q, k, and v must have the same 3D shape
• fl2va requires first_frame, last_frame, or both
• ref2va requires at least one of reference_image, reference_v
• t2va takes no conditioning inputs; pick another task

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0b796cfad9cf22ab. Report an issue: GitHub.