sgl-project/sglang · error · ValueError

token_tags must cover the full packed sequence ({seq_len}),

Error message

token_tags must cover the full packed sequence ({seq_len}), got {token_tags.shape[0]}.

What it means

When token_tags is provided it must have shape[0] equal to the packed sequence length S so every packed token carries a tag. Mismatched tag length means condition/contrast tagging cannot be aligned to tokens.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:2443

            )
        )
        # max_seqlen_q is set to cu_seqlens[1] (`used`, the real/non-padding
        # row count) by construction -- already a plain host int here, so
        # ring can reuse it as real_seq_len below with no new device sync.
        max_seqlen = int(self._psp_field(psp, "packed_seq_params", "max_seqlen_q"))
        refiner_psp = _required_kwarg(kwargs, "refiner_packed_seq_params")
        refiner_cu = self._psp_field(
            refiner_psp, "refiner_packed_seq_params", "cu_seqlens_q"
        ).to(torch.int32)
        refiner_max = int(
            self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")
        )

        if x.dim() != 3 or x.shape[0] != 1:
            raise ValueError(f"x must be [1, S, C], got {list(x.shape)}")
        seq_len = int(x.shape[1])
        if token_tags is not None and token_tags.shape[0] != seq_len:
            raise ValueError(
                "token_tags must cover the full packed sequence "
                f"({seq_len}), got {token_tags.shape[0]}."
            )
        if inverse_indices.shape[0] != seq_len:
            raise ValueError(
                f"inverse_indices must be [{seq_len}], got {list(inverse_indices.shape)}"
            )
        device = x.device
        if subblock_sparse_query_block_mask is not None and not isinstance(
            subblock_sparse_query_block_mask, torch.Tensor
        ):
            raise ValueError("subblock_sparse_query_block_mask must be a tensor")
        self._resolve_attention_backend_once()

        # Row split is 2D: ring first (an outer, contiguous ring_chunk_len
        # slice of the packed sequence), Ulysses second (an inner slice
        # within this rank's ring chunk). Only Ulysses shards heads inside
        # attention -- ring instead ring-rotates each rank's local KV chunk

View on GitHub (pinned to 0132848349)

Solutions

  1. Rebuild token_tags from the same packing metadata that produced x, ensuring len(tags) == x.shape[1]
  2. Include tags for every packed segment (text + video + audio) in packed order
  3. Add an assert len(token_tags) == x.shape[1] before calling forward

Example fix

// before
model(x=packed, token_tags=video_tags_only, ...)
// after
all_tags = torch.cat([text_tags, video_tags, audio_tags])
assert all_tags.shape[0] == packed.shape[1]
model(x=packed, token_tags=all_tags, ...)
Defensive patterns

Strategy: validation

Validate before calling

assert token_tags is None or token_tags.shape[0] == x.shape[1], (token_tags.shape, x.shape)

Prevention

When it happens

Trigger: Passing token_tags computed for a different sequence (e.g. before padding, or for only the video tokens while text tokens are also packed).

Common situations: Recomputing latents/token counts after padding changes but reusing stale tag tensors; concatenating tag arrays in the wrong order or omitting a modality segment.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f66f004c485db783. Report an issue: GitHub.