sgl-project/sglang · error · RuntimeError

mask/position shape mismatch: vs .

Error message

{context} mask/position shape mismatch: {tuple(mask.shape)} vs {tuple(pos2d.shape)}.

What it means

Raised when the boolean gather mask and the 2D positions tensor have different shapes in the DFLASH worker's req_to_token gather. The mask selects which position entries are real vs padding, so it must be elementwise-aligned with pos2d. A mismatch means the caller built the mask from a different batch layout than the positions.

Solutions

1. Find the caller building both mask and pos2d and derive the mask from pos2d itself (e.g. valid_mask = pos2d >= 0) so they can't diverge
2. Log tuple(mask.shape) and tuple(pos2d.shape) at the call site to identify which layout is stale
3. Diff against upstream sglang if local changes were made to segment gathering

assert mask.shape == pos2d.shape, (
    f"mask {tuple(mask.shape)} != positions {tuple(pos2d.shape)}"
)

When it happens

Trigger: Building the mask from a (bs, draft_token_num) layout while positions were reshaped to (total_tokens, 1), or any caller that computes mask and positions from different sources/sizes. Internal worker bug, not user-facing config.

Common situations: Editing the worker's segment assembly so mask and positions are built in separate loops that diverge; schedule changes (variable draft lengths) updating one but not the other.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

• expected 2D positions, got shape= .
• req_to_token table is empty but gather mask is non-empty.
• candidates and next_token_logits must be on the same…
• DFLASH mask_token_id is outside the target vocab size…
• DFLASH mask_token must be a non-empty string, got

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9fee4d8c2007d98e. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/speculative/dflash_worker_v2.py:709

        # target state before each draft forward, so there is nothing persistent
        # to flush here.
        pass

    def _gather_req_to_token_masked(
        self,
        *,
        req_to_token: torch.Tensor,
        req_pool_indices: torch.Tensor,
        pos2d: torch.Tensor,
        mask: torch.Tensor,
        context: str,
    ) -> torch.Tensor:
        if pos2d.ndim != 2:
            raise RuntimeError(
                f"{context} expected 2D positions, got shape={tuple(pos2d.shape)}."
            )
        if mask.shape != pos2d.shape:
            raise RuntimeError(
                f"{context} mask/position shape mismatch: {tuple(mask.shape)} vs {tuple(pos2d.shape)}."
            )

        if req_pool_indices.dtype != torch.int64:
            req_pool_indices = req_pool_indices.to(torch.int64)
        if mask.dtype != torch.bool:
            mask = mask.to(torch.bool)

        table_width = int(req_to_token.shape[1])
        if table_width <= 0:
            if bool(mask.any().item()):
                raise RuntimeError(
                    f"{context} req_to_token table is empty but gather mask is non-empty."
                )
            return torch.empty((0,), dtype=torch.int64, device=self.device)

        # Only the masked-off rectangular padding can be out of range in the normal
        # ragged-batch case. Replace those don't-care columns with a valid in-range

View on GitHub (pinned to 0132848349)