sgl-project/sglang · error · RuntimeError

{context} expected 2D positions, got shape={tuple(pos2d.shap

Error message

{context} expected 2D positions, got shape={tuple(pos2d.shape)}.

What it means

Raised by the DFLASH v2 worker's gather helper when the positions tensor passed for a req_to_token gather is not 2D. The masked gather API expects positions laid out as (num_segments_or_rows, positions_per_row) so the mask can be compared elementwise. Any 1D flattened or 3D tensor triggers this internal invariant error.

Source

Thrown at python/sglang/srt/speculative/dflash_worker_v2.py:705

    def clear_cache_pool(self):
        # The target worker owns the shared KV allocator/cache. For the compact
        # sliding-window path, the draft req->token view is rebuilt from committed
        # target state before each draft forward, so there is nothing persistent
        # to flush here.
        pass

    def _gather_req_to_token_masked(
        self,
        *,
        req_to_token: torch.Tensor,
        req_pool_indices: torch.Tensor,
        pos2d: torch.Tensor,
        mask: torch.Tensor,
        context: str,
    ) -> torch.Tensor:
        if pos2d.ndim != 2:
            raise RuntimeError(
                f"{context} expected 2D positions, got shape={tuple(pos2d.shape)}."
            )
        if mask.shape != pos2d.shape:
            raise RuntimeError(
                f"{context} mask/position shape mismatch: {tuple(mask.shape)} vs {tuple(pos2d.shape)}."
            )

        if req_pool_indices.dtype != torch.int64:
            req_pool_indices = req_pool_indices.to(torch.int64)
        if mask.dtype != torch.bool:
            mask = mask.to(torch.bool)

        table_width = int(req_to_token.shape[1])
        if table_width <= 0:
            if bool(mask.any().item()):
                raise RuntimeError(
                    f"{context} req_to_token table is empty but gather mask is non-empty."
                )

View on GitHub (pinned to 0132848349)

Solutions

Inspect the caller of _gather_req_to_token_segments to see what shape it builds for pos2d and restore the (rows, cols) layout
Add a shape assert where positions are constructed to catch divergence earlier
If you modified sglang speculative code, re-check against upstream — this is an internal invariant, not a config knob

Defensive patterns

Strategy: validation

Validate before calling

assert pos2d.ndim == 2, f"positions must be 2D, got {pos2d.ndim}D"

Prevention

Keep positions in (rows, cols) layout end-to-end; avoid ad-hoc flatten/view calls in speculative code
When patching sglang worker internals, add shape asserts at construction sites

When it happens

Trigger: Internal code paths (_gather_req_to_token_segments callers) passing flattened 1D position indices or a batched 3D tensor; almost always a bug in the worker's own batch assembly rather than user config.

Common situations: Refactors of the DFLASH worker's position bookkeeping; changes to how prefill/decode positions are computed; custom speculative schedules feeding non-2D positions.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6509cbb2199de06b. Report an issue: GitHub.