sgl-project/sglang · error · RuntimeError

{context} req_to_token table is empty but gather mask is non

Error message

{context} req_to_token table is empty but gather mask is non-empty.

What it means

Raised when the req_to_token KV-cache mapping table has zero width (no token slots allocated) but the gather mask still selects entries. The worker tolerates an empty table only when nothing needs gathering; requesting tokens from an empty table is contradictory and indicates the memory pool was never allocated or was reset mid-flight.

Source

Thrown at python/sglang/srt/speculative/dflash_worker_v2.py:721

    ) -> torch.Tensor:
        if pos2d.ndim != 2:
            raise RuntimeError(
                f"{context} expected 2D positions, got shape={tuple(pos2d.shape)}."
            )
        if mask.shape != pos2d.shape:
            raise RuntimeError(
                f"{context} mask/position shape mismatch: {tuple(mask.shape)} vs {tuple(pos2d.shape)}."
            )

        if req_pool_indices.dtype != torch.int64:
            req_pool_indices = req_pool_indices.to(torch.int64)
        if mask.dtype != torch.bool:
            mask = mask.to(torch.bool)

        table_width = int(req_to_token.shape[1])
        if table_width <= 0:
            if bool(mask.any().item()):
                raise RuntimeError(
                    f"{context} req_to_token table is empty but gather mask is non-empty."
                )
            return torch.empty((0,), dtype=torch.int64, device=self.device)

        # Only the masked-off rectangular padding can be out of range in the normal
        # ragged-batch case. Replace those don't-care columns with a valid in-range
        # position before the gather so the kernel only sees real positions.
        safe_pos2d = pos2d.masked_fill(~mask, 0)
        return req_to_token[req_pool_indices[:, None], safe_pos2d][mask].to(torch.int64)

    def _gather_req_to_token_segments(
        self,
        *,
        req_to_token: torch.Tensor,
        req_pool_indices: torch.Tensor,
        start: torch.Tensor | None,
        lengths: torch.Tensor,
    ) -> torch.Tensor:

View on GitHub (pinned to 0132848349)

Solutions

1. Check server args affecting req_to_token pool sizing (context_len, max_running_requests, page size) for degenerate zero values
2. If this follows a pool reset/reaallocation, ensure batches are drained before resetting the token pool
3. Report upstream with the full server args — likely an sglang internal bug in pool sizing

if req_to_token.shape[1] <= 0:
    assert not bool(mask.any().item()), "cannot gather from an empty req_to_token pool"

When it happens

Trigger: req_to_token tensor initialized with shape (pool_size, 0) because the token pool sizing computed zero context slots; calling gather after the memory pool was freed/reset while the batch still references tokens. Internal worker state inconsistency.

Common situations: Zero context length / max context misconfiguration causing a degenerate pool; memory-pool reinitialization during running requests; bugs in pool sizing with unusual max_prefill_tokens/context_length settings.

Related errors

• {context} expected 2D positions, got shape={tuple(pos2d.shap
• {context} mask/position shape mismatch: {tuple(mask.shape)}
• DFLASH speculative decoding only supports CUDA and NPU devic
• move_kv_cache is not yet supported for MiniMaxSparseKVPool:
• next_token_logits row count mismatch. Expected {bs * draft_t

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/81264d12521adc6b. Report an issue: GitHub.