{"record":{"id":"81264d12521adc6b","repo":"sgl-project/sglang","slug":"context-req-to-token-table-is-empty-but-gather-m","errorCode":null,"errorMessage":"{context} req_to_token table is empty but gather mask is non-empty.","messagePattern":"(.+?) req_to_token table is empty but gather mask is non-empty\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/dflash_worker_v2.py","lineNumber":721,"sourceCode":"    ) -> torch.Tensor:\n        if pos2d.ndim != 2:\n            raise RuntimeError(\n                f\"{context} expected 2D positions, got shape={tuple(pos2d.shape)}.\"\n            )\n        if mask.shape != pos2d.shape:\n            raise RuntimeError(\n                f\"{context} mask/position shape mismatch: {tuple(mask.shape)} vs {tuple(pos2d.shape)}.\"\n            )\n\n        if req_pool_indices.dtype != torch.int64:\n            req_pool_indices = req_pool_indices.to(torch.int64)\n        if mask.dtype != torch.bool:\n            mask = mask.to(torch.bool)\n\n        table_width = int(req_to_token.shape[1])\n        if table_width <= 0:\n            if bool(mask.any().item()):\n                raise RuntimeError(\n                    f\"{context} req_to_token table is empty but gather mask is non-empty.\"\n                )\n            return torch.empty((0,), dtype=torch.int64, device=self.device)\n\n        # Only the masked-off rectangular padding can be out of range in the normal\n        # ragged-batch case. Replace those don't-care columns with a valid in-range\n        # position before the gather so the kernel only sees real positions.\n        safe_pos2d = pos2d.masked_fill(~mask, 0)\n        return req_to_token[req_pool_indices[:, None], safe_pos2d][mask].to(torch.int64)\n\n    def _gather_req_to_token_segments(\n        self,\n        *,\n        req_to_token: torch.Tensor,\n        req_pool_indices: torch.Tensor,\n        start: torch.Tensor | None,\n        lengths: torch.Tensor,\n    ) -> torch.Tensor:","sourceCodeStart":703,"sourceCodeEnd":739,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/dflash_worker_v2.py#L703-L739","documentation":"Raised when the req_to_token KV-cache mapping table has zero width (no token slots allocated) but the gather mask still selects entries. The worker tolerates an empty table only when nothing needs gathering; requesting tokens from an empty table is contradictory and indicates the memory pool was never allocated or was reset mid-flight.","triggerScenarios":"req_to_token tensor initialized with shape (pool_size, 0) because the token pool sizing computed zero context slots; calling gather after the memory pool was freed/reset while the batch still references tokens. Internal worker state inconsistency.","commonSituations":"Zero context length / max context misconfiguration causing a degenerate pool; memory-pool reinitialization during running requests; bugs in pool sizing with unusual max_prefill_tokens/context_length settings.","solutions":["Check server args affecting req_to_token pool sizing (context_len, max_running_requests, page size) for degenerate zero values","If this follows a pool reset/reaallocation, ensure batches are drained before resetting the token pool","Report upstream with the full server args — likely an sglang internal bug in pool sizing"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if req_to_token.shape[1] <= 0:\n    assert not bool(mask.any().item()), \"cannot gather from an empty req_to_token pool\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Sanity-check pool sizing: req_to_token width should exceed max context length per request","Never reset/reallocate the token pool while requests are in flight"],"tags":["sglang","speculative-decoding","dflash","kv-cache","memory-pool","internal-invariant"],"backgroundTag":"kv-cache-pool-invalid-state","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}