{"record":{"id":"6faafd236a932ffc","repo":"sgl-project/sglang","slug":"forward-batch-with-seq-lens-is-required-for-topk-r","errorCode":null,"errorMessage":"forward_batch with seq_lens is required for TopK retrieval","messagePattern":"forward_batch with seq_lens is required for TopK retrieval","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/sparsity/algorithms/base_algorithm.py","lineNumber":282,"sourceCode":"        queries: torch.Tensor,\n        layer_id: int,\n        req_pool_indices: torch.Tensor,\n        sparse_mask: torch.Tensor,\n        **kwargs,\n    ) -> tuple:\n        \"\"\"\n        Default TopK retrieval: score-based selection + recent pages.\n        Subclasses can override for query-dependent retrieval.\n\n        TODO:\n            1. Using triton kernel to speed up this function\n            2. Support CUDA Graph\n        \"\"\"\n        bs, device = queries.shape[0], queries.device\n\n        seq_lens_source = kwargs.get(\"forward_batch\", None)\n        if seq_lens_source is None or not hasattr(seq_lens_source, \"seq_lens\"):\n            raise ValueError(\n                \"forward_batch with seq_lens is required for TopK retrieval\"\n            )\n        seq_lens = seq_lens_source.seq_lens.to(device)\n\n        req_to_token = self.req_to_token_pool.req_to_token\n        max_req_tokens = req_to_token.shape[1]\n\n        per_request_indices = []\n        per_request_lengths = []\n\n        for i in range(bs):\n            if not sparse_mask[i]:\n                per_request_indices.append(\n                    torch.empty(0, device=device, dtype=torch.int32)\n                )\n                per_request_lengths.append(0)\n                continue\n","sourceCodeStart":264,"sourceCodeEnd":300,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/sparsity/algorithms/base_algorithm.py#L264-L300","documentation":"BaseSparseAlgorithm.retrieve_topk requires the forward_batch keyword argument carrying seq_lens, because Top-K retrieval must gather per-request KV locations from req_to_token using each sequence's length. Without it there is no way to bound the searchable token range per request.","triggerScenarios":"Calling retrieve_topk(queries, ...) without kwargs['forward_batch'], or passing a forward_batch object that lacks a seq_lens attribute (e.g. a mocked or partially-built batch).","commonSituations":"Integrating a sparse-attention retrieval algorithm in a custom model forward that forgot to forward the batch; unit tests with stub forward_batch objects; calling retrieval during a pre-forward hook before seq_lens is populated.","solutions":["Pass forward_batch through: retrieve_topk(queries, forward_batch=forward_batch, ...)","Ensure the object passed has a non-None seq_lens tensor (use the real ForwardBatch)","In tests, build a stub with a real seq_lens tensor attribute"],"exampleFix":"# before\nout = algo.retrieve_topk(queries, top_k=64)\n# after\nout = algo.retrieve_topk(queries, top_k=64, forward_batch=forward_batch)","handlingStrategy":"type-guard","validationCode":"fb = kwargs.get(\"forward_batch\")\nif fb is None or getattr(fb, \"seq_lens\", None) is None:\n    raise ValueError(\"forward_batch with seq_lens required\")","typeGuard":"def has_seq_lens(fb) -> bool:\n    return fb is not None and getattr(fb, \"seq_lens\", None) is not None","tryCatchPattern":null,"preventionTips":["Always forward the real ForwardBatch into retrieval calls","In tests, stub seq_lens with a real tensor"],"tags":["sparse-attention","retrieval","forward-batch","seq-lens","value-error"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}