{"record":{"id":"098d8cfae13d8236","repo":"sgl-project/sglang","slug":"lora-batch-info-must-provide-max-len-or-seg-lens-098d8c","errorCode":null,"errorMessage":"LoRA batch_info must provide max_len or seg_lens.","messagePattern":"LoRA batch_info must provide max_len or seg_lens\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/gemm/trtllm_lora_temp/kv_b_lora_absorbed.py","lineNumber":89,"sourceCode":"# and a small BLOCK_N (matched to rank to avoid wasted tile lanes), while\n# the \"step_b_*\" kernels are the inverse.  Kernels aren't autotuned -- the\n# decode-shape workload is too small to benefit and the sweep surface is\n# wide.\n# ---------------------------------------------------------------------------\n\n_BLOCK_S = 16\n\n\ndef _num_segments(batch_info: LoRABatchInfo) -> int:\n    return batch_info.num_segments or batch_info.bs\n\n\ndef _max_segment_len(batch_info: LoRABatchInfo) -> int:\n    if batch_info.max_len is not None:\n        return batch_info.max_len\n    if batch_info.seg_lens is not None:\n        return int(batch_info.seg_lens.max().item())\n    raise ValueError(\"LoRA batch_info must provide max_len or seg_lens.\")\n\n\ndef _segment_grid_size(batch_info: LoRABatchInfo, num_segments: int) -> int:\n    return (\n        batch_info.weight_indices.shape[0]\n        if batch_info.use_cuda_graph\n        else num_segments\n    )\n\n\n# ---------------------------------------------------------------------------\n# Kernel 1 -- Step A_q: per-head per-slot SGMM, reads K-half of B\n#\n#     q_lora_a[t, h, r] = sum_{i<qk_nope} q_nope[t, h, i] * B[slot, h*FULL_K + i, r]\n#\n# x      : (S, H, qk_nope)\n# w (B)  : (num_lora, H*FULL_K, rank)   -- FULL_K = qk_nope + v_head_dim\n# out    : (S, H, rank)                 -- fresh allocation, no accumulate","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/gemm/trtllm_lora_temp/kv_b_lora_absorbed.py#L71-L107","documentation":"The TRT-LLM LoRA kernels for A/Q/B/V steps need the maximum segment (sequence) length to size their launch. _max_segment_len reads it from batch_info.max_len, or derives it as max(batch_info.seg_lens); if both are None it cannot size the kernel and raises.","triggerScenarios":"Calling step_a_q_fwd / step_b_q_fwd / step_a_v_fwd / step_b_v_fwd with a LoRABatchInfo where max_len is None and seg_lens is None (only weight_indices/seg_starts etc. populated).","commonSituations":"Building LoRABatchInfo by hand or from a trimmed/serialized path that drops optional fields; upgrading a version where max_len used to be inferred; CUDA-graph capture paths that pass a lightweight batch info.","solutions":["Set batch_info.max_len to the max sequence length in the batch before calling the step functions","Or populate batch_info.seg_lens (per-request lengths tensor) so it can be reduced with .max()","If constructing LoRABatchInfo in a dataclass-with-defaults style, audit all call sites for the two optional fields both being None"],"exampleFix":"# before\ninfo = LoRABatchInfo(..., max_len=None, seg_lens=None)\nstep_a_q_fwd(x, info)\n# after\ninfo = LoRABatchInfo(..., max_len=int(seq_lens.max().item()), seg_lens=seq_lens)\nstep_a_q_fwd(x, info)","handlingStrategy":"validation","validationCode":"if batch_info.max_len is None:\n    assert batch_info.seg_lens is not None, \"LoRABatchInfo needs max_len or seg_lens\"\n    batch_info.max_len = int(batch_info.seg_lens.max().item())","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always populate either max_len or seg_lens when building LoRABatchInfo","Add a dataclass __post_assert__ or factory that fills max_len from seg_lens","Unit-test batch-info construction paths including CUDA-graph capture"],"tags":["lora","trtllm","missing-argument","validation"],"backgroundTag":"missing-required-field","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}