sgl-project/sglang · error · ValueError

LoRA batch_info must provide max_len or seg_lens.

Error message

LoRA batch_info must provide max_len or seg_lens.

What it means

The KV-B absorbed LoRA GEMM steps need the maximum segment length to size Triton grids; LoRABatchInfo must carry either max_len or seg_lens. If both are None, _max_segment_len cannot derive it and raises.

Source

Thrown at python/sglang/kernels/ops/gemm/kv_b_lora_absorbed.py:89

# ---------------------------------------------------------------------------

_BLOCK_S = 16
_STEP_A_BLOCK_K = 64  # contraction over qk_nope (~128) or kv_lora_rank (~512)
_STEP_A_BLOCK_N = 16  # output is rank
_STEP_B_BLOCK_K = 16  # contraction is rank
_STEP_B_BLOCK_N = 64  # output is kv_lora_rank (~512) or v_head_dim (~128)


def _num_segments(batch_info: LoRABatchInfo) -> int:
    return batch_info.num_segments or batch_info.bs


def _max_segment_len(batch_info: LoRABatchInfo) -> int:
    if batch_info.max_len is not None:
        return batch_info.max_len
    if batch_info.seg_lens is not None:
        return int(batch_info.seg_lens.max().item())
    raise ValueError("LoRA batch_info must provide max_len or seg_lens.")


def _segment_grid_size(batch_info: LoRABatchInfo, num_segments: int) -> int:
    return (
        batch_info.weight_indices.shape[0]
        if batch_info.use_cuda_graph
        else num_segments
    )


# ---------------------------------------------------------------------------
# Kernel 1 -- Step A_q: per-head per-slot SGMM, reads K-half of B
#
#     q_lora_a[t, h, r] = sum_{i<qk_nope} q_nope[t, h, i] * B[slot, h*FULL_K + i, r]
#
# x      : (S, H, qk_nope)
# w (B)  : (num_lora, H*FULL_K, rank)   -- FULL_K = qk_nope + v_head_dim
# out    : (S, H, rank)                 -- fresh allocation, no accumulate

View on GitHub (pinned to 0132848349)

Solutions

1. Set batch_info.max_len to an upper bound on segment length (e.g. precomputed max sequence length) before the call.
2. Or populate batch_info.seg_lens (a tensor of per-segment lengths) so max can be derived.
3. Compute max_len once at batch preparation and pass it through.

Example fix

// before
info = LoRABatchInfo(..., max_len=None, seg_lens=None)
// after
info = LoRABatchInfo(..., max_len=int(seg_lens.max().item()), seg_lens=seg_lens)

if info.max_len is None:
    info.max_len = (int(info.seg_lens.max().item())
                    if info.seg_lens is not None else default_max_len)

When it happens

Trigger: Constructing LoRABatchInfo with max_len=None and seg_lens=None, then calling step_a_q_fwd / step_b_q_fwd / step_a_v_fwd / step_b_v_fwd.

Common situations: Building batch info incrementally and forgetting to set max_len, or code paths that skip seg_lens computation for single-request batches.

Related errors

• LoRA batch_info must provide max_len or seg_lens.
• lora_nickname cannot be empty
• Failed to set LoRA adapter: {str(e)}
• Failed to unset LoRA adapter: {str(e)}
• num_inference_steps is required for transformer-only mode. P

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/651b19dc0801f380. Report an issue: GitHub.