sgl-project/sglang · error · ValueError

LoRA batch_info must provide max_len or seg_lens.

Error message

LoRA batch_info must provide max_len or seg_lens.

What it means

The TRT-LLM LoRA kernels for A/Q/B/V steps need the maximum segment (sequence) length to size their launch. _max_segment_len reads it from batch_info.max_len, or derives it as max(batch_info.seg_lens); if both are None it cannot size the kernel and raises.

Source

Thrown at python/sglang/kernels/ops/gemm/trtllm_lora_temp/kv_b_lora_absorbed.py:89

# and a small BLOCK_N (matched to rank to avoid wasted tile lanes), while
# the "step_b_*" kernels are the inverse.  Kernels aren't autotuned -- the
# decode-shape workload is too small to benefit and the sweep surface is
# wide.
# ---------------------------------------------------------------------------

_BLOCK_S = 16


def _num_segments(batch_info: LoRABatchInfo) -> int:
    return batch_info.num_segments or batch_info.bs


def _max_segment_len(batch_info: LoRABatchInfo) -> int:
    if batch_info.max_len is not None:
        return batch_info.max_len
    if batch_info.seg_lens is not None:
        return int(batch_info.seg_lens.max().item())
    raise ValueError("LoRA batch_info must provide max_len or seg_lens.")


def _segment_grid_size(batch_info: LoRABatchInfo, num_segments: int) -> int:
    return (
        batch_info.weight_indices.shape[0]
        if batch_info.use_cuda_graph
        else num_segments
    )


# ---------------------------------------------------------------------------
# Kernel 1 -- Step A_q: per-head per-slot SGMM, reads K-half of B
#
#     q_lora_a[t, h, r] = sum_{i<qk_nope} q_nope[t, h, i] * B[slot, h*FULL_K + i, r]
#
# x      : (S, H, qk_nope)
# w (B)  : (num_lora, H*FULL_K, rank)   -- FULL_K = qk_nope + v_head_dim
# out    : (S, H, rank)                 -- fresh allocation, no accumulate

View on GitHub (pinned to 0132848349)

Solutions

1. Set batch_info.max_len to the max sequence length in the batch before calling the step functions
2. Or populate batch_info.seg_lens (per-request lengths tensor) so it can be reduced with .max()
3. If constructing LoRABatchInfo in a dataclass-with-defaults style, audit all call sites for the two optional fields both being None

Example fix

# before
info = LoRABatchInfo(..., max_len=None, seg_lens=None)
step_a_q_fwd(x, info)
# after
info = LoRABatchInfo(..., max_len=int(seq_lens.max().item()), seg_lens=seq_lens)
step_a_q_fwd(x, info)

if batch_info.max_len is None:
    assert batch_info.seg_lens is not None, "LoRABatchInfo needs max_len or seg_lens"
    batch_info.max_len = int(batch_info.seg_lens.max().item())

When it happens

Trigger: Calling step_a_q_fwd / step_b_q_fwd / step_a_v_fwd / step_b_v_fwd with a LoRABatchInfo where max_len is None and seg_lens is None (only weight_indices/seg_starts etc. populated).

Common situations: Building LoRABatchInfo by hand or from a trimmed/serialized path that drops optional fields; upgrading a version where max_len used to be inferred; CUDA-graph capture paths that pass a lightweight batch info.

Understand the failure class

Background: "Missing required field" and "field is required" errors: why libraries reject payloads that omit mandatory fields — this error's family across 20 libraries.

Related errors

• v_cache must be provided
• q must be provided unless qv is provided with only_qv=True
• mask_block_cnt and mask_block_idx must be provided for block
• kv-canary: launch_canary_plan_kernels_torch_reference requir
• lora_nickname cannot be empty

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/098d8cfae13d8236. Report an issue: GitHub.