sgl-project/sglang · error · ValueError

kv-canary: req_to_token_stride0={req_to_token_stride0} does

Error message

kv-canary: req_to_token_stride0={req_to_token_stride0} does not match req_to_token.stride(0)={int(req_to_token.stride(0))}

What it means

The row stride must match the actual stride(0) of the req_to_token tensor so kernel address arithmetic matches memory layout. A mismatch (e.g. from a sliced, transposed, or expanded view) would read wrong rows.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/offsets_kernel.py:214

        "out_write_seed_slot_indices",
        write_req_capacity,
    )
    _require_len(out_verify_num_valid, "out_verify_num_valid", 1)
    _require_len(out_verify_enable, "out_verify_enable", 1)
    _require_len(out_write_num_valid_reqs, "out_write_num_valid_reqs", 1)
    _require_1d(lut_tensor, "lut_tensor")

    if write_offsets_len != write_req_capacity + 1:
        raise ValueError(
            f"kv-canary: write_offsets_len must equal write_req_capacity + 1, got "
            f"{write_offsets_len} and {write_req_capacity}"
        )
    if bs > write_req_capacity:
        raise ValueError(
            f"kv-canary: bs={bs} exceeds write_req_capacity={write_req_capacity}"
        )
    if req_to_token_stride0 != int(req_to_token.stride(0)):
        raise ValueError(
            f"kv-canary: req_to_token_stride0={req_to_token_stride0} does not match "
            f"req_to_token.stride(0)={int(req_to_token.stride(0))}"
        )

    _require_same_device(
        out_verify_offsets_scratch,
        "out_verify_offsets_scratch",
        (
            (req_pool_indices, "req_pool_indices"),
            (prefix_lens, "prefix_lens"),
            (extend_seq_lens, "extend_seq_lens"),
            (req_to_token, "req_to_token"),
            (lut_tensor, "lut_tensor"),
            (out_write_offsets, "out_write_offsets"),
            (out_write_seed_slot_indices, "out_write_seed_slot_indices"),
            (out_verify_num_valid, "out_verify_num_valid"),
            (out_verify_enable, "out_verify_enable"),
            (out_write_num_valid_reqs, "out_write_num_valid_reqs"),

View on GitHub (pinned to 0132848349)

Solutions

1. Always derive the argument: req_to_token_stride0 = int(req_to_token.stride(0))
2. If a contiguous layout is assumed, call req_to_token = req_to_token.contiguous() first and pass stride accordingly

Example fix

// before
launch_plan_offsets_kernel(..., req_to_token_stride0=max_context_len)
// after
req_to_token = req_to_token.contiguous()
launch_plan_offsets_kernel(..., req_to_token_stride0=int(req_to_token.stride(0)))

req_to_token_stride0 = int(req_to_token.stride(0))
# or enforce layout
req_to_token = req_to_token.contiguous()

When it happens

Trigger: Calling launch_plan_offsets_kernel with req_to_token_stride0 != int(req_to_token.stride(0)) — passing a hardcoded max_context_len while the tensor is a non-contiguous view, or vice versa.

Common situations: req_to_token refactored to a slice/view of a larger mapping (e.g. for DP shards), changing its row stride while callers still pass the old constant.

Related errors

• Not a canonical UMMA_MN Layout: Expected stride failure.
• Not a canonical UMMA_K Layout: Expected stride failure.
• kv-canary: req_to_token_stride0 must be positive, got {req_t
• out must have stride 1 in the last dimension
• out must have stride 1 in the last dimension

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e8f84eea1eb0f3a9. Report an issue: GitHub.