sgl-project/sglang · error · ValueError

kv-canary: launch_canary_plan_kernels_torch_reference requir

Error message

kv-canary: launch_canary_plan_kernels_torch_reference requires req_to_verify_expected_tokens_valid_lens when req_to_verify_expected_tokens is set

What it means

When req_to_verify_expected_tokens is provided to the torch reference planner, the companion valid-lengths tensor req_to_verify_expected_tokens_valid_lens is mandatory — the reference needs per-request lengths to know how many entries in the ragged expected-token pool are valid. Passing the token pool without its lengths raises ValueError.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan_ref.py:59

    )
    prefix_lens_host = prefix_lens.detach().to(device=work_device, dtype=torch.int64)
    extend_seq_lens_host = extend_seq_lens.detach().to(
        device=work_device, dtype=torch.int64
    )
    req_to_token_host = req_to_token.detach().to(device=work_device, dtype=torch.int64)

    lut: Optional[torch.Tensor] = None
    if full_to_swa_index_mapping is not None:
        lut = full_to_swa_index_mapping.detach().to(device=work_device)

    expected_token_pool_host: Optional[torch.Tensor] = None
    req_to_verify_expected_tokens_valid_lens_host: Optional[torch.Tensor] = None
    if req_to_verify_expected_tokens is not None:
        expected_token_pool_host = req_to_verify_expected_tokens.detach().to(
            device=work_device, dtype=torch.int64
        )
        if req_to_verify_expected_tokens_valid_lens is None:
            raise ValueError(
                "kv-canary: launch_canary_plan_kernels_torch_reference requires "
                "req_to_verify_expected_tokens_valid_lens when req_to_verify_expected_tokens is set"
            )
        req_to_verify_expected_tokens_valid_lens_host = (
            req_to_verify_expected_tokens_valid_lens.detach().to(
                device=work_device, dtype=torch.int64
            )
        )

    total_verify = _materialize_verify_entries(
        verify_plan_out=verify_plan_out,
        req_pool_indices_host=req_pool_indices_host,
        prefix_lens_host=prefix_lens_host,
        req_to_token_host=req_to_token_host,
        swa_window_size=swa_window_size,
        lut=lut,
        verify_capacity=verify_capacity,
        work_device=work_device,

View on GitHub (pinned to 0132848349)

Solutions

1. Pass a 1-D int tensor of shape [bs] giving each request's valid token count in the pool
2. If you truly have no expected tokens, pass req_to_verify_expected_tokens=None to skip the feature entirely
3. Build both tensors together from the same loop over requests so they cannot diverge

Example fix

# before
launch_ref(..., req_to_verify_expected_tokens=tok_pool, req_to_verify_expected_tokens_valid_lens=None)
# after
launch_ref(..., req_to_verify_expected_tokens=tok_pool, req_to_verify_expected_tokens_valid_lens=valid_lens)

if req_to_verify_expected_tokens is not None:
    assert req_to_verify_expected_tokens_valid_lens is not None

def has_expected_tokens_args(tokens, valid_lens):
    return tokens is None or valid_lens is not None

When it happens

Trigger: Calling launch_canary_plan_kernels_torch_reference(..., req_to_verify_expected_tokens=tokens, req_to_verify_expected_tokens_valid_lens=None).

Common situations: Building inputs ad hoc in tests or a new integration where the ragged token pool is populated but the seq-len vector is forgotten; partial migration from an older signature that had no valid_lens parameter.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

• v_cache must be provided
• q must be provided unless qv is provided with only_qv=True
• mask_block_cnt and mask_block_idx must be provided for block
• LoRA batch_info must provide max_len or seg_lens.
• kv-canary: {name} must be on {reference_name}'s device {refe

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/18b1a9616918ca29. Report an issue: GitHub.