sgl-project/sglang · error · ValueError

miss_src must be int64 and miss_dst must be int32.

Error message

miss_src must be int64 and miss_dst must be int32.

What it means

Part of the miss-plan validation: the kernel ABI expects miss_src as int64 (host row indices) and miss_dst as int32 (device slot indices). Wrong dtypes would silently truncate or misindex in the CUDA kernel, so they are rejected before launch.

Source

Thrown at python/sglang/kernels/ops/kvcache/hisparse.py:109

    kernel, so shared-index layers can replay only the Host-to-GPU copies with
    ``copy_cache_planned_mla``.
    """
    _, num_steps, num_top_k = top_k_tokens.shape
    if not 2 <= num_steps <= 4:
        raise ValueError(
            f"HiSparse speculative swap requires 2-4 steps, got {num_steps}."
        )
    hot_buffer_size = state.cache_policy.size(1)
    page_size = device_buffer_tokens.size(1) - hot_buffer_size
    item_size_bytes = host_cache.stride(0) * host_cache.element_size()
    record_miss_plan = miss_src is not None
    if record_miss_plan:
        if miss_dst is None or miss_count is None:
            raise ValueError(
                "miss_src, miss_dst, and miss_count must be provided together."
            )
        if miss_src.dtype != torch.int64 or miss_dst.dtype != torch.int32:
            raise ValueError("miss_src must be int64 and miss_dst must be int32.")
        if miss_count.dtype != torch.int32:
            raise ValueError("miss_count must be int32.")
        plan_capacity = num_steps * num_top_k
        batch_size = top_k_tokens.size(0)
        if (
            miss_src.ndim != 2
            or miss_dst.ndim != 2
            or miss_src.size(0) < batch_size
            or miss_dst.size(0) < batch_size
            or miss_src.size(1) < plan_capacity
            or miss_dst.size(1) < plan_capacity
        ):
            raise ValueError(
                "speculative miss_src/miss_dst must have shape "
                f"[batch, >= steps * top_k] (capacity {plan_capacity})."
            )
        if miss_count.ndim != 1 or miss_count.numel() < batch_size:
            raise ValueError("speculative miss_count must have shape [batch].")

View on GitHub (pinned to 0132848349)

Solutions

1. Allocate miss_src with dtype=torch.int64 and miss_dst with dtype=torch.int32
2. Double-check miss_count is int32 as well
3. Keep one shared helper that allocates the correctly-typed triple

Example fix

# before
miss_src = torch.zeros(bs, cap, dtype=torch.int32, device=dev)
# after
miss_src = torch.zeros(bs, cap, dtype=torch.int64, device=dev)

assert miss_src.dtype == torch.int64 and miss_dst.dtype == torch.int32

def miss_plan_dtypes_ok(s, d) -> bool:
    return s is None or (s.dtype == torch.int64 and d.dtype == torch.int32)

When it happens

Trigger: Calling load_cache_to_device_buffer_spec_mla with miss_plan enabled and miss_src not torch.int64 or miss_dst not torch.int32 (e.g. both allocated as int32 or both as int64).

Common situations: Allocating plan tensors with a single dtype for convenience; converting from a numpy array whose default integer width differs by platform.

Related errors

• miss_count must be int32.
• speculative miss_src/miss_dst must have shape [batch, >= ste
• speculative miss_count must have shape [batch].
• {name} must have dtype {dtype}, got {t.dtype}
• kv-canary: scatter_req_token_ids flat_in must be int64, got

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dfc4583fea1fa0c2. Report an issue: GitHub.