sgl-project/sglang · error · ValueError

Invalid v_out shape for fused KV materialization: got {tuple

Error message

Invalid v_out shape for fused KV materialization: got {tuple(v_out.shape)}, expected {expected_shape}.

What it means

A caller-supplied v_out buffer must have shape [n_layers, total_ctx, num_kv_heads, head_dim], identical to k_out's required layout.

Source

Thrown at python/sglang/kernels/ops/speculative/fused_kv_materialize.py:201

    if k_out is None:
        k_out = torch.empty(expected_shape, dtype=kv.dtype, device=kv.device)
    else:
        if k_out.shape != expected_shape:
            raise ValueError(
                "Invalid k_out shape for fused KV materialization: "
                f"got {tuple(k_out.shape)}, expected {expected_shape}."
            )
        if k_out.device != kv.device or k_out.dtype != kv.dtype:
            raise ValueError(
                "Invalid k_out device/dtype for fused KV materialization: "
                f"got device={k_out.device}, dtype={k_out.dtype}, "
                f"expected device={kv.device}, dtype={kv.dtype}."
            )
    if v_out is None:
        v_out = torch.empty_like(k_out)
    else:
        if v_out.shape != expected_shape:
            raise ValueError(
                "Invalid v_out shape for fused KV materialization: "
                f"got {tuple(v_out.shape)}, expected {expected_shape}."
            )
        if v_out.device != kv.device or v_out.dtype != kv.dtype:
            raise ValueError(
                "Invalid v_out device/dtype for fused KV materialization: "
                f"got device={v_out.device}, dtype={v_out.dtype}, "
                f"expected device={kv.device}, dtype={kv.dtype}."
            )

    _fused_norm_rope_kernel_stacked[(total_ctx, num_kv_heads, n_layers)](
        kv,
        k_norm_weight,
        eps,
        cos_sin_cache,
        positions,
        k_out,
        v_out,

View on GitHub (pinned to 0132848349)

Solutions

1. Allocate v_out identically to k_out (torch.empty_like(k_out)) or pass None.
2. Verify total_ctx and n_layers match the kv input.

Example fix

// before
v_out = torch.empty(total_ctx, H, D)
// after
v_out = None  # or k_out.new_empty((n_layers, total_ctx, H, D))

expected = (n_layers, total_ctx, num_kv_heads, head_dim)
assert v_out is None or v_out.shape == expected

When it happens

Trigger: Passing v_out sized or ordered differently from k_out (e.g. only [total_ctx, ...] or sized for a shorter context).

Common situations: Reusing stale cache buffers sized for a previous batch, or mirroring an incorrect k_out allocation into v_out.

Related errors

• rope_pool_fused expects pool tensors to be 3-D
• k_pool has incompatible shape {k_pool.shape}
• v_pool shape must match k_pool shape, got {v_pool.shape} vs
• D={D_check} must be divisible by GROUP_SIZE={_FP8_GROUP_SIZE
• kv_scales shape {tuple(kv_scales.shape)} does not match expe

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/555b395489c9c097. Report an issue: GitHub.