sgl-project/sglang · error · ValueError

unknown absorbed-bmm K variant: {variant!r}

Error message

unknown absorbed-bmm K variant: {variant!r}

What it means

Inside the same function, once the Triton kernel path is taken, the K-block splitting strategy string must be one of the recognized modes ('two_dot','three_dot','pad','single_k', etc.). Reaching the else branch means a variant passed the top-level validation but the K-variant parser does not know it — typically an internal inconsistency after editing the variant lists.

Source

Thrown at python/sglang/kernels/ops/kvcache/cache_ops.py:709

                k1 & (k1 - 1) == 0 and k1 >= 16
            ), "two_dot needs K = pow2 + pow2 with both halves >= 16"
            k_mode = 2
        elif v == "three_dot":
            blk_k = k_dim // 3
            assert (
                k_dim % 3 == 0 and blk_k & (blk_k - 1) == 0 and blk_k >= 16
            ), "three_dot needs K = 3 * pow2 with pow2 >= 16"
            k_mode = 3
        elif v == "pad":
            blk_k = 1 << k_dim.bit_length()  # next power of 2 above K
            k_mode = 4
        elif v == "single_k":
            # Non-power-of-2 BLOCK_K == K: compiles only on Triton builds
            # that allow non-power-of-2 tl.arange (not 3.5.x).
            blk_k = k_dim
            k_mode = 0
        else:
            raise ValueError(f"unknown absorbed-bmm K variant: {variant!r}")
    extra = {"num_stages": num_stages} if num_stages else {}
    grid = (triton.cdiv(num_tokens, block_m), num_heads)
    absorbed_bmm_concat_cast_q_fp8_kernel[grid](
        q_fp8_pad,
        q_nope,
        w_kc,
        q_rope,
        num_tokens,
        q_fp8_pad.stride(0),
        q_fp8_pad.stride(1),
        q_nope.stride(0),
        q_nope.stride(1),
        w_kc.stride(0),
        w_kc.stride(1),
        w_kc.stride(2),
        q_rope.stride(0),
        q_rope.stride(1),
        K=k_dim,

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of the already-implemented variants ('auto' avoids manual selection)
2. If you added a variant to _valid, also implement its blk_k/k_mode branch above the else
3. Revert local edits to cache_ops.py or update to a consistent sglang version

Example fix

# before
_valid = (..., "my_variant")  # added but unimplemented
# after
# remove "my_variant" from _valid, or add:
# elif v == "my_variant": blk_k, k_mode = ..., ...

assert variant in ('auto','cuda','loop','two_dot','three_dot','pad','single_k')
absorbed_bmm_concat_cast_q_fp8(...)

def qprep_variant_valid(v: str) -> bool:
    return v in ('auto','cuda','loop','two_dot','three_dot','pad','single_k')

When it happens

Trigger: The variant passed the _valid tuple check but the if/elif chain over K variants (power-of-2 blocks, single_k, etc.) fell through to else, e.g. after someone added a new variant to _valid without implementing its K handling.

Common situations: Patching cache_ops.py to add a variant name without adding its branch; version skew where _valid and the parser were updated independently.

Related errors

• unknown q-prep variant {variant!r} (SGLANG_OPT_Q8KV8_QPREP_V
• {self._op_label()}: no triton backend
• sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtyp
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• unified_kv dtype mismatch: kv={unified_kv.dtype}, q={q.dtype

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c7ac9b73301eace7. Report an issue: GitHub.