sgl-project/sglang · error · ValueError

unknown q-prep variant {variant!r} (SGLANG_OPT_Q8KV8_QPREP_V

Error message

unknown q-prep variant {variant!r} (SGLANG_OPT_Q8KV8_QPREP_VARIANT); valid: {_valid}

What it means

absorbed_bmm_concat_cast_q_fp8 selects a q-preparation implementation variant (Triton loop / CUDA SM90 / fused dot strategies) for the Q8KV8 MLA path; the variant name comes from the SGLANG_OPT_Q8KV8_QPREP_VARIANT env var and must be one of the enumerated valid strings. An unknown name means the env var was misspelled or is from an incompatible version.

Source

Thrown at python/sglang/kernels/ops/kvcache/cache_ops.py:663

    assert n_dim % block_n == 0, "N must be a multiple of block_n"
    assert q_nope.stride(2) == 1 and q_rope.stride(2) == 1
    assert q_fp8_pad.stride(2) == 1
    # Env override for production dispatch (SGLANG_OPT_Q8KV8_QPREP_VARIANT):
    # "auto" (default) keeps the per-K Triton dispatch; "cuda" routes every
    # shape to the WGMMA kernel below.
    global _ENV_QPREP_VARIANT
    if _ENV_QPREP_VARIANT is None:
        _ENV_QPREP_VARIANT = _qprep_env_variant()
    if variant == "auto" and _ENV_QPREP_VARIANT != "auto":
        variant = _ENV_QPREP_VARIANT
    # Hand-written SM90 WGMMA kernel (opt-in only; "auto" never routes here).
    # Same fp32 -> bf16 -> fp8 epilogue; bitwise identical to "two_dot" on
    # SM90.  Requires K in {128, 192} and the production N-major w_kc layout
    # (see the wrapper's asserts); the Triton variants remain the
    # general-strides fallback.
    _valid = ("auto", "cuda", "loop", "two_dot", "three_dot", "pad", "single_k")
    if variant not in _valid:
        raise ValueError(
            f"unknown q-prep variant {variant!r} "
            f"(SGLANG_OPT_Q8KV8_QPREP_VARIANT); valid: {_valid}"
        )
    if variant == "cuda":
        from sglang.kernels.ops.attention.qprep_bf16_fp8_sm90 import q8kv8_qprep_fwd

        q8kv8_qprep_fwd(q_fp8_pad, q_nope, w_kc, q_rope, num_heads)
        return
    # Resolve (K_MODE, BLOCK_K) from the variant; see the kernel's K-handling
    # comment for what each mode compiles to.
    if k_dim & (k_dim - 1) == 0:
        # power-of-2 K: every variant collapses to the single-dot fast path.
        k_mode, blk_k = 0, k_dim
    else:
        v = _AUTO_NONPOW2_VARIANT if variant == "auto" else variant
        if v == "loop":
            # Largest power-of-2 divisor of K, capped at 128 (K % 16 == 0
            # makes this >= 16), unless the caller pinned block_k.

View on GitHub (pinned to 0132848349)

Solutions

Fix the env var value to one of the listed valid variants (start with 'auto')
Unset SGLANG_OPT_Q8KV8_QPREP_VARIANT to use the default auto selection
Check the version's valid list in the error message — it is authoritative for your build

Example fix

# before
export SGLANG_OPT_Q8KV8_QPREP_VARIANT=twodot
# after
export SGLANG_OPT_Q8KV8_QPREP_VARIANT=two_dot

Defensive patterns

Strategy: validation

Validate before calling

import os
from sglang.kernels.ops.kvcache.cache_ops import _valid_variants if False else None
variant = os.environ.get('SGLANG_OPT_Q8KV8_QPREP_VARIANT', 'auto')
assert variant in ('auto','cuda','loop','two_dot','three_dot','pad','single_k'), variant

Type guard

def qprep_variant_valid(v: str) -> bool:
    return v in ('auto','cuda','loop','two_dot','three_dot','pad','single_k')

Try / catch

try:
    absorbed_bmm_concat_cast_q_fp8(...)
except ValueError as e:
    if 'SGLANG_OPT_Q8KV8_QPREP_VARIANT' in str(e):
        os.environ['SGLANG_OPT_Q8KV8_QPREP_VARIANT'] = 'auto'
    else:
        raise

Prevention

Prefer 'auto' unless a specific variant is required
Validate env-var-driven knobs at startup, not deep in the kernel path

When it happens

Trigger: Setting SGLANG_OPT_Q8KV8_QPREP_VARIANT to a string not in ('auto','cuda','loop','two_dot','three_dot','pad','single_k') and invoking the absorbed BMM path (e.g. via forward_absorb_prepare).

Common situations: Typo in the env var value ('twodot', 'cuda_sm90'); a variant removed/renamed across sglang versions; copy-pasting a variant name valid only on newer/older builds.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/71062e6c55fccf20. Report an issue: GitHub.