sgl-project/sglang · error · ValueError
unknown q-prep variant
Error message
unknown q-prep variant {variant!r} (SGLANG_OPT_Q8KV8_QPREP_VARIANT); valid: {_valid} What it means
absorbed_bmm_concat_cast_q_fp8 selects a q-preparation implementation variant (Triton loop / CUDA SM90 / fused dot strategies) for the Q8KV8 MLA path; the variant name comes from the SGLANG_OPT_Q8KV8_QPREP_VARIANT env var and must be one of the enumerated valid strings. An unknown name means the env var was misspelled or is from an incompatible version.
Solutions
- Fix the env var value to one of the listed valid variants (start with 'auto')
- Unset SGLANG_OPT_Q8KV8_QPREP_VARIANT to use the default auto selection
- Check the version's valid list in the error message — it is authoritative for your build
Example fix
# before export SGLANG_OPT_Q8KV8_QPREP_VARIANT=twodot # after export SGLANG_OPT_Q8KV8_QPREP_VARIANT=two_dot
Defensive patterns
Strategy: validation
Validate before calling
import os
from sglang.kernels.ops.kvcache.cache_ops import _valid_variants if False else None
variant = os.environ.get('SGLANG_OPT_Q8KV8_QPREP_VARIANT', 'auto')
assert variant in ('auto','cuda','loop','two_dot','three_dot','pad','single_k'), variant Type guard
def qprep_variant_valid(v: str) -> bool:
return v in ('auto','cuda','loop','two_dot','three_dot','pad','single_k') Try / catch
try:
absorbed_bmm_concat_cast_q_fp8(...)
except ValueError as e:
if 'SGLANG_OPT_Q8KV8_QPREP_VARIANT' in str(e):
os.environ['SGLANG_OPT_Q8KV8_QPREP_VARIANT'] = 'auto'
else:
raise Prevention
- Prefer 'auto' unless a specific variant is required
- Validate env-var-driven knobs at startup, not deep in the kernel path
When it happens
Trigger: Setting SGLANG_OPT_Q8KV8_QPREP_VARIANT to a string not in ('auto','cuda','loop','two_dot','three_dot','pad','single_k') and invoking the absorbed BMM path (e.g. via forward_absorb_prepare).
Common situations: Typo in the env var value ('twodot', 'cuda_sm90'); a variant removed/renamed across sglang versions; copy-pasting a variant name valid only on newer/older builds.
Understand the failure class
Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.
Related errors
- MLA model is not supported without global metadata server…
- SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models…
- SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models…
- unknown absorbed-bmm K variant
- aiter is required when SGLANG_USE_AITER is set to True
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/71062e6c55fccf20.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/kernels/ops/kvcache/cache_ops.py:663
assert n_dim % block_n == 0, "N must be a multiple of block_n"
assert q_nope.stride(2) == 1 and q_rope.stride(2) == 1
assert q_fp8_pad.stride(2) == 1
# Env override for production dispatch (SGLANG_OPT_Q8KV8_QPREP_VARIANT):
# "auto" (default) keeps the per-K Triton dispatch; "cuda" routes every
# shape to the WGMMA kernel below.
global _ENV_QPREP_VARIANT
if _ENV_QPREP_VARIANT is None:
_ENV_QPREP_VARIANT = _qprep_env_variant()
if variant == "auto" and _ENV_QPREP_VARIANT != "auto":
variant = _ENV_QPREP_VARIANT
# Hand-written SM90 WGMMA kernel (opt-in only; "auto" never routes here).
# Same fp32 -> bf16 -> fp8 epilogue; bitwise identical to "two_dot" on
# SM90. Requires K in {128, 192} and the production N-major w_kc layout
# (see the wrapper's asserts); the Triton variants remain the
# general-strides fallback.
_valid = ("auto", "cuda", "loop", "two_dot", "three_dot", "pad", "single_k")
if variant not in _valid:
raise ValueError(
f"unknown q-prep variant {variant!r} "
f"(SGLANG_OPT_Q8KV8_QPREP_VARIANT); valid: {_valid}"
)
if variant == "cuda":
from sglang.kernels.ops.attention.qprep_bf16_fp8_sm90 import q8kv8_qprep_fwd
q8kv8_qprep_fwd(q_fp8_pad, q_nope, w_kc, q_rope, num_heads)
return
# Resolve (K_MODE, BLOCK_K) from the variant; see the kernel's K-handling
# comment for what each mode compiles to.
if k_dim & (k_dim - 1) == 0:
# power-of-2 K: every variant collapses to the single-dot fast path.
k_mode, blk_k = 0, k_dim
else:
v = _AUTO_NONPOW2_VARIANT if variant == "auto" else variant
if v == "loop":
# Largest power-of-2 divisor of K, capped at 128 (K % 16 == 0
# makes this >= 16), unless the caller pinned block_k.View on GitHub (pinned to 0132848349)