{"record":{"id":"71062e6c55fccf20","repo":"sgl-project/sglang","slug":"unknown-q-prep-variant-variant-r-sglang-opt-q8k","errorCode":null,"errorMessage":"unknown q-prep variant {variant!r} (SGLANG_OPT_Q8KV8_QPREP_VARIANT); valid: {_valid}","messagePattern":"unknown q-prep variant (.+?) \\(SGLANG_OPT_Q8KV8_QPREP_VARIANT\\); valid: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kvcache/cache_ops.py","lineNumber":663,"sourceCode":"    assert n_dim % block_n == 0, \"N must be a multiple of block_n\"\n    assert q_nope.stride(2) == 1 and q_rope.stride(2) == 1\n    assert q_fp8_pad.stride(2) == 1\n    # Env override for production dispatch (SGLANG_OPT_Q8KV8_QPREP_VARIANT):\n    # \"auto\" (default) keeps the per-K Triton dispatch; \"cuda\" routes every\n    # shape to the WGMMA kernel below.\n    global _ENV_QPREP_VARIANT\n    if _ENV_QPREP_VARIANT is None:\n        _ENV_QPREP_VARIANT = _qprep_env_variant()\n    if variant == \"auto\" and _ENV_QPREP_VARIANT != \"auto\":\n        variant = _ENV_QPREP_VARIANT\n    # Hand-written SM90 WGMMA kernel (opt-in only; \"auto\" never routes here).\n    # Same fp32 -> bf16 -> fp8 epilogue; bitwise identical to \"two_dot\" on\n    # SM90.  Requires K in {128, 192} and the production N-major w_kc layout\n    # (see the wrapper's asserts); the Triton variants remain the\n    # general-strides fallback.\n    _valid = (\"auto\", \"cuda\", \"loop\", \"two_dot\", \"three_dot\", \"pad\", \"single_k\")\n    if variant not in _valid:\n        raise ValueError(\n            f\"unknown q-prep variant {variant!r} \"\n            f\"(SGLANG_OPT_Q8KV8_QPREP_VARIANT); valid: {_valid}\"\n        )\n    if variant == \"cuda\":\n        from sglang.kernels.ops.attention.qprep_bf16_fp8_sm90 import q8kv8_qprep_fwd\n\n        q8kv8_qprep_fwd(q_fp8_pad, q_nope, w_kc, q_rope, num_heads)\n        return\n    # Resolve (K_MODE, BLOCK_K) from the variant; see the kernel's K-handling\n    # comment for what each mode compiles to.\n    if k_dim & (k_dim - 1) == 0:\n        # power-of-2 K: every variant collapses to the single-dot fast path.\n        k_mode, blk_k = 0, k_dim\n    else:\n        v = _AUTO_NONPOW2_VARIANT if variant == \"auto\" else variant\n        if v == \"loop\":\n            # Largest power-of-2 divisor of K, capped at 128 (K % 16 == 0\n            # makes this >= 16), unless the caller pinned block_k.","sourceCodeStart":645,"sourceCodeEnd":681,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kvcache/cache_ops.py#L645-L681","documentation":"absorbed_bmm_concat_cast_q_fp8 selects a q-preparation implementation variant (Triton loop / CUDA SM90 / fused dot strategies) for the Q8KV8 MLA path; the variant name comes from the SGLANG_OPT_Q8KV8_QPREP_VARIANT env var and must be one of the enumerated valid strings. An unknown name means the env var was misspelled or is from an incompatible version.","triggerScenarios":"Setting SGLANG_OPT_Q8KV8_QPREP_VARIANT to a string not in ('auto','cuda','loop','two_dot','three_dot','pad','single_k') and invoking the absorbed BMM path (e.g. via forward_absorb_prepare).","commonSituations":"Typo in the env var value ('twodot', 'cuda_sm90'); a variant removed/renamed across sglang versions; copy-pasting a variant name valid only on newer/older builds.","solutions":["Fix the env var value to one of the listed valid variants (start with 'auto')","Unset SGLANG_OPT_Q8KV8_QPREP_VARIANT to use the default auto selection","Check the version's valid list in the error message — it is authoritative for your build"],"exampleFix":"# before\nexport SGLANG_OPT_Q8KV8_QPREP_VARIANT=twodot\n# after\nexport SGLANG_OPT_Q8KV8_QPREP_VARIANT=two_dot","handlingStrategy":"validation","validationCode":"import os\nfrom sglang.kernels.ops.kvcache.cache_ops import _valid_variants if False else None\nvariant = os.environ.get('SGLANG_OPT_Q8KV8_QPREP_VARIANT', 'auto')\nassert variant in ('auto','cuda','loop','two_dot','three_dot','pad','single_k'), variant","typeGuard":"def qprep_variant_valid(v: str) -> bool:\n    return v in ('auto','cuda','loop','two_dot','three_dot','pad','single_k')","tryCatchPattern":"try:\n    absorbed_bmm_concat_cast_q_fp8(...)\nexcept ValueError as e:\n    if 'SGLANG_OPT_Q8KV8_QPREP_VARIANT' in str(e):\n        os.environ['SGLANG_OPT_Q8KV8_QPREP_VARIANT'] = 'auto'\n    else:\n        raise","preventionTips":["Prefer 'auto' unless a specific variant is required","Validate env-var-driven knobs at startup, not deep in the kernel path"],"tags":["q8kv8","env-var","variant-selection","mla"],"backgroundTag":"invalid-env-var-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}