sgl-project/sglang · error · ValueError

Unknown KV cache quantization method: '{name}'. Available: {

Error message

Unknown KV cache quantization method: '{name}'. Available: {list(KV_CACHE_QUANT_REGISTRY)}

What it means

Raised by get_kv_cache_quant_method when the requested KV cache quantization method name is not present in KV_CACHE_QUANT_REGISTRY. The registry only contains methods that have been explicitly registered for KV cache quantization (e.g. fp8 variants), so any unrecognized quant string from the checkpoint/server config fails fast. This is a factory lookup error, not a runtime kernel error.

Source

Thrown at python/sglang/srt/layers/quantization/fp4_kv_cache_quant_method.py:833

        raise ValueError(
            "--kv-cache-dtype=fp4_e2m1 is deprecated. "
            "Use --kv-cache-dtype=fp4_mx_block16."
        )
    if kv_cache_dtype == "mxfp4":
        raise ValueError(
            "--kv-cache-dtype=mxfp4 is reserved for true MXFP4 block-size-32 "
            "semantics. Use --kv-cache-dtype=fp4_mx_block16 for the current "
            "block-size-16 FP4 KV recipe."
        )
    if kv_cache_dtype in KV_CACHE_QUANT_REGISTRY:
        return kv_cache_dtype
    return None


def get_kv_cache_quant_method(name: str, **kwargs) -> KVCacheQuantMethodBase:
    """Instantiate a KVCacheQuantMethodBase by internal method name."""
    if name not in KV_CACHE_QUANT_REGISTRY:
        raise ValueError(
            f"Unknown KV cache quantization method: '{name}'. "
            f"Available: {list(KV_CACHE_QUANT_REGISTRY)}"
        )
    return KV_CACHE_QUANT_REGISTRY[name](**kwargs)

View on GitHub (pinned to 0132848349)

Solutions

Check the registry contents at runtime: from sglang.srt.layers.quantization.fp4_kv_cache_quant_method import KV_CACHE_QUANT_REGISTRY; print(list(KV_CACHE_QUANT_REGISTRY)) and use one of those names
Fix the quantization_config in the model's config.json (quant_method for kv cache) to a supported method such as 'fp8'
Upgrade SGLang to a version that registers the method you need
If you implemented a new KV quant method, decorate/register it in KV_CACHE_QUANT_REGISTRY

Example fix

// before
method = get_kv_cache_quant_method("int8")  # ValueError
// after
from sglang.srt.layers.quantization.fp4_kv_cache_quant_method import KV_CACHE_QUANT_REGISTRY
name = "fp8"
assert name in KV_CACHE_QUANT_REGISTRY, f"pick from {list(KV_CACHE_QUANT_REGISTRY)}"
method = get_kv_cache_quant_method(name)

Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.layers.quantization.fp4_kv_cache_quant_method import KV_CACHE_QUANT_REGISTRY, get_kv_cache_quant_method
name = cfg.quant_method
if name not in KV_CACHE_QUANT_REGISTRY:
    raise SystemExit(f"unsupported kv quant {name}; pick from {list(KV_CACHE_QUANT_REGISTRY)}")
method = get_kv_cache_quant_method(name)

Type guard

def is_supported_kv_quant(name: str) -> bool:
    from sglang.srt.layers.quantization.fp4_kv_cache_quant_method import KV_CACHE_QUANT_REGISTRY
    return name in KV_CACHE_QUANT_REGISTRY

Try / catch

try:
    m = get_kv_cache_quant_method(name)
except ValueError as e:
    logger.warning("falling back to fp8 kv quant: %s", e)
    m = get_kv_cache_quant_method("fp8")

Prevention

Validate quant method names against KV_CACHE_QUANT_REGISTRY before constructing configs
Pin SGLang versions in CI to avoid registry churn
Log the registry contents in startup diagnostics

When it happens

Trigger: Calling get_kv_cache_quant_method(name, **kwargs) with a name not in KV_CACHE_QUANT_REGISTRY — typically indirectly by launching a server or model whose quantization_config specifies an unsupported kv cache quant algo (e.g. 'int8', 'awq', or a typo like 'fp8_e5m2' when only fp8 methods are registered).

Common situations: Using a checkpoint with a quantization_config kv-cache method SGLang does not support; typo in --kv-cache-dtype or quant config; new/renamed method names after a version upgrade; passing a weight-quant method name where a KV-quant method is expected.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cc7276c2542d1705. Report an issue: GitHub.