sgl-project/sglang · error · RuntimeError

Self attention has no KV cache scaling factor attribute!

Error message

Self attention has no KV cache scaling factor attribute!

What it means

load_kv_cache_scales applies FP8 KV-cache quantization scales per layer; it expects each self_attn.attn (RadixAttention) to expose k_scale. If the running build's attention module lacks these attributes, the scales cannot be stored and it raises RuntimeError.

Source

Thrown at python/sglang/srt/models/solar.py:382

    def load_kv_cache_scales(self, quantization_param_path: str) -> None:
        tp_size = get_parallel().tp_size
        tp_rank = get_parallel().tp_rank
        for layer_idx, scaling_factor in kv_cache_scales_loader(
            quantization_param_path,
            tp_rank,
            tp_size,
            self.config.num_hidden_layers,
            self.config.__class__.model_type,
        ):
            if not isinstance(self.layers[layer_idx], nn.Identity):
                layer_self_attn = self.layers[layer_idx].self_attn

            if hasattr(layer_self_attn.attn, "k_scale"):
                layer_self_attn.attn.k_scale = scaling_factor
                layer_self_attn.attn.v_scale = scaling_factor
            else:
                raise RuntimeError(
                    "Self attention has no KV cache scaling " "factor attribute!"
                )


class SolarForCausalLM(nn.Module):

    packed_modules_mapping = {
        "qkv_proj": [
            ("q_proj", "q"),
            ("k_proj", "k"),
            ("v_proj", "v"),
        ],
        "gate_up_proj": [
            ("gate_proj", 0),
            ("up_proj", 1),
        ],
    }

View on GitHub (pinned to 0132848349)

Solutions

1. Update sglang so the model code and the scale-calibration tooling match (same version both sides)
2. Regenerate the kv_cache_scales JSON with the current version's calibration script
3. If scales aren't needed, launch without the kv-scale file / with default kv cache dtype

Example fix

# before
python calc_kv_scales.py --old-version ... ; server with new sglang
# after
pip install -U sglang[all]  # then regenerate scales and relaunch

from sglang.srt.layers.radix_attention import RadixAttention
assert hasattr(RadixAttention(num_heads=1, head_dim=1, scaling=1.0), "k_scale")

When it happens

Trigger: Calling SolarForCausalLM.load_kv_cache_scales(...) (FP8 KV scaling workflow) on a build where RadixAttention has no k_scale/v_scale attributes — typically a version mismatch between the kv-scale calibration tooling and the installed sglang.

Common situations: Using --kv-cache-dtype fp8_e4m3 with a scale JSON produced by a different sglang version; older/newer attention implementations that renamed or removed the scale attributes.

Related errors

• Can't import trtllm_fp8_block_scale_moe from flashinfer. Ple
• Can't import trtllm_fp8_block_scale_routed_moe from flashinf
• Serve backend {name!r} uses API version {backend.api_version
• scalar_type_id {scalar_type_id} doesn't exists.
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2ddfb24ee5721395. Report an issue: GitHub.