{"record":{"id":"2ddfb24ee5721395","repo":"sgl-project/sglang","slug":"self-attention-has-no-kv-cache-scaling-factor-attr-2ddfb2","errorCode":null,"errorMessage":"Self attention has no KV cache scaling factor attribute!","messagePattern":"Self attention has no KV cache scaling factor attribute!","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/solar.py","lineNumber":382,"sourceCode":"\n    def load_kv_cache_scales(self, quantization_param_path: str) -> None:\n        tp_size = get_parallel().tp_size\n        tp_rank = get_parallel().tp_rank\n        for layer_idx, scaling_factor in kv_cache_scales_loader(\n            quantization_param_path,\n            tp_rank,\n            tp_size,\n            self.config.num_hidden_layers,\n            self.config.__class__.model_type,\n        ):\n            if not isinstance(self.layers[layer_idx], nn.Identity):\n                layer_self_attn = self.layers[layer_idx].self_attn\n\n            if hasattr(layer_self_attn.attn, \"k_scale\"):\n                layer_self_attn.attn.k_scale = scaling_factor\n                layer_self_attn.attn.v_scale = scaling_factor\n            else:\n                raise RuntimeError(\n                    \"Self attention has no KV cache scaling \" \"factor attribute!\"\n                )\n\n\nclass SolarForCausalLM(nn.Module):\n\n    packed_modules_mapping = {\n        \"qkv_proj\": [\n            (\"q_proj\", \"q\"),\n            (\"k_proj\", \"k\"),\n            (\"v_proj\", \"v\"),\n        ],\n        \"gate_up_proj\": [\n            (\"gate_proj\", 0),\n            (\"up_proj\", 1),\n        ],\n    }\n","sourceCodeStart":364,"sourceCodeEnd":400,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/solar.py#L364-L400","documentation":"load_kv_cache_scales applies FP8 KV-cache quantization scales per layer; it expects each self_attn.attn (RadixAttention) to expose k_scale. If the running build's attention module lacks these attributes, the scales cannot be stored and it raises RuntimeError.","triggerScenarios":"Calling SolarForCausalLM.load_kv_cache_scales(...) (FP8 KV scaling workflow) on a build where RadixAttention has no k_scale/v_scale attributes — typically a version mismatch between the kv-scale calibration tooling and the installed sglang.","commonSituations":"Using --kv-cache-dtype fp8_e4m3 with a scale JSON produced by a different sglang version; older/newer attention implementations that renamed or removed the scale attributes.","solutions":["Update sglang so the model code and the scale-calibration tooling match (same version both sides)","Regenerate the kv_cache_scales JSON with the current version's calibration script","If scales aren't needed, launch without the kv-scale file / with default kv cache dtype"],"exampleFix":"# before\npython calc_kv_scales.py --old-version ... ; server with new sglang\n# after\npip install -U sglang[all]  # then regenerate scales and relaunch","handlingStrategy":"validation","validationCode":"from sglang.srt.layers.radix_attention import RadixAttention\nassert hasattr(RadixAttention(num_heads=1, head_dim=1, scaling=1.0), \"k_scale\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Generate kv scales and serve with the same sglang version"],"tags":["solar","fp8","kv-cache-scales","version-mismatch"],"backgroundTag":"fp8-kv-scale-attribute-missing","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}