{"record":{"id":"06bbcb0598d4778d","repo":"sgl-project/sglang","slug":"self-class-name-apply-should-not-be-call","errorCode":null,"errorMessage":"{self.__class__.__name__}.apply should not be called.","messagePattern":"(.+?)\\.apply should not be called\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/kv_cache.py","lineNumber":49,"sourceCode":"\n    def create_weights(self, layer: torch.nn.Module):\n        \"\"\"\n        Create \"weight\" (aka k_scale and v_scale) for an attention layer.\n        \"\"\"\n        # Initialize the KV cache scales to -1.0, which is an invalid value.\n        # If the k/v_scale appears in the checkpoint, it will be\n        # overwritten when loading weights.\n        layer.k_scale = torch.nn.Parameter(\n            torch.tensor(-1.0, dtype=torch.float32), requires_grad=False\n        )\n        layer.v_scale = torch.nn.Parameter(\n            torch.tensor(-1.0, dtype=torch.float32), requires_grad=False\n        )\n        layer.k_scale._skip_weight_check = True\n        layer.v_scale._skip_weight_check = True\n\n    def apply(self, layer: torch.nn.Module) -> torch.Tensor:\n        raise RuntimeError(f\"{self.__class__.__name__}.apply should not be called.\")\n\n    def process_weights_after_loading(self, layer) -> None:\n        if layer.k_scale > 0.0 and layer.v_scale > 0.0:\n            # We prefer to use separate k_scale and v_scale if present\n            k_scale = layer.k_scale.to(\"cpu\").tolist()\n            v_scale = layer.v_scale.to(\"cpu\").tolist()\n            if is_fp8_fnuz():\n                k_scale *= 2\n                v_scale *= 2\n        elif layer.k_scale <= 0.0 and layer.v_scale <= 0.0:\n            # If no scales were loaded (both scales are invalid non-positive\n            # values), use the default value of 1.0\n            k_scale = 1.0\n            v_scale = 1.0\n        else:\n            # If we find a single kv_scale in the checkpoint, we remap\n            # kv_scale to k_scale during weight loading, and duplicate\n            # k_scale to v_scale here","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/kv_cache.py#L31-L67","documentation":"BaseScalaQuantLinearMethod.apply is an intentionally unimplemented stub on the KV-cache quantization method class; the quantized KV scale is applied inside Attention.forward, not via a linear apply(). Calling apply (directly or through generic dispatch code that assumes every quant method implements it) raises this RuntimeError naming the concrete subclass.","triggerScenarios":"Generic code that iterates quant methods and calls .apply(layer); porting a code path from linear-layer quantization (where apply is the main entry) to a layer whose quant_method is a KV-cache scalar method such as BaseScalaQuantLinearMethod.","commonSituations":"Writing model support code that treats all QuantizeMethodBase implementations uniformly; newer quant method subclasses forgetting to keep apply unreachable.","solutions":["Do not call apply() on KV-cache quant methods; they only provide process_weights_after_loading and the scales are consumed by the attention backend","Branch on the method type before dispatching (see typeGuard)","Keep layer.quant_method usage aligned with the layer kind (LinearMethod vs KV method)"],"exampleFix":"# before\nout = layer.quant_method.apply(layer, x)\n# after\nif isinstance(layer.quant_method, BaseScalaQuantLinearMethod):\n    raise TypeError(\"KV-cache quant methods have no apply()\")\nout = layer.quant_method.apply(layer, x)","handlingStrategy":"type-guard","validationCode":"from sglang.srt.layers.quantization.kv_cache import BaseScalaQuantLinearMethod\nif isinstance(layer.quant_method, BaseScalaQuantLinearMethod):\n    raise TypeError(\"KV-cache quant methods expose no apply(); use attention forward\")","typeGuard":"def has_apply(quant_method) -> bool:\n    return not isinstance(quant_method, BaseScalaQuantLinearMethod) and callable(getattr(quant_method, \"apply\", None))","tryCatchPattern":"try:\n    out = layer.quant_method.apply(layer, x)\nexcept RuntimeError as e:\n    if \"should not be called\" in str(e):\n        raise TypeError(\"misrouted KV quant method\") from e\n    raise","preventionTips":["Branch on layer kind (Linear vs KV) before calling quant_method.apply","Never assume QuantizeMethodBase subclasses implement apply"],"tags":["kv-cache","quantization","api-misuse","not-implemented"],"backgroundTag":"base-method-not-implemented","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}