{"record":{"id":"6b89bc484b5065ae","repo":"sgl-project/sglang","slug":"a-scheme-must-be-defined-for-each-layer-6b89bc","errorCode":null,"errorMessage":"A scheme must be defined for each layer","messagePattern":"A scheme must be defined for each layer","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/modelslim/modelslim.py","lineNumber":524,"sourceCode":"            weight_loader=weight_loader,\n        )\n\n    def apply(\n        self,\n        layer: torch.nn.Module,\n        x: torch.Tensor,\n        bias: Optional[torch.Tensor] = None,\n    ):\n        \"\"\"\n        Use the output of create_weights and the ModelSlimLinearScheme\n        associated with the layer to apply the forward pass with the\n        layer input.  See LinearMethodBase for param details\n\n        \"\"\"\n\n        scheme = layer.scheme\n        if scheme is None:\n            raise ValueError(\"A scheme must be defined for each layer\")\n        return scheme.apply_weights(layer, x, bias=bias)\n\n\nclass ModelSlimFusedMoEMethod(FusedMoEMethodBase):\n    \"\"\"\n    Fused MoE method for ModelSlim quantization on Ascend NPU.\n\n    Delegates routing, activation, and finalization to the modular NPU MoE\n    components introduced in the hardware backend refactoring.\n    \"\"\"\n\n    def __init__(self, quantization_config: ModelSlimConfig):\n        self.quantization_config = quantization_config\n\n    def process_weights_after_loading(self, layer: torch.nn.Module) -> None:\n        layer.w13_scheme.process_weights_after_loading(layer)\n        layer.w2_scheme.process_weights_after_loading(layer)\n","sourceCodeStart":506,"sourceCodeEnd":542,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/modelslim/modelslim.py#L506-L542","documentation":"ModelSlimLinearMethod.apply (or the apply_weights wrapper) requires layer.scheme to have been set before the forward call; the scheme carries the actual weight-application logic. A None scheme means the layer was never assigned a ModelSlim scheme during quant-method creation, so there is nothing to dispatch to.","triggerScenarios":"Calling apply()/apply_weights on a linear layer whose .scheme attribute is None — typically because get_quant_method returned UnquantizedLinearMethod path or the layer was constructed without scheme assignment, yet ModelSlimLinearMethod was invoked at runtime.","commonSituations":"Custom model code manually instantiating ModelSlimLinearMethod without going through get_quant_method; process_weight_loader or a patch resetting layer.scheme; mixing quantization method objects with layers created by a different config path; subclassing FusedMoE/Linear and bypassing scheme setup.","solutions":["Ensure the layer went through ModelSlimConfig.get_quant_method so layer.scheme is populated before forward","If constructing layers manually, set layer.scheme from the config (e.g. via the same scheme resolution used in get_quant_method) before calling apply","Audit custom model code or monkey-patches that might clear or overwrite .scheme","Upgrade SGLang if hitting this on stock models — it indicates broken quant-method/layer pairing"],"exampleFix":"# before\nmethod = ModelSlimLinearMethod(config)\nout = method.apply(layer, x, bias)  # layer.scheme is None -> raises\n# after\nquant_method = config.get_quant_method(layer, prefix)  # sets layer.scheme\nout = quant_method.apply(layer, x, bias)","handlingStrategy":"try-catch","validationCode":"def scheme_ready(layer) -> bool:\n    return getattr(layer, \"scheme\", None) is not None\n\nassert scheme_ready(layer), \"layer.scheme not set; run get_quant_method first\"","typeGuard":"from sglang.srt.layers.quantization import LinearMethodBase\n\ndef has_scheme(layer) -> bool:\n    return getattr(layer, \"scheme\", None) is not None","tryCatchPattern":"try:\n    out = method.apply(layer, x, bias=bias)\nexcept ValueError as e:\n    if \"scheme must be defined\" in str(e) and getattr(layer, \"scheme\", None) is None:\n        layer.scheme = config.get_scheme(layer, prefix)  # re-initialize\n        out = method.apply(layer, x, bias=bias)\n    else:\n        raise","preventionTips":["Always obtain quant methods via config.get_quant_method, never construct ModelSlimLinearMethod directly","Verify layer.scheme is set right after model build in custom models","Avoid monkey-patches that reset layer attributes"],"tags":["modelslim","quantization","scheme-uninitialized","runtime"],"backgroundTag":"uninitialized-quantization-scheme","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}