{"record":{"id":"e1d0ba004a8b85eb","repo":"sgl-project/sglang","slug":"a-scheme-must-be-defined-for-each-layer-e1d0ba","errorCode":null,"errorMessage":"A scheme must be defined for each layer","messagePattern":"A scheme must be defined for each layer","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py","lineNumber":1233,"sourceCode":"            weight_loader=weight_loader,\n        )\n\n    def apply(\n        self,\n        layer: torch.nn.Module,\n        x: torch.Tensor,\n        bias: Optional[torch.Tensor] = None,\n    ):\n        \"\"\"\n        Use the output of create_weights and the CompressedTensorsScheme\n        associated with the layer to apply the forward pass with the\n        layer input.  See LinearMethodBase for param details\n\n        \"\"\"\n\n        scheme = layer.scheme\n        if scheme is None:\n            raise ValueError(\"A scheme must be defined for each layer\")\n        return scheme.apply_weights(layer, x, bias=bias)\n\n\nclass CompressedTensorsFusedMoEMethod(FusedMoEMethodBase):\n    def __init__(self, quantization_config: CompressedTensorsConfig):\n        self.quantization_config = quantization_config\n        self.quant_config = quantization_config\n\n    def process_weights_after_loading(self, layer: torch.nn.Module) -> None:\n        layer.scheme.process_weights_after_loading(layer)\n\n    def create_weights(\n        self,\n        layer: torch.nn.Module,\n        num_experts: int,\n        hidden_size: int,\n        intermediate_size_per_partition: int,\n        params_dtype: torch.dtype,","sourceCodeStart":1215,"sourceCodeEnd":1251,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py#L1215-L1251","documentation":"CompressedTensorsLinearMethod.apply was called on a linear layer whose layer.scheme is None. The scheme is normally assigned during create_weights/process_weights; None means the quant config never resolved a scheme for this layer.","triggerScenarios":"Forward pass through a Linear layer handled by CompressedTensorsLinearMethod where get_linear_scheme returned/assigned None — e.g. no quantization target matched the layer and no fallback scheme was set.","commonSituations":"A compressed-tensors config with targets that fail to match a layer's name/module type, leaving scheme unset; custom layer names not covered by the config's regex targets; library refactor changing scheme assignment order.","solutions":["Check the checkpoint's quantization_config targets cover all linear layers (or add ignore so unquantized fallback is used)","Inspect layer.scheme right after model load and log the matched target for the failing layer","Ensure the config has a default/fallback for unmatched layers (empty targets string)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"def assert_schemes_assigned(model):\n    for name, m in model.named_modules():\n        if hasattr(m, \"scheme\") and getattr(m, \"quant_method\", None) is not None:\n            if type(m.quant_method).__name__ == \"CompressedTensorsLinearMethod\":\n                assert m.scheme is not None, f\"no scheme for {name}\"","typeGuard":"def has_scheme(layer) -> bool:\n    return getattr(layer, \"scheme\", None) is not None","tryCatchPattern":null,"preventionTips":["After model load, sweep named_modules asserting scheme is not None for quantized layers","Ensure quant config targets cover or explicitly ignore every linear layer","Add a load-time smoke forward (dummy tokens) before serving"],"tags":["quantization","scheme","linear-layer","internal-invariant"],"backgroundTag":"null-scheme-invariant","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}