sgl-project/sglang · error · ValueError

A scheme must be defined for each layer

Error message

A scheme must be defined for each layer

What it means

CompressedTensorsLinearMethod.apply was called on a linear layer whose layer.scheme is None. The scheme is normally assigned during create_weights/process_weights; None means the quant config never resolved a scheme for this layer.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py:1233

            weight_loader=weight_loader,
        )

    def apply(
        self,
        layer: torch.nn.Module,
        x: torch.Tensor,
        bias: Optional[torch.Tensor] = None,
    ):
        """
        Use the output of create_weights and the CompressedTensorsScheme
        associated with the layer to apply the forward pass with the
        layer input.  See LinearMethodBase for param details

        """

        scheme = layer.scheme
        if scheme is None:
            raise ValueError("A scheme must be defined for each layer")
        return scheme.apply_weights(layer, x, bias=bias)


class CompressedTensorsFusedMoEMethod(FusedMoEMethodBase):
    def __init__(self, quantization_config: CompressedTensorsConfig):
        self.quantization_config = quantization_config
        self.quant_config = quantization_config

    def process_weights_after_loading(self, layer: torch.nn.Module) -> None:
        layer.scheme.process_weights_after_loading(layer)

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        params_dtype: torch.dtype,

View on GitHub (pinned to 0132848349)

Solutions

1. Check the checkpoint's quantization_config targets cover all linear layers (or add ignore so unquantized fallback is used)
2. Inspect layer.scheme right after model load and log the matched target for the failing layer
3. Ensure the config has a default/fallback for unmatched layers (empty targets string)

def assert_schemes_assigned(model):
    for name, m in model.named_modules():
        if hasattr(m, "scheme") and getattr(m, "quant_method", None) is not None:
            if type(m.quant_method).__name__ == "CompressedTensorsLinearMethod":
                assert m.scheme is not None, f"no scheme for {name}"

def has_scheme(layer) -> bool:
    return getattr(layer, "scheme", None) is not None

When it happens

Trigger: Forward pass through a Linear layer handled by CompressedTensorsLinearMethod where get_linear_scheme returned/assigned None — e.g. no quantization target matched the layer and no fallback scheme was set.

Common situations: A compressed-tensors config with targets that fail to match a layer's name/module type, leaving scheme unset; custom layer names not covered by the config's regex targets; library refactor changing scheme assignment order.

Related errors

• Unsupported quantized linear marker for {prefix!r}
• scalar_type_id {scalar_type_id} doesn't exists.
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• MXFP8 fused prologue requires head_dim-aligned Q/K/V.
• MXFP8 fused prologue requires K/V scale buffers.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e1d0ba004a8b85eb. Report an issue: GitHub.