sgl-project/sglang · error · ValueError

A scheme must be defined for each layer

Error message

A scheme must be defined for each layer

What it means

ModelSlimLinearMethod.apply (or the apply_weights wrapper) requires layer.scheme to have been set before the forward call; the scheme carries the actual weight-application logic. A None scheme means the layer was never assigned a ModelSlim scheme during quant-method creation, so there is nothing to dispatch to.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/modelslim.py:524

            weight_loader=weight_loader,
        )

    def apply(
        self,
        layer: torch.nn.Module,
        x: torch.Tensor,
        bias: Optional[torch.Tensor] = None,
    ):
        """
        Use the output of create_weights and the ModelSlimLinearScheme
        associated with the layer to apply the forward pass with the
        layer input.  See LinearMethodBase for param details

        """

        scheme = layer.scheme
        if scheme is None:
            raise ValueError("A scheme must be defined for each layer")
        return scheme.apply_weights(layer, x, bias=bias)


class ModelSlimFusedMoEMethod(FusedMoEMethodBase):
    """
    Fused MoE method for ModelSlim quantization on Ascend NPU.

    Delegates routing, activation, and finalization to the modular NPU MoE
    components introduced in the hardware backend refactoring.
    """

    def __init__(self, quantization_config: ModelSlimConfig):
        self.quantization_config = quantization_config

    def process_weights_after_loading(self, layer: torch.nn.Module) -> None:
        layer.w13_scheme.process_weights_after_loading(layer)
        layer.w2_scheme.process_weights_after_loading(layer)

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the layer went through ModelSlimConfig.get_quant_method so layer.scheme is populated before forward
2. If constructing layers manually, set layer.scheme from the config (e.g. via the same scheme resolution used in get_quant_method) before calling apply
3. Audit custom model code or monkey-patches that might clear or overwrite .scheme
4. Upgrade SGLang if hitting this on stock models — it indicates broken quant-method/layer pairing

Example fix

# before
method = ModelSlimLinearMethod(config)
out = method.apply(layer, x, bias)  # layer.scheme is None -> raises
# after
quant_method = config.get_quant_method(layer, prefix)  # sets layer.scheme
out = quant_method.apply(layer, x, bias)

def scheme_ready(layer) -> bool:
    return getattr(layer, "scheme", None) is not None

assert scheme_ready(layer), "layer.scheme not set; run get_quant_method first"

from sglang.srt.layers.quantization import LinearMethodBase

def has_scheme(layer) -> bool:
    return getattr(layer, "scheme", None) is not None

try:
    out = method.apply(layer, x, bias=bias)
except ValueError as e:
    if "scheme must be defined" in str(e) and getattr(layer, "scheme", None) is None:
        layer.scheme = config.get_scheme(layer, prefix)  # re-initialize
        out = method.apply(layer, x, bias=bias)
    else:
        raise

When it happens

Trigger: Calling apply()/apply_weights on a linear layer whose .scheme attribute is None — typically because get_quant_method returned UnquantizedLinearMethod path or the layer was constructed without scheme assignment, yet ModelSlimLinearMethod was invoked at runtime.

Common situations: Custom model code manually instantiating ModelSlimLinearMethod without going through get_quant_method; process_weight_loader or a patch resetting layer.scheme; mixing quantization method objects with layers created by a different config path; subclassing FusedMoE/Linear and bypassing scheme setup.

Related errors

• No ModelSlim MoE scheme found for layer {prefix}
• Mismatched ModelSlim quantization for W13 in layer {prefix}:
• Missing ModelSlim MoE quantization description for layer {pr
• Unsupported ModelSlim MoE schemes for layer {prefix}: W13='{
• Detected some but not all shards of {prefix} are quantized.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6b89bc484b5065ae. Report an issue: GitHub.