sgl-project/sglang · error · ValueError

Mismatched ModelSlim quantization for W13 in layer {prefix}:

Error message

Mismatched ModelSlim quantization for W13 in layer {prefix}: {w13_entries}

What it means

For a fused MoE W13 weight (gate_proj + up_proj stored together), both sub-projections must be quantized with the identical ModelSlim scheme. This error fires when the quant_description maps gate and up to different scheme names (e.g. one W8A8 and one W4A4), which cannot be represented by a single fused w13 scheme.

Source

Thrown at python/sglang/srt/layers/quantization/modelslim/modelslim.py:376

        resolved_prefix = prefix
        for candidate in self._quant_prefix_candidates(prefix):
            for gate_name, up_name, down_name in naming_conventions:
                w13_keys = [
                    f"{candidate}.0.{gate_name}.weight",
                    f"{candidate}.0.{up_name}.weight",
                ]
                w2_key = f"{candidate}.0.{down_name}.weight"
                w13_entries = {
                    key: self.quant_description[key]
                    for key in w13_keys
                    if key in self.quant_description
                }
                if w13_entries and w2_key in self.quant_description:
                    w13_names = list(w13_entries.values())
                    # For w13, both projections must agree on the scheme
                    unique_w13 = set(w13_names)
                    if len(unique_w13) > 1:
                        raise ValueError(
                            "Mismatched ModelSlim quantization for W13 in layer "
                            f"{prefix}: {w13_entries}"
                        )
                    w13_scheme_name = w13_names[0]
                    w2_scheme_name = self.quant_description[w2_key]
                    resolved_prefix = candidate
                    break
            if w13_scheme_name is not None:
                break

        if w13_scheme_name is None:
            # Build a helpful error message listing all attempted key patterns
            all_attempted = []
            for candidate in self._quant_prefix_candidates(prefix):
                for gate_name, up_name, down_name in naming_conventions:
                    w13_keys = [
                        f"{candidate}.0.{gate_name}.weight",
                        f"{candidate}.0.{up_name}.weight",

View on GitHub (pinned to 0132848349)

Solutions

Open the ModelSlim quant description JSON and locate the entries reported in the error message
Make gate_proj and up_proj use the same scheme (re-run quantization for the whole layer, not per projection)
Verify the checkpoint wasn't assembled from two differently quantized runs
Regenerate the quantization config with a single msModelSlim run covering all MoE projections

Example fix

// quant_description (JSON) before
"...block.0.moe.gate_proj.weight": "W8A8",
"...block.0.moe.up_proj.weight": "W4A8"
// after
"...block.0.moe.gate_proj.weight": "W8A8",
"...block.0.moe.up_proj.weight": "W8A8"

Defensive patterns

Strategy: validation

Validate before calling

def w13_schemes_agree(qd: dict, prefix: str):
    g = qd.get(f"{prefix}.gate_proj.weight")
    u = qd.get(f"{prefix}.up_proj.weight")
    return g is not None and g == u

assert w13_schemes_agree(config.quant_description, moe_prefix), "gate/up schemes differ"

Try / catch

try:
    config.get_quant_method(layer, prefix)
except ValueError as e:
    if "Mismatched ModelSlim quantization for W13" in str(e):
        # fix quant_description and re-run
        raise

Prevention

Quantize whole layers in one msModelSlim run, never per projection
Script a sanity check that gate_proj and up_proj scheme strings match for every MoE block
Never hand-merge quant description JSONs from different runs

When it happens

Trigger: get_moe_scheme finds w13_entries for gate_proj and up_proj with differing scheme-name values (len(set(names)) > 1) while the w2 key exists — e.g. quant_description[prefix.gate_proj.weight]=='W8A8' but quant_description[prefix.up_proj.weight]=='W4A8'.

Common situations: Manually edited or merged ModelSlim quant configs; re-quantizing only some projections of a layer; a quantization tool bug or mixed calibration runs producing inconsistent per-projection dtypes.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f7ecd87b822ba932. Report an issue: GitHub.