sgl-project/sglang · critical · ValueError

Kimi expert-pack {role} quant type is unsupported

Error message

Kimi expert-pack {role} quant type is unsupported

What it means

For each of up/gate/down the manifest's roles[role].dtype and dtype_id must match the fixed expected quantization (up/gate = Q2_K id 10, down = Q3_K id 11). Any other quantization scheme is unsupported by this loader's dequantization path.

Source

Thrown at python/sglang/srt/layers/moe/expert_pack.py:778

        pack_manifest = self.manifest["expert_pack"]
        if Path(pack_manifest["path"]).resolve() != self.path:
            raise ValueError("Kimi manifest expert-pack path does not match pack_path")
        if int(pack_manifest["size"]) != self.path.stat().st_size:
            raise ValueError("Kimi expert-pack size does not match its manifest")
        if pack_manifest.get("physical_role_order") != list(KIMI_PHYSICAL_ROLES):
            raise ValueError("Kimi expert-pack physical role order is unsupported")
        roles = pack_manifest["roles"]
        expected_roles = {
            "up": ("Q2_K", 10),
            "gate": ("Q2_K", 10),
            "down": ("Q3_K", 11),
        }
        for role, (dtype, dtype_id) in expected_roles.items():
            if (
                roles[role]["dtype"] != dtype
                or int(roles[role]["dtype_id"]) != dtype_id
            ):
                raise ValueError(f"Kimi expert-pack {role} quant type is unsupported")

        expected_entry_count = len(active_layers) * expected_experts * len(ROLE_NAMES)
        index_digest = hashlib.sha256()
        self.entries: dict[tuple[int, int, int], ExpertPackEntry] = {}
        self.object_offsets: dict[tuple[int, int], int] = {}
        object_payload_bytes = int(pack_manifest["object_bytes"])
        previous_end = int(pack_manifest["data_start"])
        role_offsets: dict[str, int] = {}
        role_nbytes = {
            role: int(roles[role]["expert_bytes"]) for role in KIMI_PHYSICAL_ROLES
        }
        running_role_offset = 0
        for role in KIMI_PHYSICAL_ROLES:
            role_offsets[role] = running_role_offset
            running_role_offset += role_nbytes[role]
        if running_role_offset != object_payload_bytes:
            raise ValueError("Kimi expert-pack role sizes do not match object bytes")

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize/pack so up and gate are Q2_K (dtype_id 10) and down is Q3_K (dtype_id 11)
2. Verify manifest roles entries: {"up": ["Q2_K", 10], "gate": ["Q2_K", 10], "down": ["Q3_K", 11]}
3. If you need other quant types, extend expected_roles and the corresponding dequant kernels

Example fix

// before
"roles": {"up": {"dtype": "Q4_K", "dtype_id": 12}, ...}
// after
"roles": {"up": {"dtype": "Q2_K", "dtype_id": 10}, "gate": {"dtype": "Q2_K", "dtype_id": 10}, "down": {"dtype": "Q3_K", "dtype_id": 11}}

expected = {'up': ('Q2_K', 10), 'gate': ('Q2_K', 10), 'down': ('Q3_K', 11)}
roles = manifest['expert_pack']['roles']
for r, (dt, did) in expected.items():
    assert roles[r]['dtype'] == dt and int(roles[r]['dtype_id']) == did

When it happens

Trigger: Loading a pack quantized with different GGML types (e.g. Q4_K for gate, Q8_0 for down), or a manifest with mismatched dtype_id values.

Common situations: Re-quantizing the pack with custom GGUF quant levels; using a third-party quantizer that emits different per-role types.

Related errors

• Unsupported dtype {dtype}. Supported: float16, bfloat16, flo
• Unsupported Quanto weight type for {prefix!r}: {quantization
• Kimi active routed MoE layers must be exactly 1..92
• Kimi manifest expert-pack path does not match pack_path
• Kimi expert-pack size does not match its manifest

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e5d4f9b0784a8f1d. Report an issue: GitHub.