sgl-project/sglang · critical · ValueError

Kimi expert-pack physical role order is unsupported

Error message

Kimi expert-pack physical role order is unsupported

What it means

The manifest's expert_pack.physical_role_order does not equal the fixed KIMI_PHYSICAL_ROLES tuple. The pack format stores each expert's up/gate/down tensors in a fixed physical order, and the loader only implements that one layout.

Source

Thrown at python/sglang/srt/layers/moe/expert_pack.py:766

            int(model["num_experts_per_token"]),
        )
        expected_dimensions = (expected_layers, expected_experts, expected_top_k)
        if dimensions != expected_dimensions or expected_top_k != 16:
            raise ValueError(
                f"Kimi expert-pack dimensions {dimensions} != {expected_dimensions}; "
                "Top-K is immutable at 16"
            )
        active_layers = tuple(int(value) for value in model["active_moe_layer_ids"])
        if active_layers != tuple(range(1, 93)):
            raise ValueError("Kimi active routed MoE layers must be exactly 1..92")

        pack_manifest = self.manifest["expert_pack"]
        if Path(pack_manifest["path"]).resolve() != self.path:
            raise ValueError("Kimi manifest expert-pack path does not match pack_path")
        if int(pack_manifest["size"]) != self.path.stat().st_size:
            raise ValueError("Kimi expert-pack size does not match its manifest")
        if pack_manifest.get("physical_role_order") != list(KIMI_PHYSICAL_ROLES):
            raise ValueError("Kimi expert-pack physical role order is unsupported")
        roles = pack_manifest["roles"]
        expected_roles = {
            "up": ("Q2_K", 10),
            "gate": ("Q2_K", 10),
            "down": ("Q3_K", 11),
        }
        for role, (dtype, dtype_id) in expected_roles.items():
            if (
                roles[role]["dtype"] != dtype
                or int(roles[role]["dtype_id"]) != dtype_id
            ):
                raise ValueError(f"Kimi expert-pack {role} quant type is unsupported")

        expected_entry_count = len(active_layers) * expected_experts * len(ROLE_NAMES)
        index_digest = hashlib.sha256()
        self.entries: dict[tuple[int, int, int], ExpertPackEntry] = {}
        self.object_offsets: dict[tuple[int, int], int] = {}
        object_payload_bytes = int(pack_manifest["object_bytes"])

View on GitHub (pinned to 0132848349)

Solutions

1. Re-pack the weights using the official packing tool so physical_role_order equals list(KIMI_PHYSICAL_ROLES)
2. Inspect KIMI_PHYSICAL_ROLES in expert_pack.py and compare with your manifest to see the expected order

Example fix

// before
"physical_role_order": ["down","gate","up"]
// after
"physical_role_order": list(KIMI_PHYSICAL_ROLES)

from sglang.srt.layers.moe.expert_pack import KIMI_PHYSICAL_ROLES
assert manifest['expert_pack']['physical_role_order'] == list(KIMI_PHYSICAL_ROLES)

When it happens

Trigger: A pack written with a different physical tensor ordering (e.g. down, gate, up), or a manifest hand-edited or produced by a third-party quantizer that permutes roles.

Common situations: See trigger scenarios.

Related errors

• Kimi active routed MoE layers must be exactly 1..92
• Kimi manifest expert-pack path does not match pack_path
• Kimi expert-pack size does not match its manifest
• Kimi expert-pack role sizes do not match object bytes
• Kimi expert-pack header is truncated

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/de4311f4857e275a. Report an issue: GitHub.