{"record":{"id":"e5d4f9b0784a8f1d","repo":"sgl-project/sglang","slug":"kimi-expert-pack-role-quant-type-is-unsupported","errorCode":null,"errorMessage":"Kimi expert-pack {role} quant type is unsupported","messagePattern":"Kimi expert-pack (.+?) quant type is unsupported","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/layers/moe/expert_pack.py","lineNumber":778,"sourceCode":"        pack_manifest = self.manifest[\"expert_pack\"]\n        if Path(pack_manifest[\"path\"]).resolve() != self.path:\n            raise ValueError(\"Kimi manifest expert-pack path does not match pack_path\")\n        if int(pack_manifest[\"size\"]) != self.path.stat().st_size:\n            raise ValueError(\"Kimi expert-pack size does not match its manifest\")\n        if pack_manifest.get(\"physical_role_order\") != list(KIMI_PHYSICAL_ROLES):\n            raise ValueError(\"Kimi expert-pack physical role order is unsupported\")\n        roles = pack_manifest[\"roles\"]\n        expected_roles = {\n            \"up\": (\"Q2_K\", 10),\n            \"gate\": (\"Q2_K\", 10),\n            \"down\": (\"Q3_K\", 11),\n        }\n        for role, (dtype, dtype_id) in expected_roles.items():\n            if (\n                roles[role][\"dtype\"] != dtype\n                or int(roles[role][\"dtype_id\"]) != dtype_id\n            ):\n                raise ValueError(f\"Kimi expert-pack {role} quant type is unsupported\")\n\n        expected_entry_count = len(active_layers) * expected_experts * len(ROLE_NAMES)\n        index_digest = hashlib.sha256()\n        self.entries: dict[tuple[int, int, int], ExpertPackEntry] = {}\n        self.object_offsets: dict[tuple[int, int], int] = {}\n        object_payload_bytes = int(pack_manifest[\"object_bytes\"])\n        previous_end = int(pack_manifest[\"data_start\"])\n        role_offsets: dict[str, int] = {}\n        role_nbytes = {\n            role: int(roles[role][\"expert_bytes\"]) for role in KIMI_PHYSICAL_ROLES\n        }\n        running_role_offset = 0\n        for role in KIMI_PHYSICAL_ROLES:\n            role_offsets[role] = running_role_offset\n            running_role_offset += role_nbytes[role]\n        if running_role_offset != object_payload_bytes:\n            raise ValueError(\"Kimi expert-pack role sizes do not match object bytes\")\n","sourceCodeStart":760,"sourceCodeEnd":796,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/expert_pack.py#L760-L796","documentation":"For each of up/gate/down the manifest's roles[role].dtype and dtype_id must match the fixed expected quantization (up/gate = Q2_K id 10, down = Q3_K id 11). Any other quantization scheme is unsupported by this loader's dequantization path.","triggerScenarios":"Loading a pack quantized with different GGML types (e.g. Q4_K for gate, Q8_0 for down), or a manifest with mismatched dtype_id values.","commonSituations":"Re-quantizing the pack with custom GGUF quant levels; using a third-party quantizer that emits different per-role types.","solutions":["Re-quantize/pack so up and gate are Q2_K (dtype_id 10) and down is Q3_K (dtype_id 11)","Verify manifest roles entries: {\"up\": [\"Q2_K\", 10], \"gate\": [\"Q2_K\", 10], \"down\": [\"Q3_K\", 11]}","If you need other quant types, extend expected_roles and the corresponding dequant kernels"],"exampleFix":"// before\n\"roles\": {\"up\": {\"dtype\": \"Q4_K\", \"dtype_id\": 12}, ...}\n// after\n\"roles\": {\"up\": {\"dtype\": \"Q2_K\", \"dtype_id\": 10}, \"gate\": {\"dtype\": \"Q2_K\", \"dtype_id\": 10}, \"down\": {\"dtype\": \"Q3_K\", \"dtype_id\": 11}}","handlingStrategy":"validation","validationCode":"expected = {'up': ('Q2_K', 10), 'gate': ('Q2_K', 10), 'down': ('Q3_K', 11)}\nroles = manifest['expert_pack']['roles']\nfor r, (dt, did) in expected.items():\n    assert roles[r]['dtype'] == dt and int(roles[r]['dtype_id']) == did","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use the official Kimi Q2_K/Q3_K quantization recipe","Don't re-quantize packs with custom GGML types unless you also patch the loader"],"tags":["kimi","moe","quantization","unsupported-dtype"],"backgroundTag":"unsupported-quantization-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}