{"record":{"id":"6c75edced53aa376","repo":"sgl-project/sglang","slug":"kimi-expert-object-is-not-contiguous-at-index-ind","errorCode":null,"errorMessage":"Kimi expert object is not contiguous at index {index}","messagePattern":"Kimi expert object is not contiguous at index (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/layers/moe/expert_pack.py","lineNumber":856,"sourceCode":"                expected_nbytes = role_nbytes[physical_role]\n                if (\n                    nbytes != expected_nbytes\n                    or offset % int(pack_manifest[\"alignment\"])\n                    or offset < previous_end\n                    or offset + nbytes > self.path.stat().st_size\n                ):\n                    raise ValueError(\n                        f\"Kimi expert-pack range mismatch at index {index}\"\n                    )\n                previous_end = offset + nbytes\n                object_key = (layer, expert)\n                if physical_role_id == 0:\n                    self.object_offsets[object_key] = offset\n                expected_offset = (\n                    self.object_offsets[object_key] + role_offsets[physical_role]\n                )\n                if offset != expected_offset:\n                    raise ValueError(\n                        f\"Kimi expert object is not contiguous at index {index}\"\n                    )\n                generation_bytes = hashlib.sha256(\n                    f\"{pack_manifest['index_sha256']}:{layer}:{expert}\".encode(\"ascii\")\n                ).digest()\n                generation = int.from_bytes(generation_bytes[:8], \"little\") or 1\n                logical_role_id = ROLE_NAMES.index(physical_role)\n                logical_shape = tuple(\n                    int(value) for value in roles[physical_role][\"logical_shape\"]\n                )\n                self.entries[(layer, expert, logical_role_id)] = ExpertPackEntry(\n                    layer=layer,\n                    expert=expert,\n                    role_id=logical_role_id,\n                    dtype_id=int(roles[physical_role][\"dtype_id\"]),\n                    dtype=str(roles[physical_role][\"dtype\"]),\n                    tensor_name=name,\n                    source_slice_offset=0,","sourceCodeStart":838,"sourceCodeEnd":874,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/expert_pack.py#L838-L874","documentation":"For each (layer, expert) object, the three role blobs must be physically contiguous: each role's offset must equal the object's base offset plus the role's cumulative role_offsets. A mismatch means the pack interleaves objects or has gaps, which the mmap-based loader cannot handle.","triggerScenarios":"A pack that stores tensors grouped by role (all ups, then all gates) instead of grouped per expert; or gaps/inserted padding between an expert's role blobs.","commonSituations":"See trigger scenarios.","solutions":["Re-pack so each expert's up/gate/down blobs are adjacent in the fixed physical role order","If writing a custom packer, compute each role offset as object_base + role_offsets[role]"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"null","typeGuard":null,"tryCatchPattern":"try:\n    pack = KimiExpertPack(path, manifest)\nexcept ValueError as e:\n    if 'not contiguous' in str(e):\n        log.error('pack layout groups by role, not per-expert; re-pack required')\n    raise","preventionTips":["Use the official packer which writes per-expert contiguous blobs"],"tags":["kimi","moe","expert-pack","layout"],"backgroundTag":"unsupported-layout-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}