{"record":{"id":"5f1385b97768ebad","repo":"sgl-project/sglang","slug":"expected-num-experts-experts-in-name-got-loa","errorCode":null,"errorMessage":"Expected {num_experts} experts in {name}, got {loaded_weight.shape[0]}","messagePattern":"Expected (.+?) experts in (.+?), got (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/interns2_mobius.py","lineNumber":122,"sourceCode":"    record_slot,\n) -> None:\n    gate_up_suffixes = {\n        \"experts.gate_up_proj\": \"experts.w13_weight\",\n        \"experts.gate_up_proj_scale_inv\": \"experts.w13_weight_scale_inv\",\n    }\n    gate_up_suffix = next(\n        (suffix for suffix in gate_up_suffixes if name.endswith(suffix)), None\n    )\n    if gate_up_suffix is not None:\n        parameter_name = (\n            name.removesuffix(gate_up_suffix) + gate_up_suffixes[gate_up_suffix]\n        )\n        if parameter_name not in params_dict:\n            raise KeyError(\n                f\"Mobius fused gate/up destination is missing: {parameter_name}\"\n            )\n        if loaded_weight.shape[0] != num_experts:\n            raise ValueError(\n                f\"Expected {num_experts} experts in {name}, got {loaded_weight.shape[0]}\"\n            )\n        gate_weights, up_weights = loaded_weight.chunk(2, dim=-2)\n        parameter = params_dict[parameter_name]\n        loader = parameter.weight_loader\n        for expert_id in range(num_experts):\n            for shard_id, expert_weight in (\n                (\"w1\", gate_weights[expert_id]),\n                (\"w3\", up_weights[expert_id]),\n            ):\n                record_slot(parameter_name, shard_id, expert_id)\n                loader(\n                    parameter,\n                    expert_weight,\n                    parameter_name,\n                    shard_id,\n                    expert_id,\n                )","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/interns2_mobius.py#L104-L140","documentation":"When loading a fused Mobius gate/up expert tensor, its leading dimension (expert count in the checkpoint) must equal the model's num_experts; otherwise slicing per expert would be wrong and it raises ValueError.","triggerScenarios":"loaded_weight.shape[0] != num_experts for a tensor matching a gate_up suffix — e.g. checkpoint trained with a different expert count than the served config.","commonSituations":"Model config num_experts changed between training and serving; partial expert-parallel shard loaded without the EP dimension handled.","solutions":["Align the config's num_local/moe experts with the checkpoint's expert dimension","If using expert parallelism, load the correct local shard so shape[0] equals the per-rank expert count"],"exampleFix":"# before\n# config.num_experts = 32, checkpoint tensor shape [64, N, K]\n# after\nconfig.num_experts = 64  # or shard the tensor for EP","handlingStrategy":"validation","validationCode":"assert loaded_weight.shape[0] == config.num_experts, (loaded_weight.shape, config.num_experts)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate expert counts of every fused expert tensor during checkpoint preflight"],"tags":["weight-loading","expert-count-mismatch","mobius","moe"],"backgroundTag":"checkpoint-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}