{"record":{"id":"4634dee256dbc1f2","repo":"sgl-project/sglang","slug":"mobius-fused-gate-up-destination-is-missing-para","errorCode":null,"errorMessage":"Mobius fused gate/up destination is missing: {parameter_name}","messagePattern":"Mobius fused gate/up destination is missing: (.+?)","errorType":"exception","errorClass":"KeyError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/interns2_mobius.py","lineNumber":118,"sourceCode":"    name: str,\n    loaded_weight: torch.Tensor,\n    params_dict: dict[str, nn.Parameter],\n    num_experts: int,\n    record_slot,\n) -> None:\n    gate_up_suffixes = {\n        \"experts.gate_up_proj\": \"experts.w13_weight\",\n        \"experts.gate_up_proj_scale_inv\": \"experts.w13_weight_scale_inv\",\n    }\n    gate_up_suffix = next(\n        (suffix for suffix in gate_up_suffixes if name.endswith(suffix)), None\n    )\n    if gate_up_suffix is not None:\n        parameter_name = (\n            name.removesuffix(gate_up_suffix) + gate_up_suffixes[gate_up_suffix]\n        )\n        if parameter_name not in params_dict:\n            raise KeyError(\n                f\"Mobius fused gate/up destination is missing: {parameter_name}\"\n            )\n        if loaded_weight.shape[0] != num_experts:\n            raise ValueError(\n                f\"Expected {num_experts} experts in {name}, got {loaded_weight.shape[0]}\"\n            )\n        gate_weights, up_weights = loaded_weight.chunk(2, dim=-2)\n        parameter = params_dict[parameter_name]\n        loader = parameter.weight_loader\n        for expert_id in range(num_experts):\n            for shard_id, expert_weight in (\n                (\"w1\", gate_weights[expert_id]),\n                (\"w3\", up_weights[expert_id]),\n            ):\n                record_slot(parameter_name, shard_id, expert_id)\n                loader(\n                    parameter,\n                    expert_weight,","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/interns2_mobius.py#L100-L136","documentation":"While loading fused Mobius expert weights, the computed destination parameter name (source name with the gate/up suffix rewritten via gate_up_suffixes mapping) was not found in params_dict, so there is no module to load into.","triggerScenarios":"A checkpoint key matching a known gate_up suffix maps to a parameter name absent from the model, e.g. the model was built without the fused expert modules or suffix mapping drifted from the checkpoint naming.","commonSituations":"Checkpoint/model version mismatch in fused expert naming; sharding config that skips creating certain expert parameters.","solutions":["Print params_dict keys and the computed parameter_name to identify the naming mismatch","Ensure the model config creates the fused Mobius expert modules the checkpoint expects","Re-export the checkpoint with naming matching the current suffix mapping"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"expected = set(params_dict)\nmapped = compute_destination(name)\nassert mapped in expected, (mapped, name)","typeGuard":null,"tryCatchPattern":"try:\n    load(...)\nexcept KeyError as e:\n    if 'fused gate/up destination is missing' in str(e):\n        log_param_names(); raise","preventionTips":["Diff checkpoint key names against params_dict before loading in CI"],"tags":["weight-loading","key-mapping","mobius","interns2"],"backgroundTag":"checkpoint-key-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}