sgl-project/sglang · error · KeyError

Mobius fused gate/up destination is missing: {parameter_name

Error message

Mobius fused gate/up destination is missing: {parameter_name}

What it means

While loading fused Mobius expert weights, the computed destination parameter name (source name with the gate/up suffix rewritten via gate_up_suffixes mapping) was not found in params_dict, so there is no module to load into.

Source

Thrown at python/sglang/srt/models/interns2_mobius.py:118

    name: str,
    loaded_weight: torch.Tensor,
    params_dict: dict[str, nn.Parameter],
    num_experts: int,
    record_slot,
) -> None:
    gate_up_suffixes = {
        "experts.gate_up_proj": "experts.w13_weight",
        "experts.gate_up_proj_scale_inv": "experts.w13_weight_scale_inv",
    }
    gate_up_suffix = next(
        (suffix for suffix in gate_up_suffixes if name.endswith(suffix)), None
    )
    if gate_up_suffix is not None:
        parameter_name = (
            name.removesuffix(gate_up_suffix) + gate_up_suffixes[gate_up_suffix]
        )
        if parameter_name not in params_dict:
            raise KeyError(
                f"Mobius fused gate/up destination is missing: {parameter_name}"
            )
        if loaded_weight.shape[0] != num_experts:
            raise ValueError(
                f"Expected {num_experts} experts in {name}, got {loaded_weight.shape[0]}"
            )
        gate_weights, up_weights = loaded_weight.chunk(2, dim=-2)
        parameter = params_dict[parameter_name]
        loader = parameter.weight_loader
        for expert_id in range(num_experts):
            for shard_id, expert_weight in (
                ("w1", gate_weights[expert_id]),
                ("w3", up_weights[expert_id]),
            ):
                record_slot(parameter_name, shard_id, expert_id)
                loader(
                    parameter,
                    expert_weight,

View on GitHub (pinned to 0132848349)

Solutions

Print params_dict keys and the computed parameter_name to identify the naming mismatch
Ensure the model config creates the fused Mobius expert modules the checkpoint expects
Re-export the checkpoint with naming matching the current suffix mapping

Defensive patterns

Strategy: validation

Validate before calling

expected = set(params_dict)
mapped = compute_destination(name)
assert mapped in expected, (mapped, name)

Try / catch

try:
    load(...)
except KeyError as e:
    if 'fused gate/up destination is missing' in str(e):
        log_param_names(); raise

Prevention

Diff checkpoint key names against params_dict before loading in CI

When it happens

Trigger: A checkpoint key matching a known gate_up suffix maps to a parameter name absent from the model, e.g. the model was built without the fused expert modules or suffix mapping drifted from the checkpoint naming.

Common situations: Checkpoint/model version mismatch in fused expert naming; sharding config that skips creating certain expert parameters.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4634dee256dbc1f2. Report an issue: GitHub.