sgl-project/sglang · critical · RuntimeError

{len(missing)} routed-expert tensors were not loaded (sample

Error message

{len(missing)} routed-expert tensors were not loaded (sample: {sample}). Expected {len(expected)} (layers={moe_layer_ids}, num_experts={self.config.num_experts}, shards=3).

What it means

After loading, Laguna verifies every routed-expert shard (per MoE layer x num_experts x {w1,w2,w3}) was filled; if any are missing it lists the count and a sample (laguna.py:846). Missing shards mean the checkpoint file set was incomplete or name mapping failed, which would leave experts with uninitialized weights.

Source

Thrown at python/sglang/srt/models/laguna.py:846

                logger.warning("Parameter %s not found in params_dict", name)
                continue
            param = params_dict[name]
            weight_loader = getattr(param, "weight_loader", default_weight_loader)
            weight_loader(param, loaded_weight)

        # If any routed-expert tensor was silently dropped (e.g. a future
        # checkpoint renaming `gate_proj`, or a ckpt-vs-mapping shape mismatch),
        # fail loud here instead of generating garbage.
        expected = {
            (layer_id, expert_id, shard_id)
            for layer_id in moe_layer_ids
            for expert_id in range(self.config.num_experts)
            for shard_id in ("w1", "w2", "w3")
        }
        missing = expected - loaded_expert_shards
        if missing:
            sample = sorted(missing)[:5]
            raise RuntimeError(
                f"{len(missing)} routed-expert tensors were not loaded "
                f"(sample: {sample}). Expected {len(expected)} (layers={moe_layer_ids}, "
                f"num_experts={self.config.num_experts}, shards=3)."
            )

    def get_embed_and_head(self):
        return self.model.embed_tokens.weight, self.lm_head.weight

    def set_embed_and_head(self, embed, head):
        del self.model.embed_tokens.weight
        del self.lm_head.weight
        self.model.embed_tokens.weight = embed
        self.lm_head.weight = head
        torch.cuda.empty_cache()
        torch.cuda.synchronize()

    def set_dflash_layers_to_capture(self, layer_ids: List[int]):
        if not self.pp_group.is_last_rank:

View on GitHub (pinned to 0132848349)

Solutions

1. Verify all checkpoint shard files are present and hashes match the hub manifest
2. Re-download the model
3. Inspect expert key names in the safetensors index and fix the mapping if a conversion renamed them

expected = len(moe_layers) * cfg.num_experts * 3
loaded = count_expert_tensors_in_index(path)
assert loaded >= expected, f"{loaded}/{expected} expert tensors"

When it happens

Trigger: Loading a sharded checkpoint with missing safetensors files, or expert tensor names that don't match the expected pattern (e.g. different expert prefix), leaving some experts.w{1,2,3}_weight slots empty.

Common situations: Interrupted checkpoint downloads, partial uploads, LoRA-merged exports with renamed expert keys, or quantized checkpoints lacking fp fallbacks for some shards.

Related errors

• QuantConfig has static quantization, but found activation sc
• Shared-sink down LoRA-A width must be divisible by {self.n_s
• Shared-sink gate/up LoRA-B height must be divisible by {self
• Expected {num_experts} experts in {name}, got {loaded_weight
• TP size {self.tp_size} > num_experts {config.num_experts}.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d0d118996b1288e0. Report an issue: GitHub.