sgl-project/sglang · critical · RuntimeError

routed-expert tensors were not loaded (sample: ). Expected…

Error message

{len(missing)} routed-expert tensors were not loaded (sample: {sample}). Expected {len(expected)} (layers={moe_layer_ids}, num_experts={self.config.num_experts}, shards=3).

What it means

After loading, Laguna verifies every routed-expert shard (per MoE layer x num_experts x {w1,w2,w3}) was filled; if any are missing it lists the count and a sample (laguna.py:846). Missing shards mean the checkpoint file set was incomplete or name mapping failed, which would leave experts with uninitialized weights.

Solutions

1. Verify all checkpoint shard files are present and hashes match the hub manifest
2. Re-download the model
3. Inspect expert key names in the safetensors index and fix the mapping if a conversion renamed them

expected = len(moe_layers) * cfg.num_experts * 3
loaded = count_expert_tensors_in_index(path)
assert loaded >= expected, f"{loaded}/{expected} expert tensors"

When it happens

Trigger: Loading a sharded checkpoint with missing safetensors files, or expert tensor names that don't match the expected pattern (e.g. different expert prefix), leaving some experts.w{1,2,3}_weight slots empty.

Common situations: Interrupted checkpoint downloads, partial uploads, LoRA-merged exports with renamed expert keys, or quantized checkpoints lacking fp fallbacks for some shards.

Related errors

• Checkpoint provides gate weight
• Expected a 3D packed tensor for
• Expected experts in , got
• Invalid gate_up_proj shape for
• QuantConfig has static quantization, but found activation…

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d0d118996b1288e0. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/models/laguna.py:846

                logger.warning("Parameter %s not found in params_dict", name)
                continue
            param = params_dict[name]
            weight_loader = getattr(param, "weight_loader", default_weight_loader)
            weight_loader(param, loaded_weight)

        # If any routed-expert tensor was silently dropped (e.g. a future
        # checkpoint renaming `gate_proj`, or a ckpt-vs-mapping shape mismatch),
        # fail loud here instead of generating garbage.
        expected = {
            (layer_id, expert_id, shard_id)
            for layer_id in moe_layer_ids
            for expert_id in range(self.config.num_experts)
            for shard_id in ("w1", "w2", "w3")
        }
        missing = expected - loaded_expert_shards
        if missing:
            sample = sorted(missing)[:5]
            raise RuntimeError(
                f"{len(missing)} routed-expert tensors were not loaded "
                f"(sample: {sample}). Expected {len(expected)} (layers={moe_layer_ids}, "
                f"num_experts={self.config.num_experts}, shards=3)."
            )

    def get_embed_and_head(self):
        return self.model.embed_tokens.weight, self.lm_head.weight

    def set_embed_and_head(self, embed, head):
        del self.model.embed_tokens.weight
        del self.lm_head.weight
        self.model.embed_tokens.weight = embed
        self.lm_head.weight = head
        torch.cuda.empty_cache()
        torch.cuda.synchronize()

    def set_dflash_layers_to_capture(self, layer_ids: List[int]):
        if not self.pp_group.is_last_rank:

View on GitHub (pinned to 0132848349)