deepseek-ai/DeepSeek-V3 · critical · AssertionError

Number of experts must be divisible by world size (world_siz

Error message

Number of experts must be divisible by world size (world_size=${world_size})

What it means

Thrown in MoE.__init__ (inference/model.py:658): routed experts are partitioned so each rank holds n_routed_experts // world_size local experts (the experts ModuleList uses None placeholders for non-local slots). n_routed_experts (256 for DeepSeek-V3) must be divisible by world_size. Assert fires at model construction.

Source

Thrown at inference/model.py:658

    Attributes:
        dim (int): Dimensionality of input features.
        n_routed_experts (int): Total number of experts in the model.
        n_local_experts (int): Number of experts handled locally in distributed systems.
        n_activated_experts (int): Number of experts activated for each input.
        gate (nn.Module): Gating mechanism to route inputs to experts.
        experts (nn.ModuleList): List of expert modules.
        shared_experts (nn.Module): Shared experts applied to all inputs.
    """
    def __init__(self, args: ModelArgs):
        """
        Initializes the MoE module.

        Args:
            args (ModelArgs): Model arguments containing MoE parameters.
        """
        super().__init__()
        self.dim = args.dim
        assert args.n_routed_experts % world_size == 0, f"Number of experts must be divisible by world size (world_size={world_size})"
        self.n_routed_experts = args.n_routed_experts
        self.n_local_experts = args.n_routed_experts // world_size
        self.n_activated_experts = args.n_activated_experts
        self.experts_start_idx = rank * self.n_local_experts
        self.experts_end_idx = self.experts_start_idx + self.n_local_experts
        self.gate = Gate(args)
        self.experts = nn.ModuleList([Expert(args.dim, args.moe_inter_dim) if self.experts_start_idx <= i < self.experts_end_idx else None
                                      for i in range(self.n_routed_experts)])
        self.shared_experts = MLP(args.dim, args.n_shared_experts * args.moe_inter_dim)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """
        Forward pass for the MoE module.

        Args:
            x (torch.Tensor): Input tensor.

        Returns:

View on GitHub (pinned to 9b4e9788e4)

Solutions

Launch with a GPU count dividing n_routed_experts (256 divides cleanly by 2,4,8,16,32,64)
Keep the convert.py sharding (--model-parallel) consistent with the inference world_size
Validate the config programmatically before launch: assert args.n_routed_experts % world_size == 0

Example fix

# before
torchrun --nproc_per_node 6 generate.py ...  # 256 % 6 != 0

# after
torchrun --nproc_per_node 8 generate.py --ckpt-path ... --config ... --input-file prompts.txt

Defensive patterns

Strategy: validation

Validate before calling

world = dist.get_world_size() if dist.is_initialized() else 1
assert args.n_routed_experts % world == 0, (
    f"n_routed_experts={args.n_routed_experts} not divisible by world_size={world}"
)

Type guard

def experts_shardable(n_experts: int, world_size: int) -> bool:
    return n_experts % world_size == 0

Prevention

Keep convert.py --model-parallel equal to the inference world_size
For 256 routed experts use 2/4/8/16/32/64 GPUs
Add a startup assert so failure happens with a clear message, not deep in model construction

When it happens

Trigger: Constructing the MoE layer with world_size not dividing args.n_routed_experts — e.g. 3/5/6/7 GPUs against 256 experts, or a custom config with n_routed_experts like 7 launched on 2 ranks.

Common situations: Odd GPU counts; experimental configs that change n_routed_experts; also note this must match the --n-experts/--model-parallel values used when running convert.py to shard the checkpoint.

Related errors

AI-assisted analysis of deepseek-ai/DeepSeek-V3@9b4e9788e4 (2026-08-14). Data as JSON: /api/errors/e06e0ae2e6b2199f. Report an issue: GitHub.