{"record":{"id":"e06e0ae2e6b2199f","repo":"deepseek-ai/DeepSeek-V3","slug":"number-of-experts-must-be-divisible-by-world-size","errorCode":null,"errorMessage":"Number of experts must be divisible by world size (world_size=${world_size})","messagePattern":"Number of experts must be divisible by world size \\(world_size=(.+?)\\)","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"critical","filePath":"inference/model.py","lineNumber":658,"sourceCode":"    Attributes:\n        dim (int): Dimensionality of input features.\n        n_routed_experts (int): Total number of experts in the model.\n        n_local_experts (int): Number of experts handled locally in distributed systems.\n        n_activated_experts (int): Number of experts activated for each input.\n        gate (nn.Module): Gating mechanism to route inputs to experts.\n        experts (nn.ModuleList): List of expert modules.\n        shared_experts (nn.Module): Shared experts applied to all inputs.\n    \"\"\"\n    def __init__(self, args: ModelArgs):\n        \"\"\"\n        Initializes the MoE module.\n\n        Args:\n            args (ModelArgs): Model arguments containing MoE parameters.\n        \"\"\"\n        super().__init__()\n        self.dim = args.dim\n        assert args.n_routed_experts % world_size == 0, f\"Number of experts must be divisible by world size (world_size={world_size})\"\n        self.n_routed_experts = args.n_routed_experts\n        self.n_local_experts = args.n_routed_experts // world_size\n        self.n_activated_experts = args.n_activated_experts\n        self.experts_start_idx = rank * self.n_local_experts\n        self.experts_end_idx = self.experts_start_idx + self.n_local_experts\n        self.gate = Gate(args)\n        self.experts = nn.ModuleList([Expert(args.dim, args.moe_inter_dim) if self.experts_start_idx <= i < self.experts_end_idx else None\n                                      for i in range(self.n_routed_experts)])\n        self.shared_experts = MLP(args.dim, args.n_shared_experts * args.moe_inter_dim)\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        \"\"\"\n        Forward pass for the MoE module.\n\n        Args:\n            x (torch.Tensor): Input tensor.\n\n        Returns:","sourceCodeStart":640,"sourceCodeEnd":676,"githubUrl":"https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L640-L676","documentation":"Thrown in MoE.__init__ (inference/model.py:658): routed experts are partitioned so each rank holds n_routed_experts // world_size local experts (the experts ModuleList uses None placeholders for non-local slots). n_routed_experts (256 for DeepSeek-V3) must be divisible by world_size. Assert fires at model construction.","triggerScenarios":"Constructing the MoE layer with world_size not dividing args.n_routed_experts — e.g. 3/5/6/7 GPUs against 256 experts, or a custom config with n_routed_experts like 7 launched on 2 ranks.","commonSituations":"Odd GPU counts; experimental configs that change n_routed_experts; also note this must match the --n-experts/--model-parallel values used when running convert.py to shard the checkpoint.","solutions":["Launch with a GPU count dividing n_routed_experts (256 divides cleanly by 2,4,8,16,32,64)","Keep the convert.py sharding (--model-parallel) consistent with the inference world_size","Validate the config programmatically before launch: assert args.n_routed_experts % world_size == 0"],"exampleFix":"# before\ntorchrun --nproc_per_node 6 generate.py ...  # 256 % 6 != 0\n\n# after\ntorchrun --nproc_per_node 8 generate.py --ckpt-path ... --config ... --input-file prompts.txt","handlingStrategy":"validation","validationCode":"world = dist.get_world_size() if dist.is_initialized() else 1\nassert args.n_routed_experts % world == 0, (\n    f\"n_routed_experts={args.n_routed_experts} not divisible by world_size={world}\"\n)","typeGuard":"def experts_shardable(n_experts: int, world_size: int) -> bool:\n    return n_experts % world_size == 0","tryCatchPattern":null,"preventionTips":["Keep convert.py --model-parallel equal to the inference world_size","For 256 routed experts use 2/4/8/16/32/64 GPUs","Add a startup assert so failure happens with a clear message, not deep in model construction"],"tags":["distributed","moe","experts","model-construction","deepseek"],"backgroundTag":null,"analyzedSha":"9b4e9788e4a3a731f7567338ed15d3ec549ce03b","analyzedAt":"2026-08-14T19:02:32.748Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}