{"record":{"id":"6fcf94def864d55e","repo":"deepseek-ai/DeepSeek-V3","slug":"output-features-must-be-divisible-by-world-size-w","errorCode":null,"errorMessage":"Output features must be divisible by world size (world_size=${world_size})","messagePattern":"Output features must be divisible by world size \\(world_size=(.+?)\\)","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"critical","filePath":"inference/model.py","lineNumber":219,"sourceCode":"\n        Returns:\n            torch.Tensor: Transformed tensor after linear computation.\n        \"\"\"\n        return linear(x, self.weight, self.bias, self.scale_fmt)\n\n\nclass ColumnParallelLinear(Linear):\n    \"\"\"\n    Linear layer with column parallelism, splitting output features across distributed processes.\n\n    Args:\n        in_features (int): Number of input features.\n        out_features (int): Total number of output features.\n        bias (bool): Whether to include a bias term. Defaults to False.\n        dtype (optional): Data type for the layer. Defaults to `torch.bfloat16`.\n    \"\"\"\n    def __init__(self, in_features: int, out_features: int, bias: bool = False, dtype = None):\n        assert out_features % world_size == 0, f\"Output features must be divisible by world size (world_size={world_size})\"\n        self.part_out_features = out_features // world_size\n        super().__init__(in_features, self.part_out_features, bias, dtype)\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        \"\"\"\n        Forward pass for column parallel linear layer.\n\n        Args:\n            x (torch.Tensor): Input tensor.\n\n        Returns:\n            torch.Tensor: Transformed tensor with column-parallel computation.\n        \"\"\"\n        y = linear(x, self.weight, self.bias)\n        return y\n\n\nclass RowParallelLinear(Linear):","sourceCodeStart":201,"sourceCodeEnd":237,"githubUrl":"https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L201-L237","documentation":"Thrown when constructing ColumnParallelLinear (inference/model.py:219), which shards a linear layer's OUTPUT features across ranks (e.g. wq, w1/w3 projections). Each rank computes out_features // world_size output columns, so out_features must be divisible by world_size. The assert fires during Transformer build, before checkpoint loading. Note the f-string interpolation of world_size is broken in this build, so the message may print the placeholder literally.","triggerScenarios":"Building the model with world_size > 1 where a column-parallel dimension (q heads' hidden dim, MoE inter dim, etc.) is not divisible by the process count. Happens with odd GPU counts, or after changing hidden_dim / moe_inter_dim / n_heads in ModelArgs without keeping them divisible by world_size.","commonSituations":"Same class of mistake as error 0: non-power-of-two GPU counts (3, 5, 6, 7) against DeepSeek-V3 dims (7168, 2048 moe_inter_dim); custom configs that shrink moe_inter_dim for testing; running on a machine with leftover distributed env vars inflating world_size.","solutions":["Use a power-of-two GPU count (2, 4, 8, 16) — all DeepSeek-V3 column-parallel dims are divisible by these","Verify your config: ensure qk_nope_head_dim + qk_rope_head_dim, moe_inter_dim etc. are divisible by world_size","Check world_size at startup (print(rank, world_size)) to catch stale MASTER_ADDR/RANK env vars from a previous torchrun","For debugging on one GPU, run the model with world_size=1 (no torchrun, plain python generate.py --interactive)"],"exampleFix":"# before\ntorchrun --nproc_per_node 6 generate.py ...  # 2048 % 6 != 0 for moe_inter_dim\n\n# after\ntorchrun --nproc_per_node 8 generate.py --ckpt-path ... --config ... --interactive","handlingStrategy":"validation","validationCode":"from model import ModelArgs\nimport torch.distributed as dist\n\nworld = dist.get_world_size() if dist.is_initialized() else 1\nargs = ModelArgs.from_json(\"configs/config_671b.json\")\nfor name, d in [(\"qk dim\", args.qk_nope_head_dim + args.qk_rope_head_dim), (\"moe_inter\", args.moe_inter_dim)]:\n    assert d % world == 0, f\"{name}={d} not divisible by world_size={world}\"","typeGuard":"def col_parallel_ok(out_features: int, world_size: int) -> bool:\n    return out_features % world_size == 0","tryCatchPattern":null,"preventionTips":["Standardize on 2/4/8/16-way tensor parallelism","Validate all ModelArgs dims against world_size in a preflight script","Keep a CI smoke test that constructs the model at the intended world_size"],"tags":["distributed","tensor-parallelism","linear-layer","model-construction","deepseek"],"backgroundTag":null,"analyzedSha":"9b4e9788e4a3a731f7567338ed15d3ec549ce03b","analyzedAt":"2026-08-14T19:02:32.748Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}