deepseek-ai/DeepSeek-V3 · critical · AssertionError

Output features must be divisible by world size (world_size=

Error message

Output features must be divisible by world size (world_size=${world_size})

What it means

Thrown when constructing ColumnParallelLinear (inference/model.py:219), which shards a linear layer's OUTPUT features across ranks (e.g. wq, w1/w3 projections). Each rank computes out_features // world_size output columns, so out_features must be divisible by world_size. The assert fires during Transformer build, before checkpoint loading. Note the f-string interpolation of world_size is broken in this build, so the message may print the placeholder literally.

Source

Thrown at inference/model.py:219

        Returns:
            torch.Tensor: Transformed tensor after linear computation.
        """
        return linear(x, self.weight, self.bias, self.scale_fmt)


class ColumnParallelLinear(Linear):
    """
    Linear layer with column parallelism, splitting output features across distributed processes.

    Args:
        in_features (int): Number of input features.
        out_features (int): Total number of output features.
        bias (bool): Whether to include a bias term. Defaults to False.
        dtype (optional): Data type for the layer. Defaults to `torch.bfloat16`.
    """
    def __init__(self, in_features: int, out_features: int, bias: bool = False, dtype = None):
        assert out_features % world_size == 0, f"Output features must be divisible by world size (world_size={world_size})"
        self.part_out_features = out_features // world_size
        super().__init__(in_features, self.part_out_features, bias, dtype)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """
        Forward pass for column parallel linear layer.

        Args:
            x (torch.Tensor): Input tensor.

        Returns:
            torch.Tensor: Transformed tensor with column-parallel computation.
        """
        y = linear(x, self.weight, self.bias)
        return y


class RowParallelLinear(Linear):

View on GitHub (pinned to 9b4e9788e4)

Solutions

  1. Use a power-of-two GPU count (2, 4, 8, 16) — all DeepSeek-V3 column-parallel dims are divisible by these
  2. Verify your config: ensure qk_nope_head_dim + qk_rope_head_dim, moe_inter_dim etc. are divisible by world_size
  3. Check world_size at startup (print(rank, world_size)) to catch stale MASTER_ADDR/RANK env vars from a previous torchrun
  4. For debugging on one GPU, run the model with world_size=1 (no torchrun, plain python generate.py --interactive)

Example fix

# before
torchrun --nproc_per_node 6 generate.py ...  # 2048 % 6 != 0 for moe_inter_dim

# after
torchrun --nproc_per_node 8 generate.py --ckpt-path ... --config ... --interactive
Defensive patterns

Strategy: validation

Validate before calling

from model import ModelArgs
import torch.distributed as dist

world = dist.get_world_size() if dist.is_initialized() else 1
args = ModelArgs.from_json("configs/config_671b.json")
for name, d in [("qk dim", args.qk_nope_head_dim + args.qk_rope_head_dim), ("moe_inter", args.moe_inter_dim)]:
    assert d % world == 0, f"{name}={d} not divisible by world_size={world}"

Type guard

def col_parallel_ok(out_features: int, world_size: int) -> bool:
    return out_features % world_size == 0

Prevention

When it happens

Trigger: Building the model with world_size > 1 where a column-parallel dimension (q heads' hidden dim, MoE inter dim, etc.) is not divisible by the process count. Happens with odd GPU counts, or after changing hidden_dim / moe_inter_dim / n_heads in ModelArgs without keeping them divisible by world_size.

Common situations: Same class of mistake as error 0: non-power-of-two GPU counts (3, 5, 6, 7) against DeepSeek-V3 dims (7168, 2048 moe_inter_dim); custom configs that shrink moe_inter_dim for testing; running on a machine with leftover distributed env vars inflating world_size.

Related errors

AI-assisted analysis of deepseek-ai/DeepSeek-V3@9b4e9788e4 (2026-08-14). Data as JSON: /api/errors/6fcf94def864d55e. Report an issue: GitHub.